2024-09-19T05:32:28.820 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T05:32:28.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:32:28.935 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691 branch: squid description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911691' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO - MDS_TRIM sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEzcFihS/AHhWrWTikY8iHDseziX16fT9Na0ygEssM+MJM5Mpp/4OmyjErb5VZionebpsAqC7hLjQ93TlC+hJrI= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO91wzOMFSawvXpgzm6IgCzlJj68/DkalEUifJvt5kdd8UZx5JkE8mdKGpyGkbWnmeUhujj/AsG/tozDFMsrs1Q= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - thrashosds: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T05:32:28.936 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T05:32:28.936 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T05:32:28.937 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T05:32:28.938 INFO:teuthology.task.internal:Checking packages... 2024-09-19T05:32:28.960 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T05:32:28.960 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T05:32:28.960 INFO:teuthology.packaging:ref: None 2024-09-19T05:32:28.960 INFO:teuthology.packaging:tag: None 2024-09-19T05:32:28.960 INFO:teuthology.packaging:branch: squid 2024-09-19T05:32:28.960 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:32:28.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T05:32:29.180 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T05:32:29.181 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T05:32:29.188 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T05:32:29.188 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T05:32:29.219 INFO:teuthology.task.internal:Saving configuration 2024-09-19T05:32:29.240 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T05:32:29.245 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T05:32:29.266 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:27:31.281786', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEzcFihS/AHhWrWTikY8iHDseziX16fT9Na0ygEssM+MJM5Mpp/4OmyjErb5VZionebpsAqC7hLjQ93TlC+hJrI='} 2024-09-19T05:32:29.285 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 05:27:31.280008', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO91wzOMFSawvXpgzm6IgCzlJj68/DkalEUifJvt5kdd8UZx5JkE8mdKGpyGkbWnmeUhujj/AsG/tozDFMsrs1Q='} 2024-09-19T05:32:29.285 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T05:32:29.290 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-19T05:32:29.291 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-19T05:32:29.291 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T05:32:29.366 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f417046f490>, signals=[15]) 2024-09-19T05:32:29.366 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T05:32:29.371 INFO:teuthology.task.internal:Opening connections... 2024-09-19T05:32:29.372 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:32:29.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:32:29.449 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:32:29.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:32:29.525 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T05:32:29.531 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-09-19T05:32:29.549 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-09-19T05:32:29.549 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-09-19T05:32:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:32:29.607 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:32:29.607 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:32:29.607 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:32:29.607 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:32:29.607 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:32:29.607 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-09-19T05:32:29.622 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-09-19T05:32:29.640 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-09-19T05:32:29.641 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-09-19T05:32:29.697 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-09-19T05:32:29.698 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-09-19T05:32:29.699 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T05:32:29.699 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-09-19T05:32:29.699 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T05:32:29.699 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T05:32:29.699 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T05:32:29.699 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-09-19T05:32:29.722 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T05:32:29.732 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T05:32:29.786 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T05:32:29.787 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:32:29.789 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-09-19T05:32:29.806 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T05:32:29.811 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T05:32:29.811 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:32:29.846 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-09-19T05:32:29.943 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T05:32:29.995 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T05:32:30.049 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T05:32:30.049 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T05:32:30.049 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T05:32:30.050 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:32:30.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:32:30.067 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-19T05:32:30.125 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-503.el9.x86_64 2024-09-19T05:32:30.126 INFO:teuthology.task.kernel:Running kernel on smithi002: 5.14.0-503.el9.x86_64 2024-09-19T05:32:30.126 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2024-09-19T05:32:32.421 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.3 MB 00:00 2024-09-19T05:32:34.497 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-19T05:32:38.822 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-19T05:32:40.298 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-19T05:32:41.185 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-19T05:32:46.858 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-19T05:32:48.823 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T05:32:48.824 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T05:32:48.907 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-19T05:32:48.914 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-19T05:32:48.914 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-09-19T05:32:48.914 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-19T05:32:48.914 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-19T05:32:48.914 INFO:teuthology.orchestra.run.smithi002.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T05:32:48.914 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-19T05:32:48.914 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T05:32:48.915 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T05:32:48.915 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T05:32:48.915 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:32:48.915 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-19T05:32:48.915 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-19T05:32:48.915 INFO:teuthology.orchestra.run.smithi002.stdout:Install 4 Packages 2024-09-19T05:32:48.915 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:32:48.916 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 79 M 2024-09-19T05:32:48.916 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 126 M 2024-09-19T05:32:48.916 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-19T05:32:49.060 INFO:teuthology.orchestra.run.smithi002.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 372 kB/s | 36 kB 00:00 2024-09-19T05:32:49.393 INFO:teuthology.orchestra.run.smithi002.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-19T05:32:49.693 INFO:teuthology.orchestra.run.smithi002.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 45 MB/s | 29 MB 00:00 2024-09-19T05:32:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 24 MB/s | 35 MB 00:01 2024-09-19T05:32:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:32:50.410 INFO:teuthology.orchestra.run.smithi002.stdout:Total 53 MB/s | 79 MB 00:01 2024-09-19T05:32:51.125 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-19T05:32:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-19T05:32:51.506 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-19T05:32:52.491 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-19T05:32:52.757 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-19T05:32:55.035 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-19T05:32:55.705 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:32:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:32:58.558 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:32:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:33:03.800 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:33:03.839 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:33:08.981 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:33:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:33:48.076 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:33:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:33:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T05:33:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T05:33:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T05:33:49.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T05:33:49.372 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:33:49.372 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-19T05:33:49.372 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T05:33:49.373 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T05:33:49.373 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T05:33:49.373 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T05:33:49.373 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:33:49.373 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-19T05:33:49.697 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:33:49.782 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:33:49.783 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:33:49.783 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T05:33:49.783 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T05:33:49.783 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T05:33:49.783 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T05:33:49.783 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T05:33:49.784 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T05:33:49.842 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T05:33:49.842 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T05:33:49.842 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2024-09-19T05:33:49.876 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2024-09-19T05:33:49.876 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2024-09-19T05:33:49.876 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.06 2024-09-19T05:33:49.877 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 82.el9 2024-09-19T05:33:49.877 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2024-09-19T05:33:49.877 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T05:33:49.877 INFO:teuthology.orchestra.run.smithi002.stdout:Group : Unspecified 2024-09-19T05:33:49.877 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 8274670 2024-09-19T05:33:49.877 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : builder@centos.org 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : CentOS 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T05:33:49.878 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T05:33:49.879 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2024-09-19T05:33:49.879 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:33:49.879 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T05:33:49.880 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T05:33:49.880 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 5.14.0-508.el9.x86_64 2024-09-19T05:33:49.880 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T05:33:50.571 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-09-19T05:33:51.898 INFO:teuthology.orchestra.run.smithi002.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T05:33:51.913 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-09-19T05:33:51.915 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-19T05:33:51.932 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.z3tYJOHtbL 2024-09-19T05:33:51.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.z3tYJOHtbL 2024-09-19T05:33:51.999 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.z3tYJOHtbL 2024-09-19T05:33:52.135 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.z3tYJOHtbL is 6KB 2024-09-19T05:33:52.184 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.z3tYJOHtbL 2024-09-19T05:33:52.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T05:33:52.261 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T05:33:52.261 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T05:33:52.261 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T05:33:52.262 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T05:33:52.263 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T05:33:52.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-09-19T05:33:52.515 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-09-19T05:34:22.442 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T05:34:22.443 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-19T05:34:22.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:34:40.957 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-09-19T05:34:49.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-19T05:34:49.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:34:53.021 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-09-19T05:35:05.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-19T05:35:05.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:35:36.856 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-19T05:35:37.267 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-19T05:35:37.268 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T05:35:37.268 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-19T05:35:37.283 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-508.el9.x86_64 2024-09-19T05:35:37.283 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T05:35:37.283 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T05:35:37.283 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T05:35:38.284 DEBUG:teuthology.parallel:result is None 2024-09-19T05:35:38.285 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T05:35:38.297 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T05:35:38.298 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:35:38.301 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T05:35:38.353 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T05:35:38.364 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T05:35:38.394 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T05:35:38.395 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:35:38.397 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T05:35:38.506 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T05:35:38.517 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T05:35:38.517 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T05:35:38.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:35:38.535 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:35:38.537 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T05:35:38.622 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:35:38.639 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:35:38.644 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:35:38.695 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T05:35:38.697 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T05:35:38.709 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T05:35:38.709 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:35:38.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T05:35:38.767 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T05:35:38.785 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T05:35:38.785 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:35:38.788 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T05:35:38.834 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:35:38.913 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:35:38.991 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:35:39.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:35:39.118 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:35:39.118 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:35:39.144 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:35:39.189 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T05:35:39.238 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:35:39.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T05:35:39.364 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:35:39.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T05:35:39.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-09-19T05:35:39.433 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-09-19T05:35:39.510 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:35:39.632 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:35:39.981 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T05:35:39.993 INFO:teuthology.task.internal:Starting timer... 2024-09-19T05:35:39.993 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T05:35:40.048 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T05:35:40.088 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T05:35:40.088 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2024-09-19T05:35:40.139 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2024-09-19T05:35:40.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-09-19T05:35:40.213 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-09-19T05:35:40.215 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T05:35:40.215 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2024-09-19T05:35:40.238 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2024-09-19T05:35:40.238 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-09-19T05:35:40.258 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-09-19T05:35:40.259 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2024-09-19T05:35:40.259 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:35:40.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:35:40.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:35:40.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:35:40.332 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T05:35:40.332 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:35:40.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-09-19T05:35:40.412 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T05:35:40.425 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T05:35:40.438 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T05:35:40.439 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T05:39:36.359 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-09-19T05:39:36.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-19T05:39:36.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:39:36.439 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-19T05:39:36.520 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-19T05:39:36.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T05:39:36.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T05:39:36.599 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-19T05:39:36.684 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-19T05:39:36.684 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T05:39:36.695 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T05:39:36.695 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:39:36.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:39:36.698 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T05:39:36.698 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:39:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:39:36.740 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:39:36.753 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T05:39:36.765 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2024-09-19T05:39:36.767 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T05:39:36.776 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2024-09-19T05:39:36.791 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:39:36.792 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-09-19T05:39:36.803 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-09-19T05:39:36.806 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:39:36.816 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T05:39:36.830 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T05:39:36.866 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:39:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:39:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-09-19T05:39:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.869 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.890 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:39:36.894 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:39:36.894 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-19T05:39:36.894 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.894 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.895 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.895 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:39:36.895 INFO:teuthology.run_tasks:Running task install... 2024-09-19T05:39:36.903 DEBUG:teuthology.task.install:project ceph 2024-09-19T05:39:36.903 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T05:39:36.903 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T05:39:36.903 INFO:teuthology.task.install:Using flavor: default 2024-09-19T05:39:36.908 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T05:39:36.909 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T05:39:36.909 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:39:36.910 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:39:36.911 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T05:39:36.912 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:39:37.055 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:39:37.055 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:39:37.062 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T05:39:37.062 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T05:39:37.167 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:39:37.168 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:39:37.168 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:39:37.177 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T05:39:37.177 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:39:37.177 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T05:39:37.194 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:39:37.194 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:39:37.203 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T05:39:37.203 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T05:39:37.260 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:39:37.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T05:39:37.336 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:39:37.343 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T05:39:37.402 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2024-09-19T05:39:37.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2024-09-19T05:39:37.407 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-09-19T05:39:37.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-09-19T05:39:37.633 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-09-19T05:39:37.650 INFO:teuthology.orchestra.run.smithi002.stdout:45 files removed 2024-09-19T05:39:37.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:39:37.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T05:39:38.184 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 333 kB/s | 84 kB 00:00 2024-09-19T05:39:38.254 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-19T05:39:38.452 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-19T05:39:38.546 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T05:39:38.711 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-19T05:39:38.830 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-19T05:39:39.313 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.3 MB 00:00 2024-09-19T05:39:40.585 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.3 MB 00:01 2024-09-19T05:39:41.081 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-19T05:39:42.362 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-19T05:39:46.274 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-19T05:39:46.587 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-19T05:39:47.737 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 102 kB/s | 19 kB 00:00 2024-09-19T05:39:48.288 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-19T05:39:48.621 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-19T05:39:49.259 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-19T05:39:54.402 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-19T05:39:54.798 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-19T05:39:56.729 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:39:56.730 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:39:56.746 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:39:56.747 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:39:56.747 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:39:56.749 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:39:56.750 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:39:56.753 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:39:56.754 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:39:56.820 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-19T05:39:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-19T05:39:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-19T05:39:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:39:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:39:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:39:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:39:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:39:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:39:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:39:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:39:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:39:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:39:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:39:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:39:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:39:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:Installing weak dependencies: 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:Install 148 Packages 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 8 Packages 2024-09-19T05:39:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:39:56.858 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 226 M 2024-09-19T05:39:56.858 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-19T05:39:56.927 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:39:56.928 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T05:39:56.950 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T05:39:56.950 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T05:39:56.951 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T05:39:56.952 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T05:39:56.953 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T05:39:56.953 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T05:39:56.953 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T05:39:57.007 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T05:39:57.041 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T05:39:57.042 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T05:39:57.043 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T05:39:57.044 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T05:39:57.045 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T05:39:57.046 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T05:39:57.047 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T05:39:57.048 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T05:39:57.049 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T05:39:57.050 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T05:39:57.051 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T05:39:57.052 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 8 Packages 2024-09-19T05:39:57.053 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:39:57.054 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 226 M 2024-09-19T05:39:57.054 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-19T05:39:57.427 INFO:teuthology.orchestra.run.smithi002.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 56 kB/s | 6.6 kB 00:00 2024-09-19T05:39:57.622 INFO:teuthology.orchestra.run.smithi002.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.5 MB/s | 896 kB 00:00 2024-09-19T05:39:57.705 INFO:teuthology.orchestra.run.smithi002.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.7 MB/s | 145 kB 00:00 2024-09-19T05:39:57.774 INFO:teuthology.orchestra.run.smithi123.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 42 kB/s | 6.6 kB 00:00 2024-09-19T05:39:57.789 INFO:teuthology.orchestra.run.smithi002.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T05:39:57.881 INFO:teuthology.orchestra.run.smithi002.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-19T05:39:57.931 INFO:teuthology.orchestra.run.smithi002.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-19T05:39:58.190 INFO:teuthology.orchestra.run.smithi002.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-19T05:39:58.357 INFO:teuthology.orchestra.run.smithi123.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 1.5 MB/s | 896 kB 00:00 2024-09-19T05:39:58.424 INFO:teuthology.orchestra.run.smithi123.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T05:39:58.540 INFO:teuthology.orchestra.run.smithi002.stdout:(8/156): ceph-common-19.1.1-361.g2f599a46.el9.x 17 MB/s | 21 MB 00:01 2024-09-19T05:39:58.590 INFO:teuthology.orchestra.run.smithi002.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 507 kB/s | 25 kB 00:00 2024-09-19T05:39:58.775 INFO:teuthology.orchestra.run.smithi123.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 4.7 MB/s | 5.4 MB 00:01 2024-09-19T05:39:59.050 INFO:teuthology.orchestra.run.smithi123.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 5.5 MB/s | 1.5 MB 00:00 2024-09-19T05:39:59.109 INFO:teuthology.orchestra.run.smithi123.stdout:(6/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.5 MB/s | 2.4 MB 00:00 2024-09-19T05:39:59.201 INFO:teuthology.orchestra.run.smithi002.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 17 MB 00:01 2024-09-19T05:39:59.274 INFO:teuthology.orchestra.run.smithi002.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 3.0 MB/s | 228 kB 00:00 2024-09-19T05:39:59.366 INFO:teuthology.orchestra.run.smithi002.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 295 kB/s | 27 kB 00:00 2024-09-19T05:39:59.458 INFO:teuthology.orchestra.run.smithi002.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 346 kB/s | 32 kB 00:00 2024-09-19T05:39:59.459 INFO:teuthology.orchestra.run.smithi123.stdout:(7/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-19T05:39:59.609 INFO:teuthology.orchestra.run.smithi002.stdout:(14/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 7.5 MB/s | 11 MB 00:01 2024-09-19T05:39:59.650 INFO:teuthology.orchestra.run.smithi002.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 3.8 MB/s | 751 kB 00:00 2024-09-19T05:39:59.700 INFO:teuthology.orchestra.run.smithi002.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 1.7 MB/s | 163 kB 00:00 2024-09-19T05:39:59.734 INFO:teuthology.orchestra.run.smithi002.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 1.5 MB/s | 127 kB 00:00 2024-09-19T05:39:59.793 INFO:teuthology.orchestra.run.smithi002.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 5.4 MB/s | 506 kB 00:00 2024-09-19T05:39:59.893 INFO:teuthology.orchestra.run.smithi002.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 451 kB/s | 45 kB 00:00 2024-09-19T05:39:59.993 INFO:teuthology.orchestra.run.smithi002.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.3 MB/s | 136 kB 00:00 2024-09-19T05:40:00.093 INFO:teuthology.orchestra.run.smithi002.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.6 MB/s | 161 kB 00:00 2024-09-19T05:40:00.202 INFO:teuthology.orchestra.run.smithi002.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 2.9 MB/s | 320 kB 00:00 2024-09-19T05:40:00.302 INFO:teuthology.orchestra.run.smithi002.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.9 MB/s | 300 kB 00:00 2024-09-19T05:40:00.411 INFO:teuthology.orchestra.run.smithi002.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 917 kB/s | 99 kB 00:00 2024-09-19T05:40:00.486 INFO:teuthology.orchestra.run.smithi002.stdout:(25/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 7.1 MB/s | 5.3 MB 00:00 2024-09-19T05:40:00.511 INFO:teuthology.orchestra.run.smithi002.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 850 kB/s | 85 kB 00:00 2024-09-19T05:40:00.517 INFO:teuthology.orchestra.run.smithi123.stdout:(8/156): ceph-radosgw-19.1.1-361.g2f599a46.el9. 10 MB/s | 11 MB 00:01 2024-09-19T05:40:00.578 INFO:teuthology.orchestra.run.smithi002.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-19T05:40:00.617 INFO:teuthology.orchestra.run.smithi123.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 252 kB/s | 25 kB 00:00 2024-09-19T05:40:00.670 INFO:teuthology.orchestra.run.smithi002.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 347 kB/s | 32 kB 00:00 2024-09-19T05:40:00.976 INFO:teuthology.orchestra.run.smithi123.stdout:(10/156): ceph-common-19.1.1-361.g2f599a46.el9. 6.3 MB/s | 21 MB 00:03 2024-09-19T05:40:01.151 INFO:teuthology.orchestra.run.smithi123.stdout:(11/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 8.3 MB/s | 17 MB 00:02 2024-09-19T05:40:01.154 INFO:teuthology.orchestra.run.smithi002.stdout:(29/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 19 MB/s | 49 MB 00:02 2024-09-19T05:40:01.177 INFO:teuthology.orchestra.run.smithi123.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.1 MB/s | 228 kB 00:00 2024-09-19T05:40:01.188 INFO:teuthology.orchestra.run.smithi002.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 283 kB/s | 146 kB 00:00 2024-09-19T05:40:01.201 INFO:teuthology.orchestra.run.smithi123.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 541 kB/s | 27 kB 00:00 2024-09-19T05:40:01.227 INFO:teuthology.orchestra.run.smithi123.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 633 kB/s | 32 kB 00:00 2024-09-19T05:40:01.263 INFO:teuthology.orchestra.run.smithi002.stdout:(31/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 4.0 MB/s | 3.1 MB 00:00 2024-09-19T05:40:01.277 INFO:teuthology.orchestra.run.smithi123.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-19T05:40:01.302 INFO:teuthology.orchestra.run.smithi123.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T05:40:01.327 INFO:teuthology.orchestra.run.smithi123.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T05:40:01.330 INFO:teuthology.orchestra.run.smithi002.stdout:(32/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 20 MB/s | 3.6 MB 00:00 2024-09-19T05:40:01.353 INFO:teuthology.orchestra.run.smithi123.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 9.9 MB/s | 506 kB 00:00 2024-09-19T05:40:01.355 INFO:teuthology.orchestra.run.smithi002.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 2.7 MB/s | 249 kB 00:00 2024-09-19T05:40:01.397 INFO:teuthology.orchestra.run.smithi002.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 740 kB/s | 49 kB 00:00 2024-09-19T05:40:01.403 INFO:teuthology.orchestra.run.smithi123.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 900 kB/s | 45 kB 00:00 2024-09-19T05:40:01.423 INFO:teuthology.orchestra.run.smithi002.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 246 kB/s | 16 kB 00:00 2024-09-19T05:40:01.465 INFO:teuthology.orchestra.run.smithi002.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.2 MB/s | 285 kB 00:00 2024-09-19T05:40:01.478 INFO:teuthology.orchestra.run.smithi123.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 1.8 MB/s | 136 kB 00:00 2024-09-19T05:40:01.540 INFO:teuthology.orchestra.run.smithi002.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 243 kB/s | 18 kB 00:00 2024-09-19T05:40:01.554 INFO:teuthology.orchestra.run.smithi123.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 2.1 MB/s | 161 kB 00:00 2024-09-19T05:40:01.574 INFO:teuthology.orchestra.run.smithi002.stdout:(38/156): cephadm-19.1.1-361.g2f599a46.el9.noar 5.0 MB/s | 765 kB 00:00 2024-09-19T05:40:01.629 INFO:teuthology.orchestra.run.smithi123.stdout:(22/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-19T05:40:01.654 INFO:teuthology.orchestra.run.smithi123.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 3.1 MB/s | 320 kB 00:00 2024-09-19T05:40:01.679 INFO:teuthology.orchestra.run.smithi123.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.9 MB/s | 300 kB 00:00 2024-09-19T05:40:01.682 INFO:teuthology.orchestra.run.smithi002.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-19T05:40:01.696 INFO:teuthology.orchestra.run.smithi123.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 2.3 MB/s | 99 kB 00:00 2024-09-19T05:40:01.730 INFO:teuthology.orchestra.run.smithi123.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T05:40:01.774 INFO:teuthology.orchestra.run.smithi002.stdout:(40/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-19T05:40:01.789 INFO:teuthology.orchestra.run.smithi123.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-19T05:40:01.799 INFO:teuthology.orchestra.run.smithi002.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-19T05:40:01.847 INFO:teuthology.orchestra.run.smithi123.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 547 kB/s | 32 kB 00:00 2024-09-19T05:40:01.883 INFO:teuthology.orchestra.run.smithi002.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-19T05:40:01.906 INFO:teuthology.orchestra.run.smithi123.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-19T05:40:01.931 INFO:teuthology.orchestra.run.smithi123.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.7 MB/s | 146 kB 00:00 2024-09-19T05:40:01.933 INFO:teuthology.orchestra.run.smithi002.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-19T05:40:01.959 INFO:teuthology.orchestra.run.smithi002.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-19T05:40:01.984 INFO:teuthology.orchestra.run.smithi002.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-19T05:40:02.009 INFO:teuthology.orchestra.run.smithi002.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T05:40:02.043 INFO:teuthology.orchestra.run.smithi002.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 5.0 MB/s | 1.2 MB 00:00 2024-09-19T05:40:02.068 INFO:teuthology.orchestra.run.smithi002.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-19T05:40:02.157 INFO:teuthology.orchestra.run.smithi123.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 14 MB/s | 3.6 MB 00:00 2024-09-19T05:40:02.215 INFO:teuthology.orchestra.run.smithi123.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.2 MB/s | 249 kB 00:00 2024-09-19T05:40:02.299 INFO:teuthology.orchestra.run.smithi123.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-19T05:40:02.315 INFO:teuthology.orchestra.run.smithi123.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 493 kB/s | 49 kB 00:00 2024-09-19T05:40:02.333 INFO:teuthology.orchestra.run.smithi123.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 489 kB/s | 16 kB 00:00 2024-09-19T05:40:02.366 INFO:teuthology.orchestra.run.smithi123.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.5 MB/s | 285 kB 00:00 2024-09-19T05:40:02.400 INFO:teuthology.orchestra.run.smithi123.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T05:40:02.425 INFO:teuthology.orchestra.run.smithi123.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 311 kB/s | 18 kB 00:00 2024-09-19T05:40:02.527 INFO:teuthology.orchestra.run.smithi002.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-19T05:40:02.567 INFO:teuthology.orchestra.run.smithi123.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-19T05:40:02.643 INFO:teuthology.orchestra.run.smithi002.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-19T05:40:02.651 INFO:teuthology.orchestra.run.smithi123.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-19T05:40:02.684 INFO:teuthology.orchestra.run.smithi123.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.0 MB/s | 801 kB 00:00 2024-09-19T05:40:02.709 INFO:teuthology.orchestra.run.smithi123.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-19T05:40:02.768 INFO:teuthology.orchestra.run.smithi123.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-19T05:40:02.785 INFO:teuthology.orchestra.run.smithi002.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 888 kB/s | 689 kB 00:00 2024-09-19T05:40:02.827 INFO:teuthology.orchestra.run.smithi123.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-09-19T05:40:02.852 INFO:teuthology.orchestra.run.smithi123.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-19T05:40:02.869 INFO:teuthology.orchestra.run.smithi002.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-19T05:40:02.877 INFO:teuthology.orchestra.run.smithi123.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T05:40:02.894 INFO:teuthology.orchestra.run.smithi002.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-19T05:40:02.902 INFO:teuthology.orchestra.run.smithi123.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-19T05:40:03.003 INFO:teuthology.orchestra.run.smithi002.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-19T05:40:03.028 INFO:teuthology.orchestra.run.smithi002.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 816 kB/s | 313 kB 00:00 2024-09-19T05:40:03.052 INFO:teuthology.orchestra.run.smithi123.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-09-19T05:40:03.103 INFO:teuthology.orchestra.run.smithi002.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-19T05:40:03.111 INFO:teuthology.orchestra.run.smithi123.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-19T05:40:03.136 INFO:teuthology.orchestra.run.smithi123.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-19T05:40:03.137 INFO:teuthology.orchestra.run.smithi002.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-19T05:40:03.161 INFO:teuthology.orchestra.run.smithi123.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 430 kB/s | 22 kB 00:00 2024-09-19T05:40:03.203 INFO:teuthology.orchestra.run.smithi123.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 790 kB/s | 33 kB 00:00 2024-09-19T05:40:03.204 INFO:teuthology.orchestra.run.smithi002.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-09-19T05:40:03.254 INFO:teuthology.orchestra.run.smithi123.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-19T05:40:03.295 INFO:teuthology.orchestra.run.smithi123.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-19T05:40:03.338 INFO:teuthology.orchestra.run.smithi123.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-19T05:40:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-09-19T05:40:03.413 INFO:teuthology.orchestra.run.smithi123.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-19T05:40:03.421 INFO:teuthology.orchestra.run.smithi002.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-19T05:40:03.455 INFO:teuthology.orchestra.run.smithi123.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-19T05:40:03.455 INFO:teuthology.orchestra.run.smithi002.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-09-19T05:40:03.480 INFO:teuthology.orchestra.run.smithi002.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 224 kB/s | 24 kB 00:00 2024-09-19T05:40:03.531 INFO:teuthology.orchestra.run.smithi002.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 434 kB/s | 47 kB 00:00 2024-09-19T05:40:03.572 INFO:teuthology.orchestra.run.smithi002.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-19T05:40:03.598 INFO:teuthology.orchestra.run.smithi002.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-19T05:40:03.640 INFO:teuthology.orchestra.run.smithi002.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-19T05:40:03.647 INFO:teuthology.orchestra.run.smithi123.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-19T05:40:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-09-19T05:40:04.007 INFO:teuthology.orchestra.run.smithi002.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-19T05:40:04.055 INFO:teuthology.orchestra.run.smithi123.stdout:(59/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 14 MB/s | 49 MB 00:03 2024-09-19T05:40:04.089 INFO:teuthology.orchestra.run.smithi123.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 1.3 MB/s | 581 kB 00:00 2024-09-19T05:40:04.123 INFO:teuthology.orchestra.run.smithi002.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-19T05:40:04.139 INFO:teuthology.orchestra.run.smithi123.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-09-19T05:40:04.164 INFO:teuthology.orchestra.run.smithi123.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-19T05:40:04.190 INFO:teuthology.orchestra.run.smithi123.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-19T05:40:04.215 INFO:teuthology.orchestra.run.smithi123.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-19T05:40:04.267 INFO:teuthology.orchestra.run.smithi123.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-19T05:40:04.290 INFO:teuthology.orchestra.run.smithi123.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-19T05:40:04.624 INFO:teuthology.orchestra.run.smithi123.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.8 MB/s | 1.6 MB 00:00 2024-09-19T05:40:04.682 INFO:teuthology.orchestra.run.smithi002.stdout:(70/156): postgresql-server-13.16-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-09-19T05:40:04.832 INFO:teuthology.orchestra.run.smithi002.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-09-19T05:40:04.891 INFO:teuthology.orchestra.run.smithi123.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-19T05:40:04.941 INFO:teuthology.orchestra.run.smithi002.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 808 kB/s | 87 kB 00:00 2024-09-19T05:40:04.958 INFO:teuthology.orchestra.run.smithi123.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-19T05:40:05.433 INFO:teuthology.orchestra.run.smithi123.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-19T05:40:05.525 INFO:teuthology.orchestra.run.smithi002.stdout:(73/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 4.4 MB 00:01 2024-09-19T05:40:05.641 INFO:teuthology.orchestra.run.smithi002.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 526 kB/s | 61 kB 00:00 2024-09-19T05:40:05.683 INFO:teuthology.orchestra.run.smithi123.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-19T05:40:05.742 INFO:teuthology.orchestra.run.smithi123.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T05:40:05.784 INFO:teuthology.orchestra.run.smithi123.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-19T05:40:05.818 INFO:teuthology.orchestra.run.smithi123.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-19T05:40:05.909 INFO:teuthology.orchestra.run.smithi123.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-19T05:40:05.993 INFO:teuthology.orchestra.run.smithi123.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T05:40:06.035 INFO:teuthology.orchestra.run.smithi123.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-19T05:40:06.067 INFO:teuthology.orchestra.run.smithi002.stdout:(75/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.0 MB/s | 4.9 MB 00:02 2024-09-19T05:40:06.077 INFO:teuthology.orchestra.run.smithi123.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-19T05:40:06.100 INFO:teuthology.orchestra.run.smithi002.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 534 kB/s | 245 kB 00:00 2024-09-19T05:40:06.144 INFO:teuthology.orchestra.run.smithi123.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-19T05:40:06.186 INFO:teuthology.orchestra.run.smithi123.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 837 kB/s | 35 kB 00:00 2024-09-19T05:40:06.209 INFO:teuthology.orchestra.run.smithi002.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-09-19T05:40:06.317 INFO:teuthology.orchestra.run.smithi002.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-09-19T05:40:06.443 INFO:teuthology.orchestra.run.smithi002.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 661 kB/s | 248 kB 00:00 2024-09-19T05:40:06.469 INFO:teuthology.orchestra.run.smithi123.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.8 MB/s | 5.8 MB 00:01 2024-09-19T05:40:06.551 INFO:teuthology.orchestra.run.smithi002.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-09-19T05:40:06.561 INFO:teuthology.orchestra.run.smithi123.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-19T05:40:06.602 INFO:teuthology.orchestra.run.smithi002.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 606 kB/s | 172 kB 00:00 2024-09-19T05:40:06.645 INFO:teuthology.orchestra.run.smithi123.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-19T05:40:06.703 INFO:teuthology.orchestra.run.smithi123.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T05:40:06.745 INFO:teuthology.orchestra.run.smithi123.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-19T05:40:06.846 INFO:teuthology.orchestra.run.smithi123.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-19T05:40:06.921 INFO:teuthology.orchestra.run.smithi123.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-19T05:40:06.977 INFO:teuthology.orchestra.run.smithi002.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 668 kB/s | 250 kB 00:00 2024-09-19T05:40:07.021 INFO:teuthology.orchestra.run.smithi123.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-19T05:40:07.063 INFO:teuthology.orchestra.run.smithi123.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-19T05:40:07.105 INFO:teuthology.orchestra.run.smithi123.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-19T05:40:07.352 INFO:teuthology.orchestra.run.smithi002.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 592 kB/s | 222 kB 00:00 2024-09-19T05:40:07.427 INFO:teuthology.orchestra.run.smithi002.stdout:(84/156): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-09-19T05:40:07.536 INFO:teuthology.orchestra.run.smithi002.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-09-19T05:40:07.556 INFO:teuthology.orchestra.run.smithi123.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-09-19T05:40:07.561 INFO:teuthology.orchestra.run.smithi002.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 369 kB/s | 77 kB 00:00 2024-09-19T05:40:07.597 INFO:teuthology.orchestra.run.smithi123.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-19T05:40:07.631 INFO:teuthology.orchestra.run.smithi123.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-19T05:40:07.723 INFO:teuthology.orchestra.run.smithi123.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-09-19T05:40:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-09-19T05:40:07.835 INFO:teuthology.orchestra.run.smithi123.stdout:(96/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-19T05:40:07.853 INFO:teuthology.orchestra.run.smithi002.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 544 kB/s | 159 kB 00:00 2024-09-19T05:40:07.857 INFO:teuthology.orchestra.run.smithi123.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-19T05:40:07.899 INFO:teuthology.orchestra.run.smithi123.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-19T05:40:07.957 INFO:teuthology.orchestra.run.smithi123.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-19T05:40:07.983 INFO:teuthology.orchestra.run.smithi123.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-19T05:40:07.995 INFO:teuthology.orchestra.run.smithi002.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 587 kB/s | 269 kB 00:00 2024-09-19T05:40:08.024 INFO:teuthology.orchestra.run.smithi123.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-09-19T05:40:08.050 INFO:teuthology.orchestra.run.smithi123.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-09-19T05:40:08.120 INFO:teuthology.orchestra.run.smithi002.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 408 kB/s | 51 kB 00:00 2024-09-19T05:40:08.142 INFO:teuthology.orchestra.run.smithi123.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-19T05:40:08.208 INFO:teuthology.orchestra.run.smithi123.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-19T05:40:08.229 INFO:teuthology.orchestra.run.smithi002.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-09-19T05:40:08.242 INFO:teuthology.orchestra.run.smithi123.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 8.9 MB/s | 308 kB 00:00 2024-09-19T05:40:08.268 INFO:teuthology.orchestra.run.smithi123.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-09-19T05:40:08.322 INFO:teuthology.orchestra.run.smithi002.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 596 kB/s | 279 kB 00:00 2024-09-19T05:40:08.322 INFO:teuthology.orchestra.run.smithi123.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-09-19T05:40:08.352 INFO:teuthology.orchestra.run.smithi123.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-09-19T05:40:08.385 INFO:teuthology.orchestra.run.smithi123.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T05:40:08.419 INFO:teuthology.orchestra.run.smithi123.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-19T05:40:08.435 INFO:teuthology.orchestra.run.smithi002.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-09-19T05:40:08.453 INFO:teuthology.orchestra.run.smithi123.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-19T05:40:08.539 INFO:teuthology.orchestra.run.smithi002.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-09-19T05:40:08.553 INFO:teuthology.orchestra.run.smithi123.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-19T05:40:08.595 INFO:teuthology.orchestra.run.smithi123.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-09-19T05:40:08.620 INFO:teuthology.orchestra.run.smithi123.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T05:40:08.646 INFO:teuthology.orchestra.run.smithi123.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T05:40:08.671 INFO:teuthology.orchestra.run.smithi123.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-19T05:40:08.697 INFO:teuthology.orchestra.run.smithi123.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-09-19T05:40:08.722 INFO:teuthology.orchestra.run.smithi123.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-19T05:40:08.755 INFO:teuthology.orchestra.run.smithi123.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-19T05:40:08.780 INFO:teuthology.orchestra.run.smithi123.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T05:40:08.806 INFO:teuthology.orchestra.run.smithi123.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T05:40:08.831 INFO:teuthology.orchestra.run.smithi123.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T05:40:08.856 INFO:teuthology.orchestra.run.smithi123.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-19T05:40:08.881 INFO:teuthology.orchestra.run.smithi123.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-19T05:40:08.907 INFO:teuthology.orchestra.run.smithi123.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-19T05:40:08.932 INFO:teuthology.orchestra.run.smithi123.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 787 kB/s | 19 kB 00:00 2024-09-19T05:40:08.957 INFO:teuthology.orchestra.run.smithi123.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-19T05:40:08.972 INFO:teuthology.orchestra.run.smithi002.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 509 kB/s | 221 kB 00:00 2024-09-19T05:40:09.018 INFO:teuthology.orchestra.run.smithi123.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-19T05:40:09.041 INFO:teuthology.orchestra.run.smithi123.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T05:40:09.066 INFO:teuthology.orchestra.run.smithi123.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T05:40:09.089 INFO:teuthology.orchestra.run.smithi002.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-19T05:40:09.091 INFO:teuthology.orchestra.run.smithi123.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T05:40:09.125 INFO:teuthology.orchestra.run.smithi123.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-19T05:40:09.150 INFO:teuthology.orchestra.run.smithi123.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-19T05:40:09.176 INFO:teuthology.orchestra.run.smithi123.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-19T05:40:09.193 INFO:teuthology.orchestra.run.smithi123.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.6 MB/s | 28 kB 00:00 2024-09-19T05:40:09.218 INFO:teuthology.orchestra.run.smithi123.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-19T05:40:09.222 INFO:teuthology.orchestra.run.smithi002.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 355 kB/s | 47 kB 00:00 2024-09-19T05:40:09.235 INFO:teuthology.orchestra.run.smithi123.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-09-19T05:40:09.252 INFO:teuthology.orchestra.run.smithi123.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-19T05:40:09.277 INFO:teuthology.orchestra.run.smithi123.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-19T05:40:09.302 INFO:teuthology.orchestra.run.smithi123.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 717 kB/s | 36 kB 00:00 2024-09-19T05:40:09.328 INFO:teuthology.orchestra.run.smithi123.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-19T05:40:09.353 INFO:teuthology.orchestra.run.smithi123.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T05:40:09.378 INFO:teuthology.orchestra.run.smithi123.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T05:40:09.403 INFO:teuthology.orchestra.run.smithi123.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-19T05:40:09.420 INFO:teuthology.orchestra.run.smithi123.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 11 MB/s | 191 kB 00:00 2024-09-19T05:40:09.445 INFO:teuthology.orchestra.run.smithi123.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-09-19T05:40:09.521 INFO:teuthology.orchestra.run.smithi123.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-19T05:40:09.689 INFO:teuthology.orchestra.run.smithi002.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 652 kB/s | 304 kB 00:00 2024-09-19T05:40:09.738 INFO:teuthology.orchestra.run.smithi123.stdout:(148/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T05:40:09.806 INFO:teuthology.orchestra.run.smithi002.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 496 kB/s | 58 kB 00:00 2024-09-19T05:40:09.838 INFO:teuthology.orchestra.run.smithi123.stdout:(149/156): librbd1-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-19T05:40:10.031 INFO:teuthology.orchestra.run.smithi002.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 426 kB/s | 96 kB 00:00 2024-09-19T05:40:10.197 INFO:teuthology.orchestra.run.smithi123.stdout:(150/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:03 2024-09-19T05:40:10.232 INFO:teuthology.orchestra.run.smithi002.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-19T05:40:10.246 INFO:teuthology.orchestra.run.smithi123.stdout:(151/156): NetworkManager-1.51.0-1.el9.x86_64.r 4.6 MB/s | 2.3 MB 00:00 2024-09-19T05:40:10.289 INFO:teuthology.orchestra.run.smithi123.stdout:(152/156): NetworkManager-libnm-1.51.0-1.el9.x8 4.0 MB/s | 1.8 MB 00:00 2024-09-19T05:40:10.314 INFO:teuthology.orchestra.run.smithi123.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 3.7 MB/s | 249 kB 00:00 2024-09-19T05:40:10.349 INFO:teuthology.orchestra.run.smithi002.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-09-19T05:40:10.356 INFO:teuthology.orchestra.run.smithi123.stdout:(154/156): NetworkManager-team-1.51.0-1.el9.x86 258 kB/s | 41 kB 00:00 2024-09-19T05:40:10.422 INFO:teuthology.orchestra.run.smithi123.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 10 MB/s | 1.4 MB 00:00 2024-09-19T05:40:10.432 INFO:teuthology.orchestra.run.smithi002.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-09-19T05:40:10.498 INFO:teuthology.orchestra.run.smithi123.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-09-19T05:40:10.502 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:40:10.502 INFO:teuthology.orchestra.run.smithi123.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-19T05:40:10.549 INFO:teuthology.orchestra.run.smithi002.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-09-19T05:40:10.600 INFO:teuthology.orchestra.run.smithi002.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-19T05:40:10.633 INFO:teuthology.orchestra.run.smithi002.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-19T05:40:10.658 INFO:teuthology.orchestra.run.smithi002.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-19T05:40:10.784 INFO:teuthology.orchestra.run.smithi002.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-19T05:40:10.809 INFO:teuthology.orchestra.run.smithi002.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T05:40:10.834 INFO:teuthology.orchestra.run.smithi002.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T05:40:10.859 INFO:teuthology.orchestra.run.smithi002.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T05:40:10.893 INFO:teuthology.orchestra.run.smithi002.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-19T05:40:10.935 INFO:teuthology.orchestra.run.smithi002.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-19T05:40:10.977 INFO:teuthology.orchestra.run.smithi002.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T05:40:10.994 INFO:teuthology.orchestra.run.smithi002.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-19T05:40:11.011 INFO:teuthology.orchestra.run.smithi002.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-19T05:40:11.028 INFO:teuthology.orchestra.run.smithi002.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 845 kB/s | 14 kB 00:00 2024-09-19T05:40:11.062 INFO:teuthology.orchestra.run.smithi002.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-19T05:40:11.095 INFO:teuthology.orchestra.run.smithi002.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-19T05:40:11.137 INFO:teuthology.orchestra.run.smithi002.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-09-19T05:40:11.163 INFO:teuthology.orchestra.run.smithi002.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.5 MB/s | 220 kB 00:00 2024-09-19T05:40:11.246 INFO:teuthology.orchestra.run.smithi002.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 25 MB/s | 2.0 MB 00:00 2024-09-19T05:40:11.271 INFO:teuthology.orchestra.run.smithi002.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T05:40:11.297 INFO:teuthology.orchestra.run.smithi002.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-19T05:40:11.322 INFO:teuthology.orchestra.run.smithi002.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-19T05:40:11.347 INFO:teuthology.orchestra.run.smithi002.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 922 kB/s | 23 kB 00:00 2024-09-19T05:40:11.373 INFO:teuthology.orchestra.run.smithi002.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-19T05:40:11.398 INFO:teuthology.orchestra.run.smithi002.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-19T05:40:11.440 INFO:teuthology.orchestra.run.smithi002.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 24 MB/s | 1.0 MB 00:00 2024-09-19T05:40:11.465 INFO:teuthology.orchestra.run.smithi002.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T05:40:11.490 INFO:teuthology.orchestra.run.smithi002.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T05:40:11.515 INFO:teuthology.orchestra.run.smithi002.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-19T05:40:11.541 INFO:teuthology.orchestra.run.smithi002.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-19T05:40:11.566 INFO:teuthology.orchestra.run.smithi002.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 664 kB/s | 16 kB 00:00 2024-09-19T05:40:11.591 INFO:teuthology.orchestra.run.smithi002.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T05:40:11.617 INFO:teuthology.orchestra.run.smithi002.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T05:40:11.642 INFO:teuthology.orchestra.run.smithi002.stdout:(136/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-19T05:40:11.667 INFO:teuthology.orchestra.run.smithi002.stdout:(137/156): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-09-19T05:40:11.692 INFO:teuthology.orchestra.run.smithi002.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T05:40:11.721 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-19T05:40:11.726 INFO:teuthology.orchestra.run.smithi002.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-19T05:40:11.752 INFO:teuthology.orchestra.run.smithi002.stdout:(140/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-19T05:40:11.777 INFO:teuthology.orchestra.run.smithi002.stdout:(141/156): python3-webob-1.8.8-2.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-09-19T05:40:11.802 INFO:teuthology.orchestra.run.smithi002.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T05:40:11.836 INFO:teuthology.orchestra.run.smithi002.stdout:(143/156): python3-werkzeug-2.0.3-3.el9.1.noarc 13 MB/s | 427 kB 00:00 2024-09-19T05:40:11.861 INFO:teuthology.orchestra.run.smithi002.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-19T05:40:11.870 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-19T05:40:11.870 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-19T05:40:11.886 INFO:teuthology.orchestra.run.smithi002.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-19T05:40:11.945 INFO:teuthology.orchestra.run.smithi002.stdout:(146/156): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-19T05:40:12.237 INFO:teuthology.orchestra.run.smithi002.stdout:(147/156): librados2-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-19T05:40:12.412 INFO:teuthology.orchestra.run.smithi002.stdout:(148/156): librbd1-19.1.1-361.g2f599a46.el9.x86 18 MB/s | 3.2 MB 00:00 2024-09-19T05:40:12.779 INFO:teuthology.orchestra.run.smithi002.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2024-09-19T05:40:12.988 INFO:teuthology.orchestra.run.smithi002.stdout:(150/156): NetworkManager-libnm-1.51.0-1.el9.x8 8.7 MB/s | 1.8 MB 00:00 2024-09-19T05:40:13.055 INFO:teuthology.orchestra.run.smithi002.stdout:(151/156): NetworkManager-team-1.51.0-1.el9.x86 614 kB/s | 41 kB 00:00 2024-09-19T05:40:13.116 INFO:teuthology.orchestra.run.smithi002.stdout:(152/156): NetworkManager-tui-1.51.0-1.el9.x86_ 4.1 MB/s | 249 kB 00:00 2024-09-19T05:40:13.256 INFO:teuthology.orchestra.run.smithi002.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-19T05:40:13.514 INFO:teuthology.orchestra.run.smithi002.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-09-19T05:40:13.674 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-19T05:40:13.675 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-19T05:40:14.490 INFO:teuthology.orchestra.run.smithi002.stdout:(155/156): python3-numpy-1.20.1-5.el9.x86_64.rp 658 kB/s | 5.1 MB 00:07 2024-09-19T05:40:19.889 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-19T05:40:19.976 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:40:20.099 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:40:20.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:40:20.378 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:40:20.411 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:40:20.463 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:40:20.806 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:40:20.920 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:40:21.044 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:40:21.071 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:40:21.163 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:40:21.192 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:40:21.338 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:40:21.397 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:40:21.425 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:40:21.521 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:40:21.543 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:40:21.606 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:40:21.627 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:40:21.650 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:40:21.845 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:40:21.982 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:40:22.114 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:40:22.199 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:40:22.310 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:40:22.537 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:40:22.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:40:22.744 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:40:22.773 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:40:22.808 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:40:22.872 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:40:22.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:40:22.949 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:40:22.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:40:23.025 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:40:23.053 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:40:23.122 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:40:23.178 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:40:23.209 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:40:23.244 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:40:23.321 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:40:23.355 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:40:23.399 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:40:23.427 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:40:23.496 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:40:23.542 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:40:23.625 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:40:23.756 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:40:23.810 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:40:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:40:24.405 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:40:24.435 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:40:24.469 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:40:24.507 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:40:24.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:40:24.707 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:40:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:40:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:40:24.896 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:40:24.928 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:40:24.964 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:40:25.010 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:40:25.055 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:40:25.097 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:40:25.140 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:40:25.275 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:40:25.475 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:40:25.517 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:40:25.622 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:40:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:40:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:40:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:40:25.748 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:40:25.822 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:40:25.957 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:40:25.997 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:40:26.025 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:40:26.059 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:40:26.098 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:40:26.130 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:40:26.160 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:40:26.194 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:40:26.227 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:40:26.269 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:40:26.303 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:40:26.390 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:40:26.442 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:40:26.473 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:40:26.576 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:40:26.627 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:40:27.343 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:40:27.389 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:40:27.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:40:27.521 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:40:27.569 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:40:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:40:27.701 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:40:27.742 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:40:27.925 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:40:27.965 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:40:28.042 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:40:28.409 INFO:teuthology.orchestra.run.smithi002.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 825 kB/s | 16 MB 00:20 2024-09-19T05:40:28.412 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-19T05:40:28.412 INFO:teuthology.orchestra.run.smithi002.stdout:Total 7.2 MB/s | 226 MB 00:31 2024-09-19T05:40:28.683 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:40:28.804 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:40:29.663 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-19T05:40:29.834 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-19T05:40:29.834 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-19T05:40:30.024 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:40:30.109 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:40:30.137 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:40:30.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:40:30.479 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:40:30.516 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:40:30.616 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:40:30.646 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:40:30.682 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:40:31.164 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:40:31.194 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:40:31.266 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:40:31.304 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:40:32.189 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-19T05:40:32.189 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-19T05:40:33.612 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:40:33.642 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:40:34.183 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:40:34.220 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:40:34.311 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:40:34.311 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:40:34.311 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:40:34.312 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:34.338 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:40:38.776 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-19T05:40:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T05:40:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T05:40:38.976 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T05:40:39.026 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T05:40:39.051 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T05:40:39.104 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T05:40:39.451 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T05:40:39.552 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:40:39.677 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:40:39.837 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:40:39.932 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:40:39.969 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:40:40.204 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T05:40:40.423 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T05:40:40.449 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:40:40.555 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:40:40.584 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:40:40.647 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T05:40:40.676 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:40:40.703 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T05:40:40.906 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:40:41.078 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T05:40:41.208 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T05:40:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:40:41.392 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:40:41.603 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T05:40:41.727 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T05:40:41.821 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T05:40:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T05:40:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T05:40:41.974 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T05:40:42.026 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T05:40:42.066 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T05:40:42.109 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T05:40:42.142 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T05:40:42.169 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T05:40:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T05:40:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T05:40:42.331 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T05:40:42.374 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T05:40:42.452 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T05:40:42.493 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T05:40:42.537 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T05:40:42.565 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T05:40:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T05:40:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T05:40:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T05:40:42.910 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T05:40:42.967 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T05:40:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T05:40:43.613 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T05:40:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T05:40:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T05:40:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T05:40:43.806 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T05:40:43.963 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T05:40:44.016 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T05:40:44.072 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T05:40:44.154 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T05:40:44.200 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T05:40:44.236 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T05:40:44.285 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T05:40:44.319 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T05:40:44.363 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T05:40:44.553 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T05:40:44.554 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T05:40:44.589 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T05:40:44.639 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T05:40:44.745 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T05:40:44.776 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T05:40:44.833 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T05:40:44.874 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T05:40:44.898 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T05:40:45.018 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T05:40:45.154 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T05:40:45.189 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T05:40:45.217 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T05:40:45.267 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T05:40:45.308 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T05:40:45.347 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T05:40:45.377 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T05:40:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T05:40:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T05:40:45.504 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T05:40:45.537 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T05:40:45.627 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T05:40:45.677 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T05:40:45.707 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T05:40:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T05:40:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T05:40:46.134 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:40:46.134 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-09-19T05:40:46.134 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-09-19T05:40:46.134 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-09-19T05:40:46.134 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-09-19T05:40:46.134 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-09-19T05:40:46.134 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-09-19T05:40:46.135 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-09-19T05:40:46.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:46.315 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:40:46.331 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:40:46.332 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:46.332 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:40:46.332 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:46.646 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T05:40:46.702 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T05:40:46.739 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T05:40:46.842 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T05:40:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T05:40:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T05:40:47.029 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T05:40:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T05:40:47.258 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T05:40:47.289 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T05:40:47.370 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T05:40:47.802 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:40:47.829 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:40:47.829 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:47.829 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:40:47.829 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:40:47.829 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:40:47.829 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T05:40:48.052 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:40:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:40:48.076 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:48.076 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:40:48.076 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:40:48.076 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:40:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T05:40:48.514 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:40:48.535 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:40:48.535 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:48.535 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:40:48.535 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:40:48.535 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:40:48.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:40:48.698 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:40:48.753 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:40:48.937 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:40:48.986 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:40:49.064 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:40:49.153 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:40:49.292 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:40:49.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:40:49.368 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:40:49.396 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:40:49.509 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T05:40:49.584 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T05:40:49.612 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T05:40:49.646 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T05:40:49.724 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:40:49.753 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:40:49.784 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:40:49.847 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:40:49.876 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:40:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T05:40:50.016 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:40:50.091 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T05:40:50.121 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T05:40:50.162 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T05:40:50.674 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T05:40:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:40:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T05:40:50.820 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T05:40:50.923 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:40:50.961 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:40:51.916 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:40:51.961 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:40:52.063 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:40:52.231 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:40:52.263 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:40:52.288 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:40:52.288 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:52.288 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:40:52.288 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:40:52.288 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:40:52.289 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:52.339 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:40:52.351 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:40:52.448 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:40:52.503 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:40:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:40:53.309 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:40:53.421 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:40:53.483 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:40:53.507 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:40:53.507 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:53.507 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:40:53.507 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:40:53.507 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:40:53.508 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:53.550 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:40:53.573 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:40:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:53.574 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:40:53.574 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:53.626 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:40:53.649 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:40:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:40:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:40:53.649 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:40:53.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:53.884 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T05:40:53.929 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:40:53.960 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:40:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:40:53.984 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:40:53.984 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:40:53.984 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:40:53.984 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:40:53.984 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:40:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T05:40:54.023 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T05:40:54.023 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T05:40:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:40:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:40:58.513 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:40:58.555 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:40:58.595 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:40:58.627 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:40:58.655 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:40:58.837 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:40:58.877 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:40:58.906 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:40:58.945 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:40:58.983 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:40:59.603 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:40:59.638 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:40:59.659 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:41:00.520 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:41:00.609 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:41:00.649 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:41:00.740 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:41:00.779 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:41:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:41:00.830 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:41:00.830 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:41:00.906 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:41:00.906 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:41:00.990 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:41:01.015 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:41:01.021 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:41:01.021 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:41:01.208 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:41:01.208 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:41:01.301 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:41:01.363 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:41:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T05:41:06.516 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2024-09-19T05:41:06.517 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2024-09-19T05:41:06.517 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2024-09-19T05:41:06.517 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2024-09-19T05:41:06.517 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2024-09-19T05:41:06.517 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /root 2024-09-19T05:41:06.517 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2024-09-19T05:41:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:06.712 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:41:06.727 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T05:41:06.727 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:06.727 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T05:41:06.727 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:41:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:41:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:41:07.030 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:41:07.031 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:41:07.032 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:41:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:41:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:41:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:41:07.033 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:41:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:41:07.046 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:41:07.047 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:41:07.048 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:41:07.049 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:41:07.050 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:41:07.051 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:41:07.052 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:41:07.053 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:41:07.054 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:41:07.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:41:07.056 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:41:07.057 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:41:07.058 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:41:08.247 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:41:08.275 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T05:41:08.275 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:08.275 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T05:41:08.275 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:41:08.275 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T05:41:08.276 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:08.514 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:41:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T05:41:08.539 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:08.539 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T05:41:08.539 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:41:08.539 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T05:41:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:08.998 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:41:09.023 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T05:41:09.023 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:09.023 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T05:41:09.023 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:41:09.023 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T05:41:09.023 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:09.129 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T05:41:09.196 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T05:41:09.267 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T05:41:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T05:41:09.524 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T05:41:09.607 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T05:41:09.655 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:41:09.655 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:09.655 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-19T05:41:09.655 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:41:09.656 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:41:09.658 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.659 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.660 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:41:09.661 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.662 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:41:09.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:41:09.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:41:09.665 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:41:09.666 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:41:09.667 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:41:09.668 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:41:09.668 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:41:09.668 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:41:09.668 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:09.668 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-19T05:41:09.702 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T05:41:09.855 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T05:41:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T05:41:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T05:41:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T05:41:10.040 DEBUG:teuthology.parallel:result is None 2024-09-19T05:41:10.381 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:41:10.408 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T05:41:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T05:41:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T05:41:10.530 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:41:11.639 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T05:41:11.674 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:41:12.686 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T05:41:12.716 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:41:12.826 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T05:41:12.994 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T05:41:13.018 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:41:13.044 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T05:41:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T05:41:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:41:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T05:41:13.044 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:13.102 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:41:13.114 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T05:41:13.216 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T05:41:13.282 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T05:41:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T05:41:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:41:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T05:41:14.313 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:14.313 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T05:41:14.313 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:41:14.313 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T05:41:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:41:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T05:41:14.379 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:14.379 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T05:41:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:41:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T05:41:14.455 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:14.455 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T05:41:14.455 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:41:14.455 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T05:41:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:41:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T05:41:14.798 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T05:41:14.798 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T05:41:14.798 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:41:14.798 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T05:41:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T05:41:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T05:41:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T05:41:19.742 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T05:41:19.769 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T05:41:19.978 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T05:41:20.023 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T05:41:20.054 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T05:41:20.084 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T05:41:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T05:41:20.729 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:41:20.769 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:41:20.791 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T05:41:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T05:41:21.807 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T05:41:21.868 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T05:41:21.967 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T05:41:22.002 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T05:41:22.036 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T05:41:22.070 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T05:41:22.070 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:41:22.187 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T05:41:22.187 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:41:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T05:41:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T05:41:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:41:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:41:22.499 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T05:41:22.499 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:41:22.583 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T05:41:22.652 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:41:28.361 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:41:28.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T05:41:28.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T05:41:28.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T05:41:28.361 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T05:41:28.362 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T05:41:28.363 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T05:41:28.364 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T05:41:28.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T05:41:28.366 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T05:41:28.367 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T05:41:28.368 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T05:41:28.369 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T05:41:28.370 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T05:41:28.371 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T05:41:28.372 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T05:41:28.373 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T05:41:28.374 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T05:41:28.375 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T05:41:28.376 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T05:41:28.376 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T05:41:28.376 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T05:41:28.376 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T05:41:28.376 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T05:41:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T05:41:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T05:41:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T05:41:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T05:41:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T05:41:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T05:41:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T05:41:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T05:41:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:31.130 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-19T05:41:31.614 DEBUG:teuthology.parallel:result is None 2024-09-19T05:41:31.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:41:31.767 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:41:31.791 INFO:teuthology.orchestra.run.smithi002.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:41:31.792 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:41:31.792 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:41:31.794 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T05:41:31.934 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T05:41:31.960 INFO:teuthology.orchestra.run.smithi123.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T05:41:31.960 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T05:41:31.960 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T05:41:31.962 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T05:41:31.962 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:31.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:41:31.992 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:31.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T05:41:32.031 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T05:41:32.031 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:32.031 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:41:32.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:41:32.125 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:32.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T05:41:32.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T05:41:32.215 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T05:41:32.215 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:32.215 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:41:32.241 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:41:32.308 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:32.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T05:41:32.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T05:41:32.397 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T05:41:32.398 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:32.398 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:41:32.423 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:41:32.489 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:32.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T05:41:32.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T05:41:32.579 INFO:teuthology.run_tasks:Running task ceph... 2024-09-19T05:41:32.654 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-19T05:41:32.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /var/log/ceph 2024-09-19T05:41:32.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-09-19T05:41:32.682 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-19T05:41:32.682 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T05:41:32.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T05:41:32.748 INFO:tasks.ceph:Creating extra log directories... 2024-09-19T05:41:32.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T05:41:32.789 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T05:41:32.820 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-19T05:41:32.821 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-19T05:41:32.821 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-19T05:41:32.821 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:32.821 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T05:41:32.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T05:41:32.983 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-19T05:41:33.008 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:33.008 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T05:41:33.023 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T05:41:33.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-19T05:41:33.163 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-19T05:41:33.163 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-19T05:41:33.163 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691', 'branch': 'squid', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7911691', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 1747, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'sleep_before_teardown': 0, 'subset': '176/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi002.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEzcFihS/AHhWrWTikY8iHDseziX16fT9Na0ygEssM+MJM5Mpp/4OmyjErb5VZionebpsAqC7hLjQ93TlC+hJrI=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO91wzOMFSawvXpgzm6IgCzlJj68/DkalEUifJvt5kdd8UZx5JkE8mdKGpyGkbWnmeUhujj/AsG/tozDFMsrs1Q='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'thrashosds': None}, {'workunit': {'clients': {'all': ['suites/iozone.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_21:24:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-19T05:41:33.164 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T05:41:33.166 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T05:41:33.213 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T05:41:33.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T05:41:33.277 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:33.277 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:41:33.302 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:41:33.302 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-09-19T05:41:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:41:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:33.361 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-19T05:41:33.361 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:33.361 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:33.361 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-19 05:41:23.012139943 +0000 2024-09-19T05:41:33.362 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-19 05:39:11.352572180 +0000 2024-09-19T05:41:33.362 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-19 05:39:11.352572180 +0000 2024-09-19T05:41:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-19 05:39:11.352572180 +0000 2024-09-19T05:41:33.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:41:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-19T05:41:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-19T05:41:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 8.7059e-05 s, 5.9 MB/s 2024-09-19T05:41:33.426 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:41:33.483 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-09-19T05:41:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-19 05:41:23.012139943 +0000 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-19 05:39:11.670564716 +0000 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-19 05:39:11.670564716 +0000 2024-09-19T05:41:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-19 05:39:11.670564716 +0000 2024-09-19T05:41:33.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:41:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-19T05:41:33.603 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-19T05:41:33.604 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000101072 s, 5.1 MB/s 2024-09-19T05:41:33.605 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:41:33.661 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-09-19T05:41:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:41:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:33.719 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-19T05:41:33.719 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:33.719 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:33.719 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-19 05:41:23.013139920 +0000 2024-09-19T05:41:33.719 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-19 05:39:12.006556831 +0000 2024-09-19T05:41:33.720 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-19 05:39:12.006556831 +0000 2024-09-19T05:41:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-19 05:39:12.006556831 +0000 2024-09-19T05:41:33.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:41:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-19T05:41:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-19T05:41:33.783 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 9.2289e-05 s, 5.5 MB/s 2024-09-19T05:41:33.784 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:41:33.841 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-09-19T05:41:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:41:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:33.898 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T05:41:33.898 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:33.898 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:33.898 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-19 05:41:23.013139920 +0000 2024-09-19T05:41:33.899 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-19 05:39:12.326549320 +0000 2024-09-19T05:41:33.899 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-19 05:39:12.326549320 +0000 2024-09-19T05:41:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-19 05:39:12.326549320 +0000 2024-09-19T05:41:33.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:41:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-19T05:41:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-19T05:41:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000101371 s, 5.1 MB/s 2024-09-19T05:41:33.963 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:41:34.020 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:34.021 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:34.021 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T05:41:34.038 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T05:41:34.038 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-09-19T05:41:34.094 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 05:41:01.714076348 +0000 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 05:39:15.863586299 +0000 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 05:39:15.863586299 +0000 2024-09-19T05:41:34.095 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 05:39:15.863586299 +0000 2024-09-19T05:41:34.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T05:41:34.159 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T05:41:34.159 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T05:41:34.159 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000207791 s, 2.5 MB/s 2024-09-19T05:41:34.160 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T05:41:34.219 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-09-19T05:41:34.273 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 05:41:01.714076348 +0000 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 05:39:16.177578585 +0000 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 05:39:16.177578585 +0000 2024-09-19T05:41:34.274 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 05:39:16.177578585 +0000 2024-09-19T05:41:34.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T05:41:34.337 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T05:41:34.337 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T05:41:34.337 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000190166 s, 2.7 MB/s 2024-09-19T05:41:34.338 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T05:41:34.397 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 05:41:01.714076348 +0000 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 05:39:16.513570331 +0000 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 05:39:16.513570331 +0000 2024-09-19T05:41:34.452 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 05:39:16.513570331 +0000 2024-09-19T05:41:34.453 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T05:41:34.514 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T05:41:34.514 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T05:41:34.514 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00016488 s, 3.1 MB/s 2024-09-19T05:41:34.516 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T05:41:34.571 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-19 05:41:01.715076323 +0000 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-19 05:39:16.836562396 +0000 2024-09-19T05:41:34.627 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-19 05:39:16.836562396 +0000 2024-09-19T05:41:34.628 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-19 05:39:16.836562396 +0000 2024-09-19T05:41:34.628 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T05:41:34.691 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-19T05:41:34.691 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-19T05:41:34.691 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000193142 s, 2.7 MB/s 2024-09-19T05:41:34.692 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T05:41:34.749 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:34.749 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T05:41:34.749 INFO:tasks.ceph:Generating config... 2024-09-19T05:41:34.751 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-19T05:41:34.751 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-19T05:41:34.751 INFO:tasks.ceph:[client] debug client = 20 2024-09-19T05:41:34.751 INFO:tasks.ceph:[client] debug ms = 1 2024-09-19T05:41:34.751 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-19T05:41:34.751 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-19T05:41:34.751 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-19T05:41:34.751 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-19T05:41:34.752 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-19T05:41:34.753 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-19T05:41:34.754 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-19T05:41:34.754 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-19T05:41:34.755 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-19T05:41:34.755 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-19T05:41:34.755 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-19T05:41:34.755 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-19T05:41:34.755 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-19T05:41:34.755 INFO:tasks.ceph:Setting up mon.a... 2024-09-19T05:41:34.755 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-19T05:41:34.823 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.keyring 2024-09-19T05:41:34.826 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-19T05:41:34.874 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T05:41:34.900 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.2'), ('mon.b', '172.21.15.123'), ('mon.c', '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]')] 2024-09-19T05:41:34.901 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.2,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-19T05:41:34.902 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:34.902 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T05:41:34.957 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.2 --add b 172.21.15.123 --addv c '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: generated fsid 896e162c-4658-4743-b70e-7ec2b5bce950 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = quincy 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 0 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 896e162c-4658-4743-b70e-7ec2b5bce950 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:last_changed 2024-09-19T05:41:35.045562+0000 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-19T05:41:35.045562+0000 2024-09-19T05:41:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:min_mon_release 17 (quincy) 2024-09-19T05:41:35.047 INFO:teuthology.orchestra.run.smithi002.stdout:election_strategy: 1 2024-09-19T05:41:35.047 INFO:teuthology.orchestra.run.smithi002.stdout:0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-19T05:41:35.047 INFO:teuthology.orchestra.run.smithi002.stdout:1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-19T05:41:35.047 INFO:teuthology.orchestra.run.smithi002.stdout:2: [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] mon.c 2024-09-19T05:41:35.047 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-19T05:41:35.048 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T05:41:35.063 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 896e162c-4658-4743-b70e-7ec2b5bce950... 2024-09-19T05:41:35.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T05:41:35.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T05:41:35.148 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-09-19T05:41:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2024-09-19T05:41:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2024-09-19T05:41:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: mon_allow_pool_size_one = true 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: mon health detail to clog = false 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.2,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on skipped message = False 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 896e162c-4658-4743-b70e-7ec2b5bce950 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: osd mclock profile = high_recovery_ops 2024-09-19T05:41:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug trim objects = true 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore compression mode = aggressive 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 20 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 20 2024-09-19T05:41:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 20 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 10 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: osd blocked scrub grace period = 3600 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: osd max markdown count = 1000 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-19T05:41:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2024-09-19T05:41:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: # 1m isn't quite enough 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: mon_down_mkfs_grace = 2m 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_filestore_osds = false 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: mon down mkfs grace = 300 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: mon op complaint time = 120 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on pool no app = False 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T05:41:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: client mount timeout = 600 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: client_shutdown_timeout = 120 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 900 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 900 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d] 2024-09-19T05:41:35.159 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f] 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout:[mds] 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds = 20 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds balancer = 20 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal fragment size max = 10000 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal merge size = 5 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split bits = 3 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds bal split size = 100 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug frag = True 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug scatterstat = True 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds oft prefetch dirfrags = True 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds op complaint time = 180 2024-09-19T05:41:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: mds verify scatter = True 2024-09-19T05:41:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: mds_dir_prefetch = True 2024-09-19T05:41:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: mds_sleep_rank_change = 5000000.0 2024-09-19T05:41:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2024-09-19T05:41:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 900 2024-09-19T05:41:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 900 2024-09-19T05:41:35.162 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-09-19T05:41:35.162 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-09-19T05:41:35.162 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T05:41:35.163 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-09-19T05:41:35.164 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-09-19T05:41:35.165 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.2,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on skipped message = False 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 896e162c-4658-4743-b70e-7ec2b5bce950 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.166 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.167 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore compression mode = aggressive 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T05:41:35.168 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: osd blocked scrub grace period = 3600 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: osd max markdown count = 1000 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-09-19T05:41:35.169 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-09-19T05:41:35.170 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: mon down mkfs grace = 300 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on pool no app = False 2024-09-19T05:41:35.171 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: client_shutdown_timeout = 120 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-09-19T05:41:35.172 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.e] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.f] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal fragment size max = 10000 2024-09-19T05:41:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal merge size = 5 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal split bits = 3 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds bal split size = 100 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds oft prefetch dirfrags = True 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds_dir_prefetch = True 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: mds_sleep_rank_change = 5000000.0 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-09-19T05:41:35.174 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-09-19T05:41:35.174 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-19T05:41:35.175 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-19T05:41:35.259 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-19T05:41:35.259 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:35.259 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-19T05:41:35.274 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:35.275 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-19T05:41:35.331 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:41:35.331 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:35.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T05:41:35.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T05:41:35.409 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:41:35.409 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:35.466 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:41:35.466 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:35.466 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T05:41:35.466 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T05:41:35.501 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:41:35.502 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:35.556 INFO:tasks.ceph:Setting up mon nodes... 2024-09-19T05:41:35.557 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-19T05:41:35.557 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T05:41:35.613 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T05:41:35.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T05:41:35.695 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T05:41:35.698 INFO:tasks.ceph:Setting up mds nodes... 2024-09-19T05:41:35.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T05:41:35.751 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T05:41:35.754 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-19T05:41:35.779 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T05:41:35.872 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T05:41:35.874 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-19T05:41:35.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-19T05:41:35.993 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-19T05:41:35.996 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-19T05:41:36.022 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T05:41:36.074 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T05:41:36.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-19T05:41:36.101 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T05:41:36.193 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T05:41:36.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-19T05:41:36.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-19T05:41:36.311 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-19T05:41:36.314 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-19T05:41:36.337 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-19T05:41:36.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T05:41:36.382 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-19T05:41:36.395 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-19T05:41:36.396 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T05:41:36.396 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-19T05:41:36.421 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:36.421 INFO:tasks.ceph:role: osd.0 2024-09-19T05:41:36.421 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:41:36.421 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T05:41:36.487 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:36.487 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:36.488 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:36.488 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:36.488 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:36.488 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:36.488 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:36.519 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-19T05:41:36.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-19T05:41:36.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-19T05:41:36.592 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-19T05:41:36.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-19T05:41:36.729 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:36.729 INFO:tasks.ceph:role: osd.1 2024-09-19T05:41:36.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:41:36.729 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T05:41:36.794 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:36.794 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:36.795 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:36.795 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:36.795 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:36.826 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-19T05:41:36.829 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-19T05:41:36.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-19T05:41:36.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-19T05:41:36.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-19T05:41:37.036 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:37.037 INFO:tasks.ceph:role: osd.2 2024-09-19T05:41:37.037 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:41:37.037 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:37.102 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:37.103 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:37.103 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:37.134 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-19T05:41:37.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-19T05:41:37.136 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-19T05:41:37.205 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-19T05:41:37.277 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-19T05:41:37.343 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:37.343 INFO:tasks.ceph:role: osd.3 2024-09-19T05:41:37.343 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:41:37.343 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T05:41:37.408 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:37.409 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:37.409 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:37.409 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:37.410 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:37.441 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-19T05:41:37.442 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-19T05:41:37.443 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-19T05:41:37.511 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-19T05:41:37.581 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:37.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:37.685+0000 7fc27842f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-19T05:41:37.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:37.685+0000 7fc27842f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-19T05:41:37.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:37.685+0000 7fc27842f640 -1 bdev(0x55dc0bc35800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-19T05:41:37.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:37.685+0000 7fc27842f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-19T05:41:40.862 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-19T05:41:40.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:40.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:40.982+0000 7fd087ba8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-19T05:41:40.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:40.982+0000 7fd087ba8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-19T05:41:40.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:40.982+0000 7fd087ba8640 -1 bdev(0x55ba7b0c5800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-19T05:41:40.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:40.982+0000 7fd087ba8640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-19T05:41:44.147 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-19T05:41:44.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:44.267+0000 7f7305c22640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-19T05:41:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:44.268+0000 7f7305c22640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-19T05:41:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:44.268+0000 7f7305c22640 -1 bdev(0x559e4d65d800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-19T05:41:44.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:44.268+0000 7f7305c22640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-19T05:41:47.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-19T05:41:47.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:47.314+0000 7f9c90a15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-19T05:41:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:47.314+0000 7f9c90a15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-19T05:41:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:47.314+0000 7f9c90a15640 -1 bdev(0x55fb7cd73800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-19T05:41:47.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:41:47.314+0000 7f9c90a15640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-19T05:41:50.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T05:41:50.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-19T05:41:50.781 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:50.781 INFO:tasks.ceph:role: osd.4 2024-09-19T05:41:50.781 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:41:50.781 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T05:41:50.857 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:50.857 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:50.857 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:50.858 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:50.858 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:50.858 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:50.858 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:51.377 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-19T05:41:51.380 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-19T05:41:51.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-19T05:41:51.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-19T05:41:51.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-19T05:41:51.644 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:51.644 INFO:tasks.ceph:role: osd.5 2024-09-19T05:41:51.644 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:41:51.644 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T05:41:51.721 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:51.721 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:51.721 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:51.721 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:51.722 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:51.722 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:51.722 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:51.722 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:51.722 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:51.722 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:52.209 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-19T05:41:52.211 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-19T05:41:52.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-19T05:41:52.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-19T05:41:52.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-19T05:41:52.399 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:52.400 INFO:tasks.ceph:role: osd.6 2024-09-19T05:41:52.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:41:52.400 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:52.472 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:52.473 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:53.001 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-19T05:41:53.002 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-19T05:41:53.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-19T05:41:53.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-19T05:41:53.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-19T05:41:53.209 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T05:41:53.209 INFO:tasks.ceph:role: osd.7 2024-09-19T05:41:53.209 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:41:53.210 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T05:41:53.281 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T05:41:53.281 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T05:41:53.281 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T05:41:53.281 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T05:41:53.282 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T05:41:53.282 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-19T05:41:53.282 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T05:41:53.282 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T05:41:53.282 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T05:41:53.282 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T05:41:53.776 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-19T05:41:53.778 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-19T05:41:53.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-19T05:41:53.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-19T05:41:53.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:54.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:54.029+0000 7f4c5940b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-19T05:41:54.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:54.029+0000 7f4c5940b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-19T05:41:54.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:54.029+0000 7f4c5940b640 -1 bdev(0x55ec6a159800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-19T05:41:54.031 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:54.029+0000 7f4c5940b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-19T05:41:57.448 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-19T05:41:57.479 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:41:57.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:57.571+0000 7f26084a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-19T05:41:57.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:57.572+0000 7f26084a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-19T05:41:57.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:57.572+0000 7f26084a5640 -1 bdev(0x560daaad3800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-19T05:41:57.574 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:41:57.572+0000 7f26084a5640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-19T05:42:00.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-19T05:42:00.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:42:00.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:00.606+0000 7fe783c1c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-19T05:42:00.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:00.606+0000 7fe783c1c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-19T05:42:00.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:00.606+0000 7fe783c1c640 -1 bdev(0x561455561800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-19T05:42:00.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:00.607+0000 7fe783c1c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-19T05:42:03.166 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:42:03.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:42:03.530 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-19T05:42:03.555 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:42:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:03.639+0000 7fe40961e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-19T05:42:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:03.639+0000 7fe40961e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-19T05:42:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:03.639+0000 7fe40961e640 -1 bdev(0x55b3fe78f800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-19T05:42:03.641 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:03.639+0000 7fe40961e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-19T05:42:06.802 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-19T05:42:06.832 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-19T05:42:06.832 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:06.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-19T05:42:06.863 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:06.863 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-19T05:42:06.933 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:06.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-19T05:42:07.003 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:07.003 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-19T05:42:07.072 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:07.073 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-19T05:42:07.142 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:07.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-19T05:42:07.210 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:07.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-19T05:42:07.278 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:07.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-19T05:42:07.345 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.345 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-19T05:42:07.374 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.375 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-19T05:42:07.443 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-19T05:42:07.511 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-19T05:42:07.577 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-19T05:42:07.642 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-19T05:42:07.708 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-19T05:42:07.773 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:07.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-19T05:42:07.839 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:07.839 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-19T05:42:07.858 INFO:tasks.ceph:Adding keys to all mons... 2024-09-19T05:42:07.858 DEBUG:teuthology.orchestra.run.smithi002:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T05:42:07.901 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T05:42:07.923 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCPuetmmLVQJBAAmRMsC3APXL3A8kbAG+y0YQ== 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCPuetmM1zALBAAsCHOvlC1ulBI1X+1j5dKmQ== 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCPuetmnt7uMxAA9v+bwmBgTxtPNIOTI9Gcdg== 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.e] 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCPuetmklIiOxAAV7GPsxdOL4FEOExBcStR2g== 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCRuetmw1HsKBAA7x8VNlrzOSR9NcYrUGNhYw== 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-09-19T05:42:07.924 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCUuetmzAKkOhAArKNowgL0BLD+i+tklzqLdw== 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCYuetmeuoLEBAAuR721HqxFtqO6+LfwN+0zw== 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCbuetmPCLNEhAAqZF1M7+fhb/SC2E4azi9Sg== 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCPuetm0WNrKRAAyy9Xqtz7AWlFpapTxVxB4w== 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCQuetmKv9fBBAAI2nzYTZd246iz/IJS6B0jA== 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCQuetmqoJ2CxAA21m0TIFqr0u2OkSahCt3IA== 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.f] 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCQuetmQoeGEhAAdrI7nlZIiPUh4OdINNbY6A== 2024-09-19T05:42:07.925 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCiuetm8f/PARAAvw2fb2hUYMOT6TSBS/5rPg== 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCluetm0EYlIhAAC5fX4P63r4aB6zHov+qgCg== 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCouetmjYI1JBAAwDIySoiMXyM2edDksXk9mw== 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCruetmKAguJhAAqQ8zI7O+sA5I7qmUnCHUyg== 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-09-19T05:42:07.926 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCQuetmLf26FhAAOzzHcTeWN9CDBuLlPe90iA== 2024-09-19T05:42:07.927 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.y] 2024-09-19T05:42:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCPuetmmLVQJBAAmRMsC3APXL3A8kbAG+y0YQ== 2024-09-19T05:42:07.927 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2024-09-19T05:42:07.927 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCPuetmM1zALBAAsCHOvlC1ulBI1X+1j5dKmQ== 2024-09-19T05:42:07.927 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c] 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCPuetmnt7uMxAA9v+bwmBgTxtPNIOTI9Gcdg== 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e] 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCPuetmklIiOxAAV7GPsxdOL4FEOExBcStR2g== 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.0] 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCRuetmw1HsKBAA7x8VNlrzOSR9NcYrUGNhYw== 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.1] 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCUuetmzAKkOhAArKNowgL0BLD+i+tklzqLdw== 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.2] 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCYuetmeuoLEBAAuR721HqxFtqO6+LfwN+0zw== 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.3] 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCbuetmPCLNEhAAqZF1M7+fhb/SC2E4azi9Sg== 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.x] 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCPuetm0WNrKRAAyy9Xqtz7AWlFpapTxVxB4w== 2024-09-19T05:42:07.928 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCQuetmKv9fBBAAI2nzYTZd246iz/IJS6B0jA== 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d] 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCQuetmqoJ2CxAA21m0TIFqr0u2OkSahCt3IA== 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f] 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCQuetmQoeGEhAAdrI7nlZIiPUh4OdINNbY6A== 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.4] 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCiuetm8f/PARAAvw2fb2hUYMOT6TSBS/5rPg== 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.5] 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCluetm0EYlIhAAC5fX4P63r4aB6zHov+qgCg== 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.6] 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCouetmjYI1JBAAwDIySoiMXyM2edDksXk9mw== 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.7] 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCruetmKAguJhAAqQ8zI7O+sA5I7qmUnCHUyg== 2024-09-19T05:42:07.929 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-09-19T05:42:07.930 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCQuetmLf26FhAAOzzHcTeWN9CDBuLlPe90iA== 2024-09-19T05:42:07.930 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T05:42:07.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T05:42:08.030 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.143 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.201 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.252 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.357 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T05:42:08.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T05:42:08.463 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.519 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.521 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.576 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T05:42:08.631 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.684 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.737 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T05:42:08.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T05:42:08.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T05:42:08.899 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-19T05:42:08.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-19T05:42:08.924 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T05:42:09.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-19T05:42:09.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-19T05:42:09.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T05:42:09.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-19T05:42:09.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-19T05:42:09.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T05:42:09.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-19T05:42:09.378 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:42:09.381 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T05:42:09.433 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-19T05:42:09.434 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-19T05:42:09.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-19T05:42:09.437 INFO:tasks.ceph.mon.a:Started 2024-09-19T05:42:09.437 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-19T05:42:09.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-19T05:42:09.476 INFO:tasks.ceph.mon.b:Started 2024-09-19T05:42:09.476 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-19T05:42:09.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-19T05:42:09.478 INFO:tasks.ceph.mon.c:Started 2024-09-19T05:42:09.478 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-19T05:42:09.478 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-19T05:42:09.478 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-19T05:42:09.480 INFO:tasks.ceph.mgr.y:Started 2024-09-19T05:42:09.481 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-19T05:42:09.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-19T05:42:09.483 INFO:tasks.ceph.mgr.x:Started 2024-09-19T05:42:09.483 INFO:tasks.ceph:Setting crush tunables to default 2024-09-19T05:42:09.483 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd crush tunables default 2024-09-19T05:42:09.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.584+0000 7f0d5abbd640 1 Processor -- start 2024-09-19T05:42:09.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.584+0000 7f0d5abbd640 1 -- start start 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d5abbd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 0x7f0d54072c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d5abbd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 0x7f0d540c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d5abbd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d540c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d5abbd640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f0d5406c7d0 con 0x7f0d54154b30 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d5abbd640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0d54073410 con 0x7f0d54154f10 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d5abbd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540731a0 con 0x7f0d5415a990 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d58959640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 0x7f0d54072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d58959640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 0x7f0d54072c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55868/0 (socket says 172.21.15.2:55868) 2024-09-19T05:42:09.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d58959640 1 -- 172.21.15.2:0/248453328 learned_addr learned my addr 172.21.15.2:0/248453328 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d58959640 1 -- 172.21.15.2:0/248453328 >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 msgr2=0x7f0d54072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d58959640 1 -- 172.21.15.2:0/248453328 >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 msgr2=0x7f0d54072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d58959640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 0x7f0d54072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d58959640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 0x7f0d54072c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d53fff640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 0x7f0d540c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.585+0000 7f0d5915a640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d540c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d53fff640 1 -- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 msgr2=0x7f0d540c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d53fff640 1 -- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 msgr2=0x7f0d540c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d53fff640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 0x7f0d540c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d53fff640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 0x7f0d540c8ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d5915a640 1 -- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 msgr2=0x7f0d540c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-19T05:42:09.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d5915a640 1 -- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 msgr2=0x7f0d540c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T05:42:09.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d5915a640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d540c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T05:42:09.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.586+0000 7f0d5915a640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d540c7330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:42:09.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.786+0000 7f0d58959640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.2:3300/0 conn(0x7f0d54154b30 0x7f0d54072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:09.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.786+0000 7f0d58959640 1 -- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 msgr2=0x7f0d540c7330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:42:09.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.786+0000 7f0d58959640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d540c7330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.786+0000 7f0d58959640 1 -- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 msgr2=0x7f0d540c8ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:42:09.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.786+0000 7f0d58959640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 0x7f0d540c8ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.786+0000 7f0d58959640 1 -- 172.21.15.2:0/248453328 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d540c9420 con 0x7f0d54154b30 2024-09-19T05:42:09.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.787+0000 7f0d58959640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d54072c60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d540c80f0 tx=0x7f0d4c02a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb83db392d564a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:09.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.787+0000 7f0d537fe640 1 -- 172.21.15.2:0/248453328 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c039070 con 0x7f0d54154b30 2024-09-19T05:42:09.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.787+0000 7f0d537fe640 1 -- 172.21.15.2:0/248453328 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c034620 con 0x7f0d54154b30 2024-09-19T05:42:09.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.787+0000 7f0d537fe640 1 -- 172.21.15.2:0/248453328 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c034a10 con 0x7f0d54154b30 2024-09-19T05:42:09.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 -- 172.21.15.2:0/248453328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 msgr2=0x7f0d54072c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:09.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d54072c60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0d540c80f0 tx=0x7f0d4c02a860 comp rx=0 tx=0).stop 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 -- 172.21.15.2:0/248453328 shutdown_connections 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d540c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/248453328 >> v2:172.21.15.123:3300/0 conn(0x7f0d54154f10 0x7f0d540c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/248453328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d54072c60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 -- 172.21.15.2:0/248453328 >> 172.21.15.2:0/248453328 conn(0x7f0d5406d820 msgr2=0x7f0d540cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 -- 172.21.15.2:0/248453328 shutdown_connections 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.788+0000 7f0d5abbd640 1 -- 172.21.15.2:0/248453328 wait complete. 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.789+0000 7f0d5abbd640 1 Processor -- start 2024-09-19T05:42:09.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.789+0000 7f0d5abbd640 1 -- start start 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d5abbd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d541e4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d5abbd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d541e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d5abbd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d540cbe60 0x7f0d541e1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d5abbd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54073e50 con 0x7f0d54154b30 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d5abbd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d54073b70 con 0x7f0d540cbe60 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d5abbd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0d540c86e0 con 0x7f0d5415a990 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d541e4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d541e4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55910/0 (socket says 172.21.15.2:55910) 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 -- 172.21.15.2:0/1839850565 learned_addr learned my addr 172.21.15.2:0/1839850565 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d5915a640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d540cbe60 0x7f0d541e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d53fff640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d541e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:09.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 -- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 msgr2=0x7f0d541e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d541e5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 -- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d540cbe60 msgr2=0x7f0d541e1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d540cbe60 0x7f0d541e1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 -- 172.21.15.2:0/1839850565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d541e0ad0 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.790+0000 7f0d58959640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d541e4c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c037040 tx=0x7f0d4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.791+0000 7f0d517fa640 1 -- 172.21.15.2:0/1839850565 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c039040 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.791+0000 7f0d517fa640 1 -- 172.21.15.2:0/1839850565 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c044070 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.791+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d541e3400 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.791+0000 7f0d517fa640 1 -- 172.21.15.2:0/1839850565 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c034cf0 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.791+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d540cd200 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.791+0000 7f0d517fa640 1 -- 172.21.15.2:0/1839850565 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d4c004040 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.791+0000 7f0d517fa640 1 -- 172.21.15.2:0/1839850565 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d4c049580 con 0x7f0d54154b30 2024-09-19T05:42:09.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.792+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d540c89c0 con 0x7f0d54154b30 2024-09-19T05:42:09.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.794+0000 7f0d517fa640 1 -- 172.21.15.2:0/1839850565 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0d4c02f070 con 0x7f0d54154b30 2024-09-19T05:42:09.856 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:09.854+0000 7f3085342040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:42:09.857 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:09.856+0000 7f8cc7836040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T05:42:09.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.858+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0d54072df0 con 0x7f0d54154b30 2024-09-19T05:42:09.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.860+0000 7f0d517fa640 1 -- 172.21.15.2:0/1839850565 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f0d4c004470 con 0x7f0d54154b30 2024-09-19T05:42:09.861 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-09-19T05:42:09.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 msgr2=0x7f0d541e4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:09.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d541e4c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c037040 tx=0x7f0d4c004ab0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 shutdown_connections 2024-09-19T05:42:09.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d540cbe60 0x7f0d541e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5415a990 0x7f0d541e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 --2- 172.21.15.2:0/1839850565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d54154b30 0x7f0d541e4c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:09.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 >> 172.21.15.2:0/1839850565 conn(0x7f0d5406d820 msgr2=0x7f0d5406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:09.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 shutdown_connections 2024-09-19T05:42:09.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:09.861+0000 7f0d5abbd640 1 -- 172.21.15.2:0/1839850565 wait complete. 2024-09-19T05:42:09.873 INFO:tasks.ceph:check_enable_crimson: False 2024-09-19T05:42:09.873 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-19T05:42:09.873 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:09.873 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-19T05:42:09.896 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:09.895+0000 7f3085342040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:42:09.899 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:09.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-19T05:42:09.902 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:09.901+0000 7f8cc7836040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T05:42:09.928 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:09.928 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-19T05:42:09.993 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:09.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-19T05:42:10.036 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:10.034+0000 7f3085342040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:42:10.052 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:10.051+0000 7f8cc7836040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T05:42:10.059 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:10.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-19T05:42:10.084 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:10.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-19T05:42:10.088 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:10.086+0000 7f3085342040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:42:10.105 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:10.104+0000 7f8cc7836040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T05:42:10.112 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:10.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-19T05:42:10.176 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-19T05:42:10.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-19T05:42:10.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 665c2ac5-a964-4f38-8fa6-f4e95cac9272 0 2024-09-19T05:42:10.256 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:10.254+0000 7f3085342040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:42:10.281 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:10.280+0000 7f8cc7836040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:42:10.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.351+0000 7f196532c640 1 Processor -- start 2024-09-19T05:42:10.353 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.351+0000 7f196532c640 1 -- start start 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.352+0000 7f196532c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f19600c9000 0x7f196006bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.352+0000 7f196532c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196006c580 con 0x7f1960155040 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.352+0000 7f196532c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f196006c820 con 0x7f19600706a0 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.352+0000 7f196532c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f19600c5490 con 0x7f19600c9000 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.352+0000 7f195effd640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f19600706a0 0x7f1960154930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47708/0 (socket says 172.21.15.123:47708) 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.352+0000 7f195effd640 1 -- 172.21.15.123:0/4236799082 learned_addr learned my addr 172.21.15.123:0/4236799082 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.352+0000 7f195f7fe640 1 --2- 172.21.15.123:0/4236799082 >> v2:172.21.15.2:3300/0 conn(0x7f19600c9000 0x7f196006bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.355 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.353+0000 7f195dffb640 1 -- 172.21.15.123:0/4236799082 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433283779 0 0) 0x7f196006c820 con 0x7f19600706a0 2024-09-19T05:42:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.353+0000 7f195dffb640 1 -- 172.21.15.123:0/4236799082 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1948003520 con 0x7f19600706a0 2024-09-19T05:42:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.353+0000 7f195dffb640 1 -- 172.21.15.123:0/4236799082 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2515049787 0 0) 0x7f196006c580 con 0x7f1960155040 2024-09-19T05:42:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.353+0000 7f195dffb640 1 -- 172.21.15.123:0/4236799082 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f196006c820 con 0x7f1960155040 2024-09-19T05:42:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.353+0000 7f195f7fe640 1 -- 172.21.15.123:0/4236799082 >> v1:172.21.15.123:6789/0 conn(0x7f19600706a0 legacy=0x7f1960154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.353+0000 7f195f7fe640 1 -- 172.21.15.123:0/4236799082 >> v1:172.21.15.2:6789/0 conn(0x7f1960155040 legacy=0x7f19600c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.353+0000 7f195f7fe640 1 -- 172.21.15.123:0/4236799082 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196006cb00 con 0x7f19600c9000 2024-09-19T05:42:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f195f7fe640 1 --2- 172.21.15.123:0/4236799082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19600c9000 0x7f196006bf20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1958009d90 tx=0x7f195802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ab2d0b1389f6bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f195dffb640 1 -- 172.21.15.123:0/4236799082 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958007740 con 0x7f19600c9000 2024-09-19T05:42:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f195dffb640 1 -- 172.21.15.123:0/4236799082 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19580078e0 con 0x7f19600c9000 2024-09-19T05:42:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f195dffb640 1 -- 172.21.15.123:0/4236799082 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958007c90 con 0x7f19600c9000 2024-09-19T05:42:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f196532c640 1 -- 172.21.15.123:0/4236799082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19600c9000 msgr2=0x7f196006bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f196532c640 1 --2- 172.21.15.123:0/4236799082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19600c9000 0x7f196006bf20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1958009d90 tx=0x7f195802a6f0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f196532c640 1 -- 172.21.15.123:0/4236799082 shutdown_connections 2024-09-19T05:42:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f196532c640 1 --2- 172.21.15.123:0/4236799082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19600c9000 0x7f196006bf20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f196532c640 1 -- 172.21.15.123:0/4236799082 >> 172.21.15.123:0/4236799082 conn(0x7f19600cd4d0 msgr2=0x7f19600cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f196532c640 1 -- 172.21.15.123:0/4236799082 shutdown_connections 2024-09-19T05:42:10.358 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.354+0000 7f196532c640 1 -- 172.21.15.123:0/4236799082 wait complete. 2024-09-19T05:42:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 Processor -- start 2024-09-19T05:42:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 -- start start 2024-09-19T05:42:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19600706a0 0x7f196014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19600c9000 0x7f1960157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.360 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1960155040 0x7f1960158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f196006d3a0 con 0x7f1960155040 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f196006d0c0 con 0x7f19600706a0 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.356+0000 7f196532c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f19600c8240 con 0x7f19600c9000 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19600706a0 0x7f196014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195e7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19600c9000 0x7f1960157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19600706a0 0x7f196014c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44004/0 (socket says 172.21.15.123:44004) 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195e7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19600c9000 0x7f1960157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:39998/0 (socket says 172.21.15.123:39998) 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 -- 172.21.15.123:0/875764700 learned_addr learned my addr 172.21.15.123:0/875764700 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195f7fe640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1960155040 0x7f1960158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 -- 172.21.15.123:0/875764700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19600c9000 msgr2=0x7f1960157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19600c9000 0x7f1960157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 -- 172.21.15.123:0/875764700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1960155040 msgr2=0x7f1960158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.361 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1960155040 0x7f1960158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 -- 172.21.15.123:0/875764700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196014fd80 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f195effd640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19600706a0 0x7f196014c670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f195400c440 tx=0x7f195400c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f19477fe640 1 -- 172.21.15.123:0/875764700 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954017070 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f19477fe640 1 -- 172.21.15.123:0/875764700 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1954005d00 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196014faa0 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f19477fe640 1 -- 172.21.15.123:0/875764700 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540072b0 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601f43c0 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f19477fe640 1 -- 172.21.15.123:0/875764700 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f195401a070 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.357+0000 7f19477fe640 1 -- 172.21.15.123:0/875764700 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f1954012740 con 0x7f19600706a0 2024-09-19T05:42:10.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.358+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196015e150 con 0x7f19600706a0 2024-09-19T05:42:10.363 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.360+0000 7f19477fe640 1 -- 172.21.15.123:0/875764700 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1954021020 con 0x7f19600706a0 2024-09-19T05:42:10.372 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:10.370+0000 7f3085342040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:42:10.396 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:10.395+0000 7f8cc7836040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T05:42:10.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.428+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "665c2ac5-a964-4f38-8fa6-f4e95cac9272", "id": 0} v 0) -- 0x7f19600715c0 con 0x7f19600706a0 2024-09-19T05:42:10.436 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.434+0000 7f19477fe640 1 -- 172.21.15.123:0/875764700 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "665c2ac5-a964-4f38-8fa6-f4e95cac9272", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f19540076c0 con 0x7f19600706a0 2024-09-19T05:42:10.436 INFO:teuthology.orchestra.run.smithi123.stdout:0 2024-09-19T05:42:10.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19600706a0 msgr2=0x7f196014c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19600706a0 0x7f196014c670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f195400c440 tx=0x7f195400c900 comp rx=0 tx=0).stop 2024-09-19T05:42:10.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 shutdown_connections 2024-09-19T05:42:10.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1960155040 0x7f1960158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.437 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19600c9000 0x7f1960157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 --2- 172.21.15.123:0/875764700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19600706a0 0x7f196014c670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 >> 172.21.15.123:0/875764700 conn(0x7f19600cd4d0 msgr2=0x7f19600cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:10.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 shutdown_connections 2024-09-19T05:42:10.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.435+0000 7f196532c640 1 -- 172.21.15.123:0/875764700 wait complete. 2024-09-19T05:42:10.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new a4443f63-2c47-4c6f-a2df-6765b78afcc2 1 2024-09-19T05:42:10.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.542+0000 7f2c2dcc0640 1 Processor -- start 2024-09-19T05:42:10.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c2dcc0640 1 -- start start 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c2dcc0640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c28152ba0 0x7f2c281588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c2dcc0640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c280c3f10 con 0x7f2c280cbb10 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c2dcc0640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c280cb6f0 con 0x7f2c280c3ae0 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c2dcc0640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2c280c9bb0 con 0x7f2c28152ba0 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c277fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c28152ba0 0x7f2c281588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c27fff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2c280c3ae0 0x7f2c280718a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47720/0 (socket says 172.21.15.123:47720) 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c277fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c28152ba0 0x7f2c281588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44018/0 (socket says 172.21.15.123:44018) 2024-09-19T05:42:10.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.543+0000 7f2c27fff640 1 -- 172.21.15.123:0/920517041 learned_addr learned my addr 172.21.15.123:0/920517041 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-19T05:42:10.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.544+0000 7f2c277fe640 1 -- 172.21.15.123:0/920517041 >> v1:172.21.15.123:6789/0 conn(0x7f2c280c3ae0 legacy=0x7f2c280718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.544+0000 7f2c277fe640 1 -- 172.21.15.123:0/920517041 >> v1:172.21.15.2:6789/0 conn(0x7f2c280cbb10 legacy=0x7f2c280c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:10.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.544+0000 7f2c277fe640 1 -- 172.21.15.123:0/920517041 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c280c9e00 con 0x7f2c28152ba0 2024-09-19T05:42:10.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.544+0000 7f2c277fe640 1 --2- 172.21.15.123:0/920517041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c281588f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009720 tx=0x7f2c1c029f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c309155e0695304 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c267fc640 1 -- 172.21.15.123:0/920517041 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c037070 con 0x7f2c28152ba0 2024-09-19T05:42:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c267fc640 1 -- 172.21.15.123:0/920517041 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c02ac50 con 0x7f2c28152ba0 2024-09-19T05:42:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c267fc640 1 -- 172.21.15.123:0/920517041 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c032440 con 0x7f2c28152ba0 2024-09-19T05:42:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/920517041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 msgr2=0x7f2c281588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c2dcc0640 1 --2- 172.21.15.123:0/920517041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c281588f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009720 tx=0x7f2c1c029f40 comp rx=0 tx=0).stop 2024-09-19T05:42:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/920517041 shutdown_connections 2024-09-19T05:42:10.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c2dcc0640 1 --2- 172.21.15.123:0/920517041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c281588f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/920517041 >> 172.21.15.123:0/920517041 conn(0x7f2c2806bdf0 msgr2=0x7f2c2806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:10.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/920517041 shutdown_connections 2024-09-19T05:42:10.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.545+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/920517041 wait complete. 2024-09-19T05:42:10.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.547+0000 7f2c2dcc0640 1 Processor -- start 2024-09-19T05:42:10.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.547+0000 7f2c2dcc0640 1 -- start start 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c2dcc0640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c280c3ae0 0x7f2c28155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c2dcc0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c280cbb10 0x7f2c281e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c2dcc0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c280c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c2dcc0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c280ca3c0 con 0x7f2c280cbb10 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c2dcc0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c280c35c0 con 0x7f2c28152ba0 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c2dcc0640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2c280cb320 con 0x7f2c280c3ae0 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c280c3ae0 0x7f2c28155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c27fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c280c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c27fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c280c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44026/0 (socket says 172.21.15.123:44026) 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c280c3ae0 0x7f2c28155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40010/0 (socket says 172.21.15.123:40010) 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c27fff640 1 -- 172.21.15.123:0/3428934687 learned_addr learned my addr 172.21.15.123:0/3428934687 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c26ffd640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c280cbb10 0x7f2c281e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 -- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 msgr2=0x7f2c280c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c280c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 -- 172.21.15.123:0/3428934687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c280cbb10 msgr2=0x7f2c281e2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c280cbb10 0x7f2c281e2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 -- 172.21.15.123:0/3428934687 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c281f2310 con 0x7f2c280c3ae0 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c27fff640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c280c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c277fe640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c280c3ae0 0x7f2c28155a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c031000 tx=0x7f2c1c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c0ffff640 1 -- 172.21.15.123:0/3428934687 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c037040 con 0x7f2c280c3ae0 2024-09-19T05:42:10.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c0ffff640 1 -- 172.21.15.123:0/3428934687 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c004020 con 0x7f2c280c3ae0 2024-09-19T05:42:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c0ffff640 1 -- 172.21.15.123:0/3428934687 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c004410 con 0x7f2c280c3ae0 2024-09-19T05:42:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.549+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c281f25c0 con 0x7f2c280c3ae0 2024-09-19T05:42:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.550+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c280b3640 con 0x7f2c280c3ae0 2024-09-19T05:42:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.550+0000 7f2c0ffff640 1 -- 172.21.15.123:0/3428934687 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c1c03a070 con 0x7f2c280c3ae0 2024-09-19T05:42:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.550+0000 7f2c0ffff640 1 -- 172.21.15.123:0/3428934687 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f2c1c0039c0 con 0x7f2c280c3ae0 2024-09-19T05:42:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.550+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c281588f0 con 0x7f2c280c3ae0 2024-09-19T05:42:10.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.553+0000 7f2c0ffff640 1 -- 172.21.15.123:0/3428934687 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2c1c04a030 con 0x7f2c280c3ae0 2024-09-19T05:42:10.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.621+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4443f63-2c47-4c6f-a2df-6765b78afcc2", "id": 1} v 0) -- 0x7f2c280c89d0 con 0x7f2c280c3ae0 2024-09-19T05:42:10.630 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c0ffff640 1 -- 172.21.15.123:0/3428934687 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4443f63-2c47-4c6f-a2df-6765b78afcc2", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f2c1c041030 con 0x7f2c280c3ae0 2024-09-19T05:42:10.631 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c280c3ae0 msgr2=0x7f2c28155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c2dcc0640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c280c3ae0 0x7f2c28155a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c031000 tx=0x7f2c1c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 shutdown_connections 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c2dcc0640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c28152ba0 0x7f2c280c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c2dcc0640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c280cbb10 0x7f2c281e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c2dcc0640 1 --2- 172.21.15.123:0/3428934687 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c280c3ae0 0x7f2c28155a70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.629+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 >> 172.21.15.123:0/3428934687 conn(0x7f2c2806bdf0 msgr2=0x7f2c280c8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.630+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 shutdown_connections 2024-09-19T05:42:10.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.630+0000 7f2c2dcc0640 1 -- 172.21.15.123:0/3428934687 wait complete. 2024-09-19T05:42:10.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 61465313-1ba1-4f26-b488-6ea73981419b 2 2024-09-19T05:42:10.683 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:10.682+0000 7f3085342040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:42:10.734 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:10.732+0000 7f3085342040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:42:10.734 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:10.733+0000 7f8cc7836040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T05:42:10.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.738+0000 7f24de2ac640 1 Processor -- start 2024-09-19T05:42:10.740 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.738+0000 7f24de2ac640 1 -- start start 2024-09-19T05:42:10.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24de2ac640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d80c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24de2ac640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24d806f2b0 0x7f24d80c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24de2ac640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24d80c3990 0x7f24d81515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24de2ac640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f24d8158530 con 0x7f24d80c3990 2024-09-19T05:42:10.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24de2ac640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f24d80c3e80 con 0x7f24d806f2b0 2024-09-19T05:42:10.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24de2ac640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f24d8157c00 con 0x7f24d806eed0 2024-09-19T05:42:10.742 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d80c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d77fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24d806f2b0 0x7f24d80c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d80c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40012/0 (socket says 172.21.15.123:40012) 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d77fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24d806f2b0 0x7f24d80c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44036/0 (socket says 172.21.15.123:44036) 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 -- 172.21.15.123:0/4240487237 learned_addr learned my addr 172.21.15.123:0/4240487237 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24dc849640 1 --2- 172.21.15.123:0/4240487237 >> v2:172.21.15.2:3300/0 conn(0x7f24d80c3990 0x7f24d81515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 -- 172.21.15.123:0/4240487237 >> v2:172.21.15.123:3300/0 conn(0x7f24d806f2b0 msgr2=0x7f24d80c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 --2- 172.21.15.123:0/4240487237 >> v2:172.21.15.123:3300/0 conn(0x7f24d806f2b0 0x7f24d80c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 -- 172.21.15.123:0/4240487237 >> v2:172.21.15.2:3300/0 conn(0x7f24d80c3990 msgr2=0x7f24d81515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 --2- 172.21.15.123:0/4240487237 >> v2:172.21.15.2:3300/0 conn(0x7f24d80c3990 0x7f24d81515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d7fff640 1 -- 172.21.15.123:0/4240487237 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d8157ee0 con 0x7f24d806eed0 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.739+0000 7f24d77fe640 1 --2- 172.21.15.123:0/4240487237 >> v2:172.21.15.123:3300/0 conn(0x7f24d806f2b0 0x7f24d80c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.740+0000 7f24d7fff640 1 --2- 172.21.15.123:0/4240487237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d80c77e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f24d8067c20 tx=0x7f24c802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283ad5053c03cf02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.743 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.740+0000 7f24d6ffd640 1 -- 172.21.15.123:0/4240487237 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8037070 con 0x7f24d806eed0 2024-09-19T05:42:10.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.740+0000 7f24d6ffd640 1 -- 172.21.15.123:0/4240487237 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c8032440 con 0x7f24d806eed0 2024-09-19T05:42:10.744 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.740+0000 7f24d6ffd640 1 -- 172.21.15.123:0/4240487237 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8032810 con 0x7f24d806eed0 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 -- 172.21.15.123:0/4240487237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 msgr2=0x7f24d80c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4240487237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d80c77e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f24d8067c20 tx=0x7f24c802a1e0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 -- 172.21.15.123:0/4240487237 shutdown_connections 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4240487237 >> v2:172.21.15.2:3300/0 conn(0x7f24d80c3990 0x7f24d81515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4240487237 >> v2:172.21.15.123:3300/0 conn(0x7f24d806f2b0 0x7f24d80c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4240487237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d80c77e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 -- 172.21.15.123:0/4240487237 >> 172.21.15.123:0/4240487237 conn(0x7f24d806fed0 msgr2=0x7f24d80702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 -- 172.21.15.123:0/4240487237 shutdown_connections 2024-09-19T05:42:10.747 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.741+0000 7f24de2ac640 1 -- 172.21.15.123:0/4240487237 wait complete. 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.742+0000 7f24de2ac640 1 Processor -- start 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.743+0000 7f24de2ac640 1 -- start start 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.743+0000 7f24de2ac640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.743+0000 7f24de2ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24d806f2b0 0x7f24d814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.743+0000 7f24de2ac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d80c3990 0x7f24d81ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.743+0000 7f24de2ac640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81578b0 con 0x7f24d806f2b0 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.743+0000 7f24de2ac640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24d806d370 con 0x7f24d80c3990 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.743+0000 7f24de2ac640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f24d80c9450 con 0x7f24d806eed0 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d8156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40018/0 (socket says 172.21.15.123:40018) 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 -- 172.21.15.123:0/4101795470 learned_addr learned my addr 172.21.15.123:0/4101795470 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24dc849640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d80c3990 0x7f24d81ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d77fe640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24d806f2b0 0x7f24d814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 -- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d80c3990 msgr2=0x7f24d81ef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d80c3990 0x7f24d81ef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 -- 172.21.15.123:0/4101795470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24d806f2b0 msgr2=0x7f24d814cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24d806f2b0 0x7f24d814cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 -- 172.21.15.123:0/4101795470 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d81eea90 con 0x7f24d806eed0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24dc849640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d80c3990 0x7f24d81ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d7fff640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d8156bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24c8000c00 tx=0x7f24c80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d4ff9640 1 -- 172.21.15.123:0/4101795470 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8041020 con 0x7f24d806eed0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d4ff9640 1 -- 172.21.15.123:0/4101795470 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c803d620 con 0x7f24d806eed0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d814daa0 con 0x7f24d806eed0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d4ff9640 1 -- 172.21.15.123:0/4101795470 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8037040 con 0x7f24d806eed0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81565b0 con 0x7f24d806eed0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d4ff9640 1 -- 172.21.15.123:0/4101795470 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f24c8044020 con 0x7f24d806eed0 2024-09-19T05:42:10.749 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.744+0000 7f24d4ff9640 1 -- 172.21.15.123:0/4101795470 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f24c803b3f0 con 0x7f24d806eed0 2024-09-19T05:42:10.750 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.745+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24d80c1e90 con 0x7f24d806eed0 2024-09-19T05:42:10.751 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.749+0000 7f24d4ff9640 1 -- 172.21.15.123:0/4101795470 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f24c804b030 con 0x7f24d806eed0 2024-09-19T05:42:10.789 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:10.789+0000 7f8cc7836040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T05:42:10.820 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.818+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "61465313-1ba1-4f26-b488-6ea73981419b", "id": 2} v 0) -- 0x7f24d80b2bf0 con 0x7f24d806eed0 2024-09-19T05:42:10.824 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.822+0000 7f24d4ff9640 1 -- 172.21.15.123:0/4101795470 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "61465313-1ba1-4f26-b488-6ea73981419b", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f24c8046bd0 con 0x7f24d806eed0 2024-09-19T05:42:10.824 INFO:teuthology.orchestra.run.smithi123.stdout:2 2024-09-19T05:42:10.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 msgr2=0x7f24d8156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d8156bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f24c8000c00 tx=0x7f24c80046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 shutdown_connections 2024-09-19T05:42:10.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24d80c3990 0x7f24d81ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24d806f2b0 0x7f24d814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.825 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 --2- 172.21.15.123:0/4101795470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24d806eed0 0x7f24d8156bf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 >> 172.21.15.123:0/4101795470 conn(0x7f24d806fed0 msgr2=0x7f24d80702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:10.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 shutdown_connections 2024-09-19T05:42:10.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.823+0000 7f24de2ac640 1 -- 172.21.15.123:0/4101795470 wait complete. 2024-09-19T05:42:10.835 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new fa9d27d5-68d4-4467-a2a5-79f767a071aa 3 2024-09-19T05:42:10.894 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:10.892+0000 7f3085342040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:42:10.947 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:10.946+0000 7f8cc7836040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T05:42:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.973+0000 7f97c38a6640 1 Processor -- start 2024-09-19T05:42:10.975 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.973+0000 7f97c38a6640 1 -- start start 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c38a6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97bc0c9700 0x7f97bc0c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c38a6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f97bc159050 0x7f97bc0c7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c38a6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97bc06c1a0 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c38a6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc06c480 con 0x7f97bc159050 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c38a6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc06c760 con 0x7f97bc0c9700 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1642640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97bc0c9700 0x7f97bc0c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1642640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97bc0c9700 0x7f97bc0c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44060/0 (socket says 172.21.15.123:44060) 2024-09-19T05:42:10.977 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1642640 1 -- 172.21.15.123:0/1849067537 learned_addr learned my addr 172.21.15.123:0/1849067537 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c0e41640 1 --1- 172.21.15.123:0/1849067537 >> v1:172.21.15.123:6789/0 conn(0x7f97bc0c9ae0 0x7f97bc158940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:47732/0 (socket says 172.21.15.123:47732) 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1e43640 1 --2- 172.21.15.123:0/1849067537 >> v2:172.21.15.2:3300/0 conn(0x7f97bc159050 0x7f97bc0c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1642640 1 -- 172.21.15.123:0/1849067537 >> v2:172.21.15.2:3300/0 conn(0x7f97bc159050 msgr2=0x7f97bc0c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1642640 1 --2- 172.21.15.123:0/1849067537 >> v2:172.21.15.2:3300/0 conn(0x7f97bc159050 0x7f97bc0c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1642640 1 -- 172.21.15.123:0/1849067537 >> v1:172.21.15.123:6789/0 conn(0x7f97bc0c9ae0 legacy=0x7f97bc158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97b3fff640 1 -- 172.21.15.123:0/1849067537 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423528452 0 0) 0x7f97bc06c1a0 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.974+0000 7f97c1642640 1 -- 172.21.15.123:0/1849067537 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc073630 con 0x7f97bc0c9700 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.975+0000 7f97c1642640 1 --2- 172.21.15.123:0/1849067537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9700 0x7f97bc0c6be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97bc071730 tx=0x7f97b802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31627d142c7a19bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.975+0000 7f97b3fff640 1 -- 172.21.15.123:0/1849067537 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8037070 con 0x7f97bc0c9700 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.975+0000 7f97b3fff640 1 -- 172.21.15.123:0/1849067537 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b802ae50 con 0x7f97bc0c9700 2024-09-19T05:42:10.978 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.975+0000 7f97b3fff640 1 -- 172.21.15.123:0/1849067537 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8032860 con 0x7f97bc0c9700 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 -- 172.21.15.123:0/1849067537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9700 msgr2=0x7f97bc0c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 --2- 172.21.15.123:0/1849067537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9700 0x7f97bc0c6be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97bc071730 tx=0x7f97b802a300 comp rx=0 tx=0).stop 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 -- 172.21.15.123:0/1849067537 shutdown_connections 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 --2- 172.21.15.123:0/1849067537 >> v2:172.21.15.2:3300/0 conn(0x7f97bc159050 0x7f97bc0c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 --2- 172.21.15.123:0/1849067537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9700 0x7f97bc0c6be0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 -- 172.21.15.123:0/1849067537 >> 172.21.15.123:0/1849067537 conn(0x7f97bc0cd4d0 msgr2=0x7f97bc0cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 -- 172.21.15.123:0/1849067537 shutdown_connections 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.976+0000 7f97c38a6640 1 -- 172.21.15.123:0/1849067537 wait complete. 2024-09-19T05:42:10.981 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.977+0000 7f97c38a6640 1 Processor -- start 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.977+0000 7f97c38a6640 1 -- start start 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c38a6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97bc0c9700 0x7f97bc1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c38a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9ae0 0x7f97bc14b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c38a6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97bc159050 0x7f97bc14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c38a6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc06bd90 con 0x7f97bc0c9700 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c38a6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc06bc10 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c38a6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc06bf10 con 0x7f97bc159050 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9ae0 0x7f97bc14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c1e43640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97bc159050 0x7f97bc14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9ae0 0x7f97bc14b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44070/0 (socket says 172.21.15.123:44070) 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 -- 172.21.15.123:0/1120191020 learned_addr learned my addr 172.21.15.123:0/1120191020 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c1e43640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97bc159050 0x7f97bc14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40034/0 (socket says 172.21.15.123:40034) 2024-09-19T05:42:10.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c1642640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97bc0c9700 0x7f97bc1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 -- 172.21.15.123:0/1120191020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97bc159050 msgr2=0x7f97bc14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97bc159050 0x7f97bc14ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 -- 172.21.15.123:0/1120191020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97bc0c9700 msgr2=0x7f97bc1f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97bc0c9700 0x7f97bc1f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 -- 172.21.15.123:0/1120191020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc1563a0 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c0e41640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9ae0 0x7f97bc14b850 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c500 tx=0x7f97ac00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97b1ffb640 1 -- 172.21.15.123:0/1120191020 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac018070 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97b1ffb640 1 -- 172.21.15.123:0/1120191020 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97ac007580 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc155de0 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.978+0000 7f97b1ffb640 1 -- 172.21.15.123:0/1120191020 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac004a20 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.979+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc14ad70 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.979+0000 7f97b1ffb640 1 -- 172.21.15.123:0/1120191020 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f97ac01b070 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.983 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.979+0000 7f97b1ffb640 1 -- 172.21.15.123:0/1120191020 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f97ac01c840 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.980+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc158940 con 0x7f97bc0c9ae0 2024-09-19T05:42:10.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:10.982+0000 7f97b1ffb640 1 -- 172.21.15.123:0/1120191020 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f97ac01ce20 con 0x7f97bc0c9ae0 2024-09-19T05:42:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.051+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fa9d27d5-68d4-4467-a2a5-79f767a071aa", "id": 3} v 0) -- 0x7f97bc157250 con 0x7f97bc0c9ae0 2024-09-19T05:42:11.057 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.055+0000 7f97b1ffb640 1 -- 172.21.15.123:0/1120191020 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa9d27d5-68d4-4467-a2a5-79f767a071aa", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f97ac01c3b0 con 0x7f97bc0c9ae0 2024-09-19T05:42:11.057 INFO:teuthology.orchestra.run.smithi123.stdout:3 2024-09-19T05:42:11.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9ae0 msgr2=0x7f97bc14b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9ae0 0x7f97bc14b850 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c500 tx=0x7f97ac00c9c0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 shutdown_connections 2024-09-19T05:42:11.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97bc0c9ae0 0x7f97bc14b850 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f97bc159050 0x7f97bc14ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 --2- 172.21.15.123:0/1120191020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97bc0c9700 0x7f97bc1f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.058 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 >> 172.21.15.123:0/1120191020 conn(0x7f97bc0cd4d0 msgr2=0x7f97bc1565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 shutdown_connections 2024-09-19T05:42:11.059 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.056+0000 7f97c38a6640 1 -- 172.21.15.123:0/1120191020 wait complete. 2024-09-19T05:42:11.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 85a040ff-ad8b-44ff-b82e-a36b05d83087 4 2024-09-19T05:42:11.205 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.203+0000 7fac6ba50640 1 Processor -- start 2024-09-19T05:42:11.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.204+0000 7fac6ba50640 1 -- start start 2024-09-19T05:42:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.204+0000 7fac6ba50640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac641584c0 0x7fac640c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.204+0000 7fac6ba50640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac640c86f0 0x7fac6415e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.204+0000 7fac6ba50640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac640725c0 con 0x7fac64073660 2024-09-19T05:42:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.204+0000 7fac6ba50640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fac64072370 con 0x7fac641584c0 2024-09-19T05:42:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.204+0000 7fac6ba50640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac640728a0 con 0x7fac640c86f0 2024-09-19T05:42:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac697ec640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac641584c0 0x7fac640c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac69fed640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac640c86f0 0x7fac6415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac697ec640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac641584c0 0x7fac640c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44080/0 (socket says 172.21.15.123:44080) 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac69fed640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac640c86f0 0x7fac6415e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40050/0 (socket says 172.21.15.123:40050) 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac697ec640 1 -- 172.21.15.123:0/2795860028 learned_addr learned my addr 172.21.15.123:0/2795860028 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac697ec640 1 -- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac640c86f0 msgr2=0x7fac6415e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac697ec640 1 --2- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac640c86f0 0x7fac6415e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac697ec640 1 -- 172.21.15.123:0/2795860028 >> v1:172.21.15.2:6789/0 conn(0x7fac64073660 legacy=0x7fac6406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac697ec640 1 -- 172.21.15.123:0/2795860028 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac64072b80 con 0x7fac641584c0 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.205+0000 7fac69fed640 1 --2- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac640c86f0 0x7fac6415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac697ec640 1 --2- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac640c4400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac58009870 tx=0x7fac5802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9c632629f4d27d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.211 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac57fff640 1 -- 172.21.15.123:0/2795860028 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58037070 con 0x7fac641584c0 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac57fff640 1 -- 172.21.15.123:0/2795860028 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac58032440 con 0x7fac641584c0 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac57fff640 1 -- 172.21.15.123:0/2795860028 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58032810 con 0x7fac641584c0 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 -- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 msgr2=0x7fac640c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 --2- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac640c4400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac58009870 tx=0x7fac5802a130 comp rx=0 tx=0).stop 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 -- 172.21.15.123:0/2795860028 shutdown_connections 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 --2- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac640c86f0 0x7fac6415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 --2- 172.21.15.123:0/2795860028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac640c4400 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 -- 172.21.15.123:0/2795860028 >> 172.21.15.123:0/2795860028 conn(0x7fac640cd470 msgr2=0x7fac640c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 -- 172.21.15.123:0/2795860028 shutdown_connections 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.206+0000 7fac6ba50640 1 -- 172.21.15.123:0/2795860028 wait complete. 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.207+0000 7fac6ba50640 1 Processor -- start 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac6ba50640 1 -- start start 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac6ba50640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac64073660 0x7fac6414c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac6ba50640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac640c86f0 0x7fac640c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac6ba50640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac64152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac6ba50640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac641588a0 con 0x7fac640c86f0 2024-09-19T05:42:11.213 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac6ba50640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac64074120 con 0x7fac641584c0 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac6ba50640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac64158a20 con 0x7fac64073660 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac697ec640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac64073660 0x7fac6414c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac69fed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac64152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac697ec640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac64073660 0x7fac6414c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40058/0 (socket says 172.21.15.123:40058) 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac697ec640 1 -- 172.21.15.123:0/481706457 learned_addr learned my addr 172.21.15.123:0/481706457 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.208+0000 7fac68feb640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac640c86f0 0x7fac640c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac697ec640 1 -- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 msgr2=0x7fac64152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac697ec640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac64152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac697ec640 1 -- 172.21.15.123:0/481706457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac640c86f0 msgr2=0x7fac640c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac697ec640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac640c86f0 0x7fac640c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac697ec640 1 -- 172.21.15.123:0/481706457 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6414bce0 con 0x7fac64073660 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac69fed640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac64152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.214 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac697ec640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac64073660 0x7fac6414c2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac58000c00 tx=0x7fac580046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac55ffb640 1 -- 172.21.15.123:0/481706457 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58037040 con 0x7fac64073660 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac55ffb640 1 -- 172.21.15.123:0/481706457 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac580304c0 con 0x7fac64073660 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac55ffb640 1 -- 172.21.15.123:0/481706457 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58030890 con 0x7fac64073660 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6414d4d0 con 0x7fac64073660 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fac640c63f0 con 0x7fac64073660 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac55ffb640 1 -- 172.21.15.123:0/481706457 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac58044030 con 0x7fac64073660 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.209+0000 7fac55ffb640 1 -- 172.21.15.123:0/481706457 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fac5803bb60 con 0x7fac64073660 2024-09-19T05:42:11.215 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.210+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac64155890 con 0x7fac64073660 2024-09-19T05:42:11.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.212+0000 7fac55ffb640 1 -- 172.21.15.123:0/481706457 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fac5804b060 con 0x7fac64073660 2024-09-19T05:42:11.295 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.293+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "85a040ff-ad8b-44ff-b82e-a36b05d83087", "id": 4} v 0) -- 0x7fac640c0ac0 con 0x7fac64073660 2024-09-19T05:42:11.298 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.296+0000 7fac55ffb640 1 -- 172.21.15.123:0/481706457 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "85a040ff-ad8b-44ff-b82e-a36b05d83087", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fac5803bdd0 con 0x7fac64073660 2024-09-19T05:42:11.298 INFO:teuthology.orchestra.run.smithi123.stdout:4 2024-09-19T05:42:11.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac64073660 msgr2=0x7fac6414c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac64073660 0x7fac6414c2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fac58000c00 tx=0x7fac580046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 shutdown_connections 2024-09-19T05:42:11.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac641584c0 0x7fac64152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.300 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac640c86f0 0x7fac640c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 --2- 172.21.15.123:0/481706457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac64073660 0x7fac6414c2f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 >> 172.21.15.123:0/481706457 conn(0x7fac640cd470 msgr2=0x7fac640cb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 shutdown_connections 2024-09-19T05:42:11.301 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.297+0000 7fac6ba50640 1 -- 172.21.15.123:0/481706457 wait complete. 2024-09-19T05:42:11.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new f90f8fa4-4afc-4272-89eb-5e90123155d9 5 2024-09-19T05:42:11.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.407+0000 7f54531e8640 1 Processor -- start 2024-09-19T05:42:11.409 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.407+0000 7f54531e8640 1 -- start start 2024-09-19T05:42:11.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f54531e8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f544c070cf0 0x7f544c073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f54531e8640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544c0c6430 con 0x7f544c073300 2024-09-19T05:42:11.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f54531e8640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544c073d90 con 0x7f544c070910 2024-09-19T05:42:11.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f54531e8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f544c15b530 con 0x7f544c070cf0 2024-09-19T05:42:11.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f5450f84640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f544c070910 0x7f544c15ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48828/0 (socket says 172.21.15.123:48828) 2024-09-19T05:42:11.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f5450f84640 1 -- 172.21.15.123:0/3184482475 learned_addr learned my addr 172.21.15.123:0/3184482475 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-19T05:42:11.410 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f5451785640 1 --2- 172.21.15.123:0/3184482475 >> v2:172.21.15.2:3300/0 conn(0x7f544c070cf0 0x7f544c073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f544b7fe640 1 -- 172.21.15.123:0/3184482475 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2767439445 0 0) 0x7f544c073d90 con 0x7f544c070910 2024-09-19T05:42:11.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f544b7fe640 1 -- 172.21.15.123:0/3184482475 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f543c003520 con 0x7f544c070910 2024-09-19T05:42:11.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f5451785640 1 -- 172.21.15.123:0/3184482475 >> v1:172.21.15.123:6789/0 conn(0x7f544c070910 legacy=0x7f544c15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f5451785640 1 -- 172.21.15.123:0/3184482475 >> v1:172.21.15.2:6789/0 conn(0x7f544c073300 legacy=0x7f544c151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.408+0000 7f5451785640 1 -- 172.21.15.123:0/3184482475 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c157830 con 0x7f544c070cf0 2024-09-19T05:42:11.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.409+0000 7f5451785640 1 --2- 172.21.15.123:0/3184482475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c073700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5444009d90 tx=0x7f544402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a39a08f920cef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.409+0000 7f544b7fe640 1 -- 172.21.15.123:0/3184482475 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444007740 con 0x7f544c070cf0 2024-09-19T05:42:11.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.409+0000 7f544b7fe640 1 -- 172.21.15.123:0/3184482475 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54440078e0 con 0x7f544c070cf0 2024-09-19T05:42:11.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.409+0000 7f544b7fe640 1 -- 172.21.15.123:0/3184482475 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444007cb0 con 0x7f544c070cf0 2024-09-19T05:42:11.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.410+0000 7f54531e8640 1 -- 172.21.15.123:0/3184482475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 msgr2=0x7f544c073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.410+0000 7f54531e8640 1 --2- 172.21.15.123:0/3184482475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c073700 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5444009d90 tx=0x7f544402a6f0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.410+0000 7f54531e8640 1 -- 172.21.15.123:0/3184482475 shutdown_connections 2024-09-19T05:42:11.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.410+0000 7f54531e8640 1 --2- 172.21.15.123:0/3184482475 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c073700 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.410+0000 7f54531e8640 1 -- 172.21.15.123:0/3184482475 >> 172.21.15.123:0/3184482475 conn(0x7f544c0cd4d0 msgr2=0x7f544c0cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.410+0000 7f54531e8640 1 -- 172.21.15.123:0/3184482475 shutdown_connections 2024-09-19T05:42:11.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.410+0000 7f54531e8640 1 -- 172.21.15.123:0/3184482475 wait complete. 2024-09-19T05:42:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.411+0000 7f54531e8640 1 Processor -- start 2024-09-19T05:42:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.411+0000 7f54531e8640 1 -- start start 2024-09-19T05:42:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f54531e8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f544c070910 0x7f544c154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f54531e8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.415 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f54531e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c073300 0x7f544c1f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f54531e8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f544c157dd0 con 0x7f544c070cf0 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f54531e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f544c157af0 con 0x7f544c073300 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f54531e8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f544c1585d0 con 0x7f544c070910 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5451785640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c073300 0x7f544c1f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5450f84640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f544c070910 0x7f544c154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5451785640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c073300 0x7f544c1f13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58160/0 (socket says 172.21.15.123:58160) 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5451785640 1 -- 172.21.15.123:0/3225454445 learned_addr learned my addr 172.21.15.123:0/3225454445 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f544bfff640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5450f84640 1 -- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c073300 msgr2=0x7f544c1f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5450f84640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c073300 0x7f544c1f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5450f84640 1 -- 172.21.15.123:0/3225454445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 msgr2=0x7f544c164900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5450f84640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c164900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.416 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5450f84640 1 -- 172.21.15.123:0/3225454445 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c154b00 con 0x7f544c070910 2024-09-19T05:42:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f544bfff640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5451785640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c073300 0x7f544c1f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.412+0000 7f5450f84640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f544c070910 0x7f544c154120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f544000c430 tx=0x7f544000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.413+0000 7f54497fa640 1 -- 172.21.15.123:0/3225454445 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440017070 con 0x7f544c070910 2024-09-19T05:42:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.413+0000 7f54497fa640 1 -- 172.21.15.123:0/3225454445 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5440005ce0 con 0x7f544c070910 2024-09-19T05:42:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.413+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544c0cbef0 con 0x7f544c070910 2024-09-19T05:42:11.417 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.413+0000 7f54497fa640 1 -- 172.21.15.123:0/3225454445 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54400072d0 con 0x7f544c070910 2024-09-19T05:42:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.413+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f544c15ae70 con 0x7f544c070910 2024-09-19T05:42:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.413+0000 7f54497fa640 1 -- 172.21.15.123:0/3225454445 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f544001a070 con 0x7f544c070910 2024-09-19T05:42:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.413+0000 7f54497fa640 1 -- 172.21.15.123:0/3225454445 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f544001ccb0 con 0x7f544c070910 2024-09-19T05:42:11.418 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.414+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544c0c4240 con 0x7f544c070910 2024-09-19T05:42:11.419 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.416+0000 7f54497fa640 1 -- 172.21.15.123:0/3225454445 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5440007470 con 0x7f544c070910 2024-09-19T05:42:11.487 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.485+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f90f8fa4-4afc-4272-89eb-5e90123155d9", "id": 5} v 0) -- 0x7f544c154cd0 con 0x7f544c070910 2024-09-19T05:42:11.490 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.488+0000 7f54497fa640 1 -- 172.21.15.123:0/3225454445 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f90f8fa4-4afc-4272-89eb-5e90123155d9", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5440007650 con 0x7f544c070910 2024-09-19T05:42:11.490 INFO:teuthology.orchestra.run.smithi123.stdout:5 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f544c070910 msgr2=0x7f544c154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f544c070910 0x7f544c154120 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f544000c430 tx=0x7f544000c8f0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 shutdown_connections 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544c073300 0x7f544c1f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f544c070cf0 0x7f544c164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 --2- 172.21.15.123:0/3225454445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f544c070910 0x7f544c154120 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 >> 172.21.15.123:0/3225454445 conn(0x7f544c0cd4d0 msgr2=0x7f544c14f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 shutdown_connections 2024-09-19T05:42:11.491 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.489+0000 7f54531e8640 1 -- 172.21.15.123:0/3225454445 wait complete. 2024-09-19T05:42:11.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 48561580-ba9a-4096-9722-16c308f414ee 6 2024-09-19T05:42:11.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.598+0000 7f2985926640 1 Processor -- start 2024-09-19T05:42:11.600 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.598+0000 7f2985926640 1 -- start start 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f2985926640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f298006ce90 0x7f29801593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f2985926640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29800c7ea0 0x7f29800c8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f2985926640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29800c8850 con 0x7f298006d270 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f2985926640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f298006eda0 con 0x7f29800c7ea0 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f2985926640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2980159920 con 0x7f298006ce90 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297effd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f298006ce90 0x7f29801593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297f7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29800c7ea0 0x7f29800c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297f7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29800c7ea0 0x7f29800c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58176/0 (socket says 172.21.15.123:58176) 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297effd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f298006ce90 0x7f29801593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40856/0 (socket says 172.21.15.123:40856) 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297f7fe640 1 -- 172.21.15.123:0/2953309340 learned_addr learned my addr 172.21.15.123:0/2953309340 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:11.602 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297f7fe640 1 -- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f298006ce90 msgr2=0x7f29801593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297f7fe640 1 --2- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f298006ce90 0x7f29801593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297f7fe640 1 -- 172.21.15.123:0/2953309340 >> v1:172.21.15.2:6789/0 conn(0x7f298006d270 legacy=0x7f298006da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297f7fe640 1 -- 172.21.15.123:0/2953309340 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298006e220 con 0x7f29800c7ea0 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.599+0000 7f297effd640 1 --2- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f298006ce90 0x7f29801593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.600+0000 7f297f7fe640 1 --2- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29800c7ea0 0x7f29800c8310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2978009e50 tx=0x7f297802a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e263162524fd100c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.600+0000 7f297dffb640 1 -- 172.21.15.123:0/2953309340 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978036070 con 0x7f29800c7ea0 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.600+0000 7f297dffb640 1 -- 172.21.15.123:0/2953309340 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2978031600 con 0x7f29800c7ea0 2024-09-19T05:42:11.603 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.600+0000 7f297dffb640 1 -- 172.21.15.123:0/2953309340 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29780319d0 con 0x7f29800c7ea0 2024-09-19T05:42:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 -- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29800c7ea0 msgr2=0x7f29800c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 --2- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29800c7ea0 0x7f29800c8310 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2978009e50 tx=0x7f297802a7a0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 -- 172.21.15.123:0/2953309340 shutdown_connections 2024-09-19T05:42:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 --2- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29800c7ea0 0x7f29800c8310 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 --2- 172.21.15.123:0/2953309340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f298006ce90 0x7f29801593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 -- 172.21.15.123:0/2953309340 >> 172.21.15.123:0/2953309340 conn(0x7f298006fed0 msgr2=0x7f29800702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 -- 172.21.15.123:0/2953309340 shutdown_connections 2024-09-19T05:42:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.601+0000 7f2985926640 1 -- 172.21.15.123:0/2953309340 wait complete. 2024-09-19T05:42:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.602+0000 7f2985926640 1 Processor -- start 2024-09-19T05:42:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.602+0000 7f2985926640 1 -- start start 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f2985926640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f298006ce90 0x7f2980147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f2985926640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298006d270 0x7f29800c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f2985926640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29800c7ea0 0x7f29801f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f2985926640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f298006eac0 con 0x7f298006ce90 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f2985926640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f298006e7e0 con 0x7f298006d270 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f2985926640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f29800c7980 con 0x7f29800c7ea0 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29800c7ea0 0x7f29801f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297e7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298006d270 0x7f29800c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.607 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29800c7ea0 0x7f29801f3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:40872/0 (socket says 172.21.15.123:40872) 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297f7fe640 1 -- 172.21.15.123:0/269476911 learned_addr learned my addr 172.21.15.123:0/269476911 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297effd640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f298006ce90 0x7f2980147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297e7fc640 1 -- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29800c7ea0 msgr2=0x7f29801f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297e7fc640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29800c7ea0 0x7f29801f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297e7fc640 1 -- 172.21.15.123:0/269476911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f298006ce90 msgr2=0x7f2980147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297e7fc640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f298006ce90 0x7f2980147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297e7fc640 1 -- 172.21.15.123:0/269476911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29800c66f0 con 0x7f298006d270 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297f7fe640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29800c7ea0 0x7f29801f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.603+0000 7f297e7fc640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298006d270 0x7f29800c1e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f297000c340 tx=0x7f297000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.604+0000 7f2967fff640 1 -- 172.21.15.123:0/269476911 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700052d0 con 0x7f298006d270 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.604+0000 7f2967fff640 1 -- 172.21.15.123:0/269476911 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2970005490 con 0x7f298006d270 2024-09-19T05:42:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.604+0000 7f2967fff640 1 -- 172.21.15.123:0/269476911 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970007260 con 0x7f298006d270 2024-09-19T05:42:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.604+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29800c3930 con 0x7f298006d270 2024-09-19T05:42:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.604+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29800c6990 con 0x7f298006d270 2024-09-19T05:42:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.604+0000 7f2967fff640 1 -- 172.21.15.123:0/269476911 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2970007400 con 0x7f298006d270 2024-09-19T05:42:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.604+0000 7f2967fff640 1 -- 172.21.15.123:0/269476911 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f2970011590 con 0x7f298006d270 2024-09-19T05:42:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.605+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29800b31b0 con 0x7f298006d270 2024-09-19T05:42:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.607+0000 7f2967fff640 1 -- 172.21.15.123:0/269476911 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2970003a00 con 0x7f298006d270 2024-09-19T05:42:11.677 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.676+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "48561580-ba9a-4096-9722-16c308f414ee", "id": 6} v 0) -- 0x7f29800706a0 con 0x7f298006d270 2024-09-19T05:42:11.681 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.679+0000 7f2967fff640 1 -- 172.21.15.123:0/269476911 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "48561580-ba9a-4096-9722-16c308f414ee", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f2970003be0 con 0x7f298006d270 2024-09-19T05:42:11.681 INFO:teuthology.orchestra.run.smithi123.stdout:6 2024-09-19T05:42:11.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298006d270 msgr2=0x7f29800c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298006d270 0x7f29800c1e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f297000c340 tx=0x7f297000c800 comp rx=0 tx=0).stop 2024-09-19T05:42:11.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 shutdown_connections 2024-09-19T05:42:11.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298006d270 0x7f29800c1e90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29800c7ea0 0x7f29801f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.682 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 --2- 172.21.15.123:0/269476911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f298006ce90 0x7f2980147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 >> 172.21.15.123:0/269476911 conn(0x7f298006fed0 msgr2=0x7f2980154ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 shutdown_connections 2024-09-19T05:42:11.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.680+0000 7f2985926640 1 -- 172.21.15.123:0/269476911 wait complete. 2024-09-19T05:42:11.692 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new d1ee7357-25a5-432c-b96a-e808139a1cf3 7 2024-09-19T05:42:11.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.790+0000 7f705ec1e640 1 Processor -- start 2024-09-19T05:42:11.792 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.790+0000 7f705ec1e640 1 -- start start 2024-09-19T05:42:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705ec1e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070910 0x7f7058073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.793 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705ec1e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7058070cf0 0x7f7058151170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705ec1e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70580c6430 con 0x7f7058073300 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705ec1e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f705815b530 con 0x7f7058070cf0 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705ec1e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7058073d90 con 0x7f7058070910 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705c9ba640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070910 0x7f7058073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f7057fff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7058073300 0x7f705815ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:48832/0 (socket says 172.21.15.123:48832) 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f7057fff640 1 -- 172.21.15.123:0/2122147368 learned_addr learned my addr 172.21.15.123:0/2122147368 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705d1bb640 1 --2- 172.21.15.123:0/2122147368 >> v2:172.21.15.123:3300/0 conn(0x7f7058070cf0 0x7f7058151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705c9ba640 1 -- 172.21.15.123:0/2122147368 >> v2:172.21.15.123:3300/0 conn(0x7f7058070cf0 msgr2=0x7f7058151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705c9ba640 1 --2- 172.21.15.123:0/2122147368 >> v2:172.21.15.123:3300/0 conn(0x7f7058070cf0 0x7f7058151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705c9ba640 1 -- 172.21.15.123:0/2122147368 >> v1:172.21.15.123:6789/0 conn(0x7f7058073300 legacy=0x7f705815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705c9ba640 1 -- 172.21.15.123:0/2122147368 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7058157830 con 0x7f7058070910 2024-09-19T05:42:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.791+0000 7f705d1bb640 1 --2- 172.21.15.123:0/2122147368 >> v2:172.21.15.123:3300/0 conn(0x7f7058070cf0 0x7f7058151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.792+0000 7f705c9ba640 1 --2- 172.21.15.123:0/2122147368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070910 0x7f7058073700 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f704c00ba40 tx=0x7f704c02c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=162b8646ae4b7deb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.792+0000 7f70577fe640 1 -- 172.21.15.123:0/2122147368 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c032750 con 0x7f7058070910 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.792+0000 7f70577fe640 1 -- 172.21.15.123:0/2122147368 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c030050 con 0x7f7058070910 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.792+0000 7f70577fe640 1 -- 172.21.15.123:0/2122147368 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c032b10 con 0x7f7058070910 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 -- 172.21.15.123:0/2122147368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070910 msgr2=0x7f7058073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 --2- 172.21.15.123:0/2122147368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070910 0x7f7058073700 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f704c00ba40 tx=0x7f704c02c850 comp rx=0 tx=0).stop 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 -- 172.21.15.123:0/2122147368 shutdown_connections 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 --2- 172.21.15.123:0/2122147368 >> v2:172.21.15.123:3300/0 conn(0x7f7058070cf0 0x7f7058151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.796 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 --2- 172.21.15.123:0/2122147368 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070910 0x7f7058073700 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 -- 172.21.15.123:0/2122147368 >> 172.21.15.123:0/2122147368 conn(0x7f70580cd4d0 msgr2=0x7f70580cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 -- 172.21.15.123:0/2122147368 shutdown_connections 2024-09-19T05:42:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.793+0000 7f705ec1e640 1 -- 172.21.15.123:0/2122147368 wait complete. 2024-09-19T05:42:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.794+0000 7f705ec1e640 1 Processor -- start 2024-09-19T05:42:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.794+0000 7f705ec1e640 1 -- start start 2024-09-19T05:42:11.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705ec1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7058070910 0x7f705815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705ec1e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070cf0 0x7f705814d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.799 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705ec1e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7058073300 0x7f70581f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705ec1e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7058157dd0 con 0x7f7058070910 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705ec1e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7058157af0 con 0x7f7058073300 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705ec1e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70581585d0 con 0x7f7058070cf0 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7058073300 0x7f70581f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f7057fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070cf0 0x7f705814d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7058073300 0x7f70581f0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:58206/0 (socket says 172.21.15.123:58206) 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 -- 172.21.15.123:0/497724665 learned_addr learned my addr 172.21.15.123:0/497724665 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705c9ba640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7058070910 0x7f705815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 -- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070cf0 msgr2=0x7f705814d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070cf0 0x7f705814d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 -- 172.21.15.123:0/497724665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7058070910 msgr2=0x7f705815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7058070910 0x7f705815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f705d1bb640 1 -- 172.21.15.123:0/497724665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70580c8610 con 0x7f7058073300 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.795+0000 7f7057fff640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070cf0 0x7f705814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f705d1bb640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7058073300 0x7f70581f0730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f705000efc0 tx=0x7f705000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f70557fa640 1 -- 172.21.15.123:0/497724665 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050018020 con 0x7f7058073300 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f70557fa640 1 -- 172.21.15.123:0/497724665 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7050014480 con 0x7f7058073300 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f70557fa640 1 -- 172.21.15.123:0/497724665 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7050014870 con 0x7f7058073300 2024-09-19T05:42:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70581f45a0 con 0x7f7058073300 2024-09-19T05:42:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70580c8b90 con 0x7f7058073300 2024-09-19T05:42:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f70557fa640 1 -- 172.21.15.123:0/497724665 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7050005d10 con 0x7f7058073300 2024-09-19T05:42:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.796+0000 7f70557fa640 1 -- 172.21.15.123:0/497724665 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f705001dd60 con 0x7f7058073300 2024-09-19T05:42:11.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.797+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705814fa00 con 0x7f7058073300 2024-09-19T05:42:11.803 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.799+0000 7f70557fa640 1 -- 172.21.15.123:0/497724665 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f705814fa00 con 0x7f7058073300 2024-09-19T05:42:11.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.868+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d1ee7357-25a5-432c-b96a-e808139a1cf3", "id": 7} v 0) -- 0x7f70580c4a00 con 0x7f7058073300 2024-09-19T05:42:11.876 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.874+0000 7f70557fa640 1 -- 172.21.15.123:0/497724665 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1ee7357-25a5-432c-b96a-e808139a1cf3", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f7050022030 con 0x7f7058073300 2024-09-19T05:42:11.876 INFO:teuthology.orchestra.run.smithi123.stdout:7 2024-09-19T05:42:11.877 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7058073300 msgr2=0x7f70581f0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7058073300 0x7f70581f0730 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f705000efc0 tx=0x7f705000c490 comp rx=0 tx=0).stop 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 shutdown_connections 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7058073300 0x7f70581f0730 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7058070cf0 0x7f705814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 --2- 172.21.15.123:0/497724665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7058070910 0x7f705815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 >> 172.21.15.123:0/497724665 conn(0x7f70580cd4d0 msgr2=0x7f70581569b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 shutdown_connections 2024-09-19T05:42:11.878 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-19T05:42:11.875+0000 7f705ec1e640 1 -- 172.21.15.123:0/497724665 wait complete. 2024-09-19T05:42:11.884 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:11.882+0000 7f3085342040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:42:11.887 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-19T05:42:11.887 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-19T05:42:11.890 INFO:tasks.ceph.osd.0:Started 2024-09-19T05:42:11.890 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-19T05:42:11.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-19T05:42:11.892 INFO:tasks.ceph.osd.1:Started 2024-09-19T05:42:11.893 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-19T05:42:11.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-19T05:42:11.895 INFO:tasks.ceph.osd.2:Started 2024-09-19T05:42:11.895 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-19T05:42:11.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-19T05:42:11.897 INFO:tasks.ceph.osd.3:Started 2024-09-19T05:42:11.897 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-19T05:42:11.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-19T05:42:11.900 INFO:tasks.ceph.osd.4:Started 2024-09-19T05:42:11.900 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-19T05:42:11.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-19T05:42:11.902 INFO:tasks.ceph.osd.5:Started 2024-09-19T05:42:11.903 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-19T05:42:11.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-19T05:42:11.905 INFO:tasks.ceph.osd.6:Started 2024-09-19T05:42:11.905 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-19T05:42:11.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-19T05:42:11.908 INFO:tasks.ceph.osd.7:Started 2024-09-19T05:42:11.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:11.949 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:11.948+0000 7f3085342040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:42:11.953 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:11.952+0000 7f8cc7836040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T05:42:12.024 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.022+0000 7f8cc7836040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T05:42:12.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.031+0000 7f06d1204640 1 Processor -- start 2024-09-19T05:42:12.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.031+0000 7f06d1204640 1 -- start start 2024-09-19T05:42:12.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06d1204640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc1529b0 0x7f06cc15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:12.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06d1204640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f06cc0cb0c0 0x7f06cc0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:12.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06d1204640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06cc158880 con 0x7f06cc159320 2024-09-19T05:42:12.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06d1204640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f06cc159740 con 0x7f06cc0cb0c0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06d1204640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc0c3730 con 0x7f06cc1529b0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cb59e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f06cc0cb0c0 0x7f06cc0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cb59e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f06cc0cb0c0 0x7f06cc0cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58850/0 (socket says 172.21.15.2:58850) 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cb59e640 1 -- 172.21.15.2:0/2859235004 learned_addr learned my addr 172.21.15.2:0/2859235004 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cad9d640 1 --2- 172.21.15.2:0/2859235004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc1529b0 0x7f06cc15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cb59e640 1 -- 172.21.15.2:0/2859235004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc1529b0 msgr2=0x7f06cc15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cb59e640 1 --2- 172.21.15.2:0/2859235004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc1529b0 0x7f06cc15bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cb59e640 1 -- 172.21.15.2:0/2859235004 >> v1:172.21.15.123:6789/0 conn(0x7f06cc159320 legacy=0x7f06cc0ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.032+0000 7f06cb59e640 1 -- 172.21.15.2:0/2859235004 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc0c3a10 con 0x7f06cc0cb0c0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.033+0000 7f06cb59e640 1 --2- 172.21.15.2:0/2859235004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc0cb0c0 0x7f06cc0cb570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009e50 tx=0x7f06c402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbe0e39063b1bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.033+0000 7f06ca59c640 1 -- 172.21.15.2:0/2859235004 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4036070 con 0x7f06cc0cb0c0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.033+0000 7f06ca59c640 1 -- 172.21.15.2:0/2859235004 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c4031600 con 0x7f06cc0cb0c0 2024-09-19T05:42:12.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.033+0000 7f06ca59c640 1 -- 172.21.15.2:0/2859235004 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c40319d0 con 0x7f06cc0cb0c0 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 -- 172.21.15.2:0/2859235004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc0cb0c0 msgr2=0x7f06cc0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 --2- 172.21.15.2:0/2859235004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc0cb0c0 0x7f06cc0cb570 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009e50 tx=0x7f06c402a710 comp rx=0 tx=0).stop 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 -- 172.21.15.2:0/2859235004 shutdown_connections 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 --2- 172.21.15.2:0/2859235004 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc0cb0c0 0x7f06cc0cb570 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 --2- 172.21.15.2:0/2859235004 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc1529b0 0x7f06cc15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 -- 172.21.15.2:0/2859235004 >> 172.21.15.2:0/2859235004 conn(0x7f06cc06bc10 msgr2=0x7f06cc06f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 -- 172.21.15.2:0/2859235004 shutdown_connections 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.034+0000 7f06d1204640 1 -- 172.21.15.2:0/2859235004 wait complete. 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.035+0000 7f06d1204640 1 Processor -- start 2024-09-19T05:42:12.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.035+0000 7f06d1204640 1 -- start start 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06d1204640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc0cb0c0 0x7f06cc0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06d1204640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc1529b0 0x7f06cc14b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06d1204640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc159320 0x7f06cc1ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06d1204640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc0c3cf0 con 0x7f06cc1529b0 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06d1204640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06cc158e00 con 0x7f06cc159320 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06d1204640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f06cc0c3e70 con 0x7f06cc0cb0c0 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc1529b0 0x7f06cc14b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:12.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06cad9d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc0cb0c0 0x7f06cc0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06c3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc1529b0 0x7f06cc14b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58858/0 (socket says 172.21.15.2:58858) 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06cad9d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc0cb0c0 0x7f06cc0ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:34144/0 (socket says 172.21.15.2:34144) 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06c3fff640 1 -- 172.21.15.2:0/824407262 learned_addr learned my addr 172.21.15.2:0/824407262 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.036+0000 7f06cb59e640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc159320 0x7f06cc1ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c3fff640 1 -- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc0cb0c0 msgr2=0x7f06cc0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c3fff640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc0cb0c0 0x7f06cc0ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c3fff640 1 -- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc159320 msgr2=0x7f06cc1ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c3fff640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc159320 0x7f06cc1ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c3fff640 1 -- 172.21.15.2:0/824407262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06cc14bea0 con 0x7f06cc1529b0 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06cad9d640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc0cb0c0 0x7f06cc0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c3fff640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc1529b0 0x7f06cc14b170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c440 tx=0x7f06b400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c37fe640 1 -- 172.21.15.2:0/824407262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40052d0 con 0x7f06cc1529b0 2024-09-19T05:42:12.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c37fe640 1 -- 172.21.15.2:0/824407262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b4005470 con 0x7f06cc1529b0 2024-09-19T05:42:12.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06cc14c180 con 0x7f06cc1529b0 2024-09-19T05:42:12.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c37fe640 1 -- 172.21.15.2:0/824407262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4007260 con 0x7f06cc1529b0 2024-09-19T05:42:12.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06cc14c740 con 0x7f06cc1529b0 2024-09-19T05:42:12.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c37fe640 1 -- 172.21.15.2:0/824407262 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f06b40074c0 con 0x7f06cc1529b0 2024-09-19T05:42:12.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.037+0000 7f06c37fe640 1 -- 172.21.15.2:0/824407262 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f06b4003a00 con 0x7f06cc1529b0 2024-09-19T05:42:12.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.038+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06cc152d90 con 0x7f06cc1529b0 2024-09-19T05:42:12.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.040+0000 7f06c37fe640 1 -- 172.21.15.2:0/824407262 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f06b40035f0 con 0x7f06cc1529b0 2024-09-19T05:42:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.100+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06cc159a20 con 0x7f06cc1529b0 2024-09-19T05:42:12.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.100+0000 7f06c37fe640 1 -- 172.21.15.2:0/824407262 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f06b4003c20 con 0x7f06cc1529b0 2024-09-19T05:42:12.102 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:12.102 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:11.870630+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:12.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc1529b0 msgr2=0x7f06cc14b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:12.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc1529b0 0x7f06cc14b170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c440 tx=0x7f06b400c900 comp rx=0 tx=0).stop 2024-09-19T05:42:12.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 shutdown_connections 2024-09-19T05:42:12.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06cc159320 0x7f06cc1ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06cc1529b0 0x7f06cc14b170 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 --2- 172.21.15.2:0/824407262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06cc0cb0c0 0x7f06cc0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:12.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 >> 172.21.15.2:0/824407262 conn(0x7f06cc06bc10 msgr2=0x7f06cc0c5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:12.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 shutdown_connections 2024-09-19T05:42:12.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:12.101+0000 7f06d1204640 1 -- 172.21.15.2:0/824407262 wait complete. 2024-09-19T05:42:12.113 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-19T05:42:12.113 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-19T05:42:12.331 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.329+0000 7f3085342040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:42:12.411 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.409+0000 7f8cc7836040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T05:42:12.418 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.416+0000 7f3085342040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:42:12.499 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.497+0000 7f8cc7836040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T05:42:12.526 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.524+0000 7f3085342040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:42:12.573 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.571+0000 7f3085342040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:42:12.611 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.609+0000 7f8cc7836040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T05:42:12.659 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.657+0000 7f8cc7836040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T05:42:12.685 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.683+0000 7f3085342040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:42:12.738 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.736+0000 7f3085342040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:42:12.777 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.775+0000 7f8cc7836040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T05:42:12.783 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.781+0000 7f3085342040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:42:12.832 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.830+0000 7f8cc7836040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T05:42:12.879 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.877+0000 7f8cc7836040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T05:42:12.895 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.893+0000 7f3085342040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:42:12.919 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:12.928 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:12.926+0000 7f2740211640 -1 Falling back to public interface 2024-09-19T05:42:12.945 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.943+0000 7f3085342040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:42:12.996 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:12.994+0000 7f3085342040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:42:12.998 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:12.996+0000 7f8cc7836040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T05:42:13.012 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.010+0000 7f81abd28640 1 Processor -- start 2024-09-19T05:42:13.012 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.010+0000 7f81abd28640 1 -- start start 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81abd28640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40ce690 0x7f81a40c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81abd28640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f81a40cea70 0x7f81a415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81abd28640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a40c97f0 con 0x7f81a406e490 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81abd28640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f81a4071110 con 0x7f81a40cea70 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81abd28640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f81a40c95c0 con 0x7f81a40ce690 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81a92c3640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f81a40cea70 0x7f81a415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81a92c3640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f81a40cea70 0x7f81a415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58868/0 (socket says 172.21.15.2:58868) 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81a92c3640 1 -- 172.21.15.2:0/3551077409 learned_addr learned my addr 172.21.15.2:0/3551077409 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:13.014 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.011+0000 7f81a9ac4640 1 --2- 172.21.15.2:0/3551077409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40ce690 0x7f81a40c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:13.015 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a92c3640 1 -- 172.21.15.2:0/3551077409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40ce690 msgr2=0x7f81a40c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:13.015 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a92c3640 1 --2- 172.21.15.2:0/3551077409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40ce690 0x7f81a40c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.015 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a92c3640 1 -- 172.21.15.2:0/3551077409 >> v1:172.21.15.123:6789/0 conn(0x7f81a406e490 legacy=0x7f81a415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:13.015 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a92c3640 1 -- 172.21.15.2:0/3551077409 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a40c9ab0 con 0x7f81a40cea70 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a92c3640 1 --2- 172.21.15.2:0/3551077409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a40cea70 0x7f81a415ac00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009870 tx=0x7f81a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c86cdb85a3e9866 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a8ac2640 1 -- 172.21.15.2:0/3551077409 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0037070 con 0x7f81a40cea70 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a8ac2640 1 -- 172.21.15.2:0/3551077409 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a0032440 con 0x7f81a40cea70 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.012+0000 7f81a8ac2640 1 -- 172.21.15.2:0/3551077409 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0032810 con 0x7f81a40cea70 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 -- 172.21.15.2:0/3551077409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a40cea70 msgr2=0x7f81a415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 --2- 172.21.15.2:0/3551077409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a40cea70 0x7f81a415ac00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009870 tx=0x7f81a002a270 comp rx=0 tx=0).stop 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 -- 172.21.15.2:0/3551077409 shutdown_connections 2024-09-19T05:42:13.016 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 --2- 172.21.15.2:0/3551077409 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a40cea70 0x7f81a415ac00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.017 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 --2- 172.21.15.2:0/3551077409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40ce690 0x7f81a40c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.017 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 -- 172.21.15.2:0/3551077409 >> 172.21.15.2:0/3551077409 conn(0x7f81a406c1f0 msgr2=0x7f81a4067aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:13.017 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 -- 172.21.15.2:0/3551077409 shutdown_connections 2024-09-19T05:42:13.017 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.013+0000 7f81abd28640 1 -- 172.21.15.2:0/3551077409 wait complete. 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.014+0000 7f81abd28640 1 Processor -- start 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81abd28640 1 -- start start 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81abd28640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a406e490 0x7f81a415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81abd28640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a40ce690 0x7f81a40c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81abd28640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40cea70 0x7f81a4151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81abd28640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4070c80 con 0x7f81a406e490 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81abd28640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81a40c9d90 con 0x7f81a40ce690 2024-09-19T05:42:13.019 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81abd28640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f81a4070910 con 0x7f81a40cea70 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81a9ac4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a406e490 0x7f81a415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81a9ac4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a406e490 0x7f81a415e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58884/0 (socket says 172.21.15.2:58884) 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81a9ac4640 1 -- 172.21.15.2:0/3125212005 learned_addr learned my addr 172.21.15.2:0/3125212005 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81aa2c5640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40cea70 0x7f81a4151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.015+0000 7f81a92c3640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a40ce690 0x7f81a40c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81a9ac4640 1 -- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40cea70 msgr2=0x7f81a4151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81a9ac4640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40cea70 0x7f81a4151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81a9ac4640 1 -- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a40ce690 msgr2=0x7f81a40c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81a9ac4640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a40ce690 0x7f81a40c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81a9ac4640 1 -- 172.21.15.2:0/3125212005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a41533a0 con 0x7f81a406e490 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81aa2c5640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40cea70 0x7f81a4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81a9ac4640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a406e490 0x7f81a415e9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9e0 tx=0x7f819800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:13.020 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81967fc640 1 -- 172.21.15.2:0/3125212005 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198017070 con 0x7f81a406e490 2024-09-19T05:42:13.021 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81967fc640 1 -- 172.21.15.2:0/3125212005 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819800c9d0 con 0x7f81a406e490 2024-09-19T05:42:13.021 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a41f3710 con 0x7f81a406e490 2024-09-19T05:42:13.021 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81967fc640 1 -- 172.21.15.2:0/3125212005 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800cdc0 con 0x7f81a406e490 2024-09-19T05:42:13.021 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41f3cd0 con 0x7f81a406e490 2024-09-19T05:42:13.021 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81967fc640 1 -- 172.21.15.2:0/3125212005 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f819801a070 con 0x7f81a406e490 2024-09-19T05:42:13.021 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.016+0000 7f81967fc640 1 -- 172.21.15.2:0/3125212005 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f819801cc10 con 0x7f81a406e490 2024-09-19T05:42:13.022 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.017+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a406eb20 con 0x7f81a406e490 2024-09-19T05:42:13.022 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.019+0000 7f81967fc640 1 -- 172.21.15.2:0/3125212005 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f819801c540 con 0x7f81a406e490 2024-09-19T05:42:13.063 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:13.061+0000 7f3085342040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:42:13.073 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:13.071+0000 7f8cc7836040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T05:42:13.108 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:13.106+0000 7f3085342040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:42:13.112 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.110+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81a40cc400 con 0x7f81a406e490 2024-09-19T05:42:13.113 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.111+0000 7f81967fc640 1 -- 172.21.15.2:0/3125212005 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f819801c7e0 con 0x7f81a406e490 2024-09-19T05:42:13.114 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-19T05:42:13.114 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":10,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:11.870630+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a406e490 msgr2=0x7f81a415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a406e490 0x7f81a415e9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f819800e9e0 tx=0x7f819800eea0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 shutdown_connections 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81a40ce690 0x7f81a40c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f81a40cea70 0x7f81a4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 --2- 172.21.15.2:0/3125212005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81a406e490 0x7f81a415e9f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 >> 172.21.15.2:0/3125212005 conn(0x7f81a406c1f0 msgr2=0x7f81a40ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 shutdown_connections 2024-09-19T05:42:13.115 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:13.112+0000 7f81abd28640 1 -- 172.21.15.2:0/3125212005 wait complete. 2024-09-19T05:42:13.123 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-19T05:42:13.131 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:13.129+0000 7f8cc7836040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T05:42:13.135 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:13.133+0000 7fa14aa9c640 -1 Falling back to public interface 2024-09-19T05:42:13.152 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:13.151+0000 7f3085342040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:42:13.204 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:13.202+0000 7f8cc7836040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T05:42:13.251 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:13.249+0000 7f8cc7836040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T05:42:13.297 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:13.295+0000 7f8cc7836040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T05:42:13.346 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:13.344+0000 7f5c01a18640 -1 Falling back to public interface 2024-09-19T05:42:13.349 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:13.347+0000 7f7e3cf20640 -1 Falling back to public interface 2024-09-19T05:42:13.351 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:13.349+0000 7fa5d3345640 -1 Falling back to public interface 2024-09-19T05:42:13.368 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:13.366+0000 7fb95ca9c640 -1 Falling back to public interface 2024-09-19T05:42:13.621 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:13.618+0000 7f5a4c345640 -1 Falling back to public interface 2024-09-19T05:42:13.635 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:13.633+0000 7f11b89a4640 -1 Falling back to public interface 2024-09-19T05:42:13.808 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:13.806+0000 7f3085342040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:42:13.988 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-19T05:42:13.986+0000 7f8cc7836040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T05:42:14.825 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:14.823+0000 7f3025c06640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T05:42:14.825 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-19T05:42:14.823+0000 7f3025c06640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-19T05:42:16.455 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:16.453+0000 7f2740211640 -1 osd.5 0 log_to_monitors true 2024-09-19T05:42:16.956 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:16.954+0000 7f7e3cf20640 -1 osd.1 0 log_to_monitors true 2024-09-19T05:42:17.152 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:17.150+0000 7f5c01a18640 -1 osd.0 0 log_to_monitors true 2024-09-19T05:42:17.162 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:17.160+0000 7fb95ca9c640 -1 osd.4 0 log_to_monitors true 2024-09-19T05:42:17.221 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:17.210+0000 7fa14aa9c640 -1 osd.3 0 log_to_monitors true 2024-09-19T05:42:17.360 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:17.358+0000 7fa5d3345640 -1 osd.7 0 log_to_monitors true 2024-09-19T05:42:17.446 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:17.444+0000 7f11b89a4640 -1 osd.6 0 log_to_monitors true 2024-09-19T05:42:17.952 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:17.950+0000 7f5a4c345640 -1 osd.2 0 log_to_monitors true 2024-09-19T05:42:18.374 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:18.372+0000 7f273bb92640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T05:42:18.383 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:18.382+0000 7f27339b2640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:19.676 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:19.674+0000 7fa5cf2d4640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T05:42:19.692 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:19.690+0000 7fa5c70f4640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:19.708 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:19.705+0000 7f5bfd39a640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T05:42:19.719 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:19.715+0000 7fa146a2b640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T05:42:19.721 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:19.719+0000 7f5bf51ba640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:19.723 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:19.721+0000 7f7e38eaf640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T05:42:19.731 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:19.730+0000 7f11b4933640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T05:42:19.732 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:19.730+0000 7fa13e84b640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:19.741 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:19.739+0000 7f7e30ccf640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:19.741 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:19.740+0000 7f11ac753640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:19.744 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:19.742+0000 7fb958a2b640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T05:42:19.764 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:19.763+0000 7fb95084b640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:19.930 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:20.029 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.027+0000 7fda18a53640 1 Processor -- start 2024-09-19T05:42:20.067 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.027+0000 7fda18a53640 1 -- start start 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda18a53640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fda14151950 0x7fda140c4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda18a53640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda140cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda18a53640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda14157840 con 0x7fda140c7350 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda18a53640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fda140c6fc0 con 0x7fda14151950 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda18a53640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fda14158450 con 0x7fda140c7d40 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda1259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fda14151950 0x7fda140c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda12d9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fda140c7350 0x7fda1415ad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49028/0 (socket says 172.21.15.2:49028) 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda1259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fda14151950 0x7fda140c4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58920/0 (socket says 172.21.15.2:58920) 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda12d9d640 1 -- 172.21.15.2:0/3035481252 learned_addr learned my addr 172.21.15.2:0/3035481252 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda0bfff640 1 --2- 172.21.15.2:0/3035481252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda140cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:20.068 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda1259c640 1 -- 172.21.15.2:0/3035481252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 msgr2=0x7fda140cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda1259c640 1 --2- 172.21.15.2:0/3035481252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda140cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda1259c640 1 -- 172.21.15.2:0/3035481252 >> v1:172.21.15.2:6789/0 conn(0x7fda140c7350 legacy=0x7fda1415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda11d9b640 1 -- 172.21.15.2:0/3035481252 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898368600 0 0) 0x7fda14157840 con 0x7fda140c7350 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.028+0000 7fda1259c640 1 -- 172.21.15.2:0/3035481252 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda14157b20 con 0x7fda14151950 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.029+0000 7fda1259c640 1 --2- 172.21.15.2:0/3035481252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda140c4840 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda040097a0 tx=0x7fda0402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae36711f37a24fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.029+0000 7fda11d9b640 1 -- 172.21.15.2:0/3035481252 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04037070 con 0x7fda14151950 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.029+0000 7fda11d9b640 1 -- 172.21.15.2:0/3035481252 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda04032440 con 0x7fda14151950 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.029+0000 7fda11d9b640 1 -- 172.21.15.2:0/3035481252 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04032810 con 0x7fda14151950 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 -- 172.21.15.2:0/3035481252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 msgr2=0x7fda140c4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 --2- 172.21.15.2:0/3035481252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda140c4840 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda040097a0 tx=0x7fda0402a210 comp rx=0 tx=0).stop 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 -- 172.21.15.2:0/3035481252 shutdown_connections 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 --2- 172.21.15.2:0/3035481252 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda140cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.069 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 --2- 172.21.15.2:0/3035481252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda140c4840 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.070 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 -- 172.21.15.2:0/3035481252 >> 172.21.15.2:0/3035481252 conn(0x7fda14072a10 msgr2=0x7fda14072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:20.070 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 -- 172.21.15.2:0/3035481252 shutdown_connections 2024-09-19T05:42:20.070 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.030+0000 7fda18a53640 1 -- 172.21.15.2:0/3035481252 wait complete. 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.067+0000 7fda18a53640 1 Processor -- start 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.067+0000 7fda18a53640 1 -- start start 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.068+0000 7fda18a53640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda140c7350 0x7fda140cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.068+0000 7fda18a53640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda1414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.068+0000 7fda18a53640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda14154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda18a53640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fda14151d30 con 0x7fda14151950 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda18a53640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda141580e0 con 0x7fda140c7350 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda18a53640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fda1415b460 con 0x7fda140c7d40 2024-09-19T05:42:20.072 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda14154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda14154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58934/0 (socket says 172.21.15.2:58934) 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 -- 172.21.15.2:0/311826650 learned_addr learned my addr 172.21.15.2:0/311826650 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda0bfff640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda1414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda1259c640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda140c7350 0x7fda140cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 -- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 msgr2=0x7fda1414df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda1414df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 -- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda140c7350 msgr2=0x7fda140cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda140c7350 0x7fda140cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 -- 172.21.15.2:0/311826650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1408c8f0 con 0x7fda14151950 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda0bfff640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda1414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda1259c640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda140c7350 0x7fda140cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda12d9d640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda14154c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c780 tx=0x7fda0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.069+0000 7fda0affd640 1 -- 172.21.15.2:0/311826650 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c018070 con 0x7fda14151950 2024-09-19T05:42:20.073 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.070+0000 7fda0affd640 1 -- 172.21.15.2:0/311826650 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda0c00a640 con 0x7fda14151950 2024-09-19T05:42:20.074 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.070+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1408cb90 con 0x7fda14151950 2024-09-19T05:42:20.074 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.070+0000 7fda0affd640 1 -- 172.21.15.2:0/311826650 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00aa70 con 0x7fda14151950 2024-09-19T05:42:20.074 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.070+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda141562f0 con 0x7fda14151950 2024-09-19T05:42:20.074 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.071+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda140c4840 con 0x7fda14151950 2024-09-19T05:42:20.075 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.071+0000 7fda0affd640 1 -- 172.21.15.2:0/311826650 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) ==== 90790+0+0 (secure 0 0 0) 0x7fda0c01b070 con 0x7fda14151950 2024-09-19T05:42:20.075 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.072+0000 7fda0affd640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd9f406c220 0x7fd9f406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:20.075 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.072+0000 7fda0affd640 1 -- 172.21.15.2:0/311826650 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 4191+0+0 (secure 0 0 0) 0x7fda0c08a270 con 0x7fda14151950 2024-09-19T05:42:20.075 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.072+0000 7fda1259c640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd9f406c220 0x7fd9f406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:20.075 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.073+0000 7fda1259c640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd9f406c220 0x7fd9f406e6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda04034000 tx=0x7fda0402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:20.079 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.077+0000 7fda0affd640 1 -- 172.21.15.2:0/311826650 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fda0c058a60 con 0x7fda14151950 2024-09-19T05:42:20.440 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.438+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda1414ef10 con 0x7fda14151950 2024-09-19T05:42:20.441 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.439+0000 7fda0affd640 1 -- 172.21.15.2:0/311826650 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+10410 (secure 0 0 0) 0x7fda0c08a020 con 0x7fda14151950 2024-09-19T05:42:20.441 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-19T05:42:20.441 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":14,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:19.826945+0000","last_up_change":"2024-09-19T05:42:19.826945+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:20.445 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd9f406c220 msgr2=0x7fd9f406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.445 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd9f406c220 0x7fd9f406e6e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda04034000 tx=0x7fda0402e000 comp rx=0 tx=0).stop 2024-09-19T05:42:20.445 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 msgr2=0x7fda14154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:20.445 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda14154c40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00c780 tx=0x7fda0c00cc40 comp rx=0 tx=0).stop 2024-09-19T05:42:20.445 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 shutdown_connections 2024-09-19T05:42:20.446 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd9f406c220 0x7fd9f406e6e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.446 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda14151950 0x7fda14154c40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.446 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda140c7d40 0x7fda1414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.446 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 --2- 172.21.15.2:0/311826650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda140c7350 0x7fda140cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:20.446 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.442+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 >> 172.21.15.2:0/311826650 conn(0x7fda14072a10 msgr2=0x7fda14150be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:20.446 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.443+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 shutdown_connections 2024-09-19T05:42:20.446 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:20.443+0000 7fda18a53640 1 -- 172.21.15.2:0/311826650 wait complete. 2024-09-19T05:42:20.451 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:20.449+0000 7f5a482d4640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T05:42:20.456 DEBUG:teuthology.misc:7 of 8 OSDs are up 2024-09-19T05:42:20.461 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:20.459+0000 7f5a400f4640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:42:27.263 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:27.368 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.366+0000 7f7057877640 1 Processor -- start 2024-09-19T05:42:27.368 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.366+0000 7f7057877640 1 -- start start 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7057877640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cb480 0x7f70500cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7057877640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70500cb860 0x7f705015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7057877640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f70500cc8c0 0x7f7050154570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7057877640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f705015ee60 con 0x7f70500cc8c0 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7057877640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f70500c5820 con 0x7f70500cb860 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7057877640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f705015e0a0 con 0x7f70500cb480 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7055e14640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f70500cc8c0 0x7f7050154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7055e14640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f70500cc8c0 0x7f7050154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57052/0 (socket says 172.21.15.2:57052) 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7055e14640 1 -- 172.21.15.2:0/2868421322 learned_addr learned my addr 172.21.15.2:0/2868421322 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7054e12640 1 --2- 172.21.15.2:0/2868421322 >> v2:172.21.15.123:3300/0 conn(0x7f70500cb860 0x7f705015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.370 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.367+0000 7f7055613640 1 --2- 172.21.15.2:0/2868421322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cb480 0x7f70500cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.371 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7055e14640 1 -- 172.21.15.2:0/2868421322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cb480 msgr2=0x7f70500cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:27.371 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7055e14640 1 --2- 172.21.15.2:0/2868421322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cb480 0x7f70500cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.371 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7055e14640 1 -- 172.21.15.2:0/2868421322 >> v2:172.21.15.123:3300/0 conn(0x7f70500cb860 msgr2=0x7f705015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:27.371 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7055e14640 1 --2- 172.21.15.2:0/2868421322 >> v2:172.21.15.123:3300/0 conn(0x7f70500cb860 0x7f705015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.371 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7055e14640 1 -- 172.21.15.2:0/2868421322 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705015e380 con 0x7f70500cc8c0 2024-09-19T05:42:27.371 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7055613640 1 --2- 172.21.15.2:0/2868421322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cb480 0x7f70500cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:27.372 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7055e14640 1 --2- 172.21.15.2:0/2868421322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cc8c0 0x7f7050154570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f705008e0c0 tx=0x7f704802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25d5f2b96b42d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:27.372 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.368+0000 7f7047fff640 1 -- 172.21.15.2:0/2868421322 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048037070 con 0x7f70500cc8c0 2024-09-19T05:42:27.372 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7047fff640 1 -- 172.21.15.2:0/2868421322 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7048032440 con 0x7f70500cc8c0 2024-09-19T05:42:27.372 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7047fff640 1 -- 172.21.15.2:0/2868421322 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048032810 con 0x7f70500cc8c0 2024-09-19T05:42:27.372 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 -- 172.21.15.2:0/2868421322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cc8c0 msgr2=0x7f7050154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.372 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 --2- 172.21.15.2:0/2868421322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cc8c0 0x7f7050154570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f705008e0c0 tx=0x7f704802a270 comp rx=0 tx=0).stop 2024-09-19T05:42:27.373 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 -- 172.21.15.2:0/2868421322 shutdown_connections 2024-09-19T05:42:27.373 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 --2- 172.21.15.2:0/2868421322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cc8c0 0x7f7050154570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.373 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 --2- 172.21.15.2:0/2868421322 >> v2:172.21.15.123:3300/0 conn(0x7f70500cb860 0x7f705015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.373 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 --2- 172.21.15.2:0/2868421322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cb480 0x7f70500cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.373 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 -- 172.21.15.2:0/2868421322 >> 172.21.15.2:0/2868421322 conn(0x7f7050067ec0 msgr2=0x7f705006e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:27.373 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 -- 172.21.15.2:0/2868421322 shutdown_connections 2024-09-19T05:42:27.373 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.369+0000 7f7057877640 1 -- 172.21.15.2:0/2868421322 wait complete. 2024-09-19T05:42:27.374 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.370+0000 7f7057877640 1 Processor -- start 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.370+0000 7f7057877640 1 -- start start 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7057877640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cb480 0x7f705014a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7057877640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70500cb860 0x7f70501f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7057877640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cc8c0 0x7f70501f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7057877640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f705015ae50 con 0x7f70500cb480 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7057877640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f705015e940 con 0x7f70500cb860 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7057877640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f705015afd0 con 0x7f70500cc8c0 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7055613640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cb480 0x7f705014a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7055613640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cb480 0x7f705014a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57060/0 (socket says 172.21.15.2:57060) 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7055613640 1 -- 172.21.15.2:0/2005958440 learned_addr learned my addr 172.21.15.2:0/2005958440 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7054e12640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70500cb860 0x7f70501f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.371+0000 7f7055e14640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cc8c0 0x7f70501f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7055613640 1 -- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cc8c0 msgr2=0x7f70501f07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.375 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7055613640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cc8c0 0x7f70501f07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7055613640 1 -- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70500cb860 msgr2=0x7f70501f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7055613640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70500cb860 0x7f70501f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7055613640 1 -- 172.21.15.2:0/2005958440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705015a1a0 con 0x7f70500cb480 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7055613640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cb480 0x7f705014a2a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f704c00ce30 tx=0x7f704c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7045ffb640 1 -- 172.21.15.2:0/2005958440 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c00f680 con 0x7f70500cb480 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7045ffb640 1 -- 172.21.15.2:0/2005958440 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c00f820 con 0x7f70500cb480 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705015ab10 con 0x7f70500cb480 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7045ffb640 1 -- 172.21.15.2:0/2005958440 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c00fc10 con 0x7f70500cb480 2024-09-19T05:42:27.376 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.372+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70501f3c50 con 0x7f70500cb480 2024-09-19T05:42:27.377 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.373+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70500c5250 con 0x7f70500cb480 2024-09-19T05:42:27.377 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.374+0000 7f7045ffb640 1 -- 172.21.15.2:0/2005958440 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f704c018400 con 0x7f70500cb480 2024-09-19T05:42:27.377 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.374+0000 7f7045ffb640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f703406c2b0 0x7f703406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.377 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.374+0000 7f7045ffb640 1 -- 172.21.15.2:0/2005958440 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f704c08be90 con 0x7f70500cb480 2024-09-19T05:42:27.377 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.374+0000 7f7054e12640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f703406c2b0 0x7f703406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.377 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.375+0000 7f7054e12640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f703406c2b0 0x7f703406e770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f70500d14d0 tx=0x7f7040008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:27.381 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.379+0000 7f7045ffb640 1 -- 172.21.15.2:0/2005958440 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f704c014030 con 0x7f70500cb480 2024-09-19T05:42:27.559 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.557+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7050159e70 con 0x7f70500cb480 2024-09-19T05:42:27.560 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.558+0000 7f7045ffb640 1 -- 172.21.15.2:0/2005958440 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12848 (secure 0 0 0) 0x7f704c05a390 con 0x7f70500cb480 2024-09-19T05:42:27.560 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-19T05:42:27.560 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":18,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:23.873042+0000","last_up_change":"2024-09-19T05:42:20.842106+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f703406c2b0 msgr2=0x7f703406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f703406c2b0 0x7f703406e770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f70500d14d0 tx=0x7f7040008040 comp rx=0 tx=0).stop 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cb480 msgr2=0x7f705014a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cb480 0x7f705014a2a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f704c00ce30 tx=0x7f704c00a650 comp rx=0 tx=0).stop 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 shutdown_connections 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f703406c2b0 0x7f703406e770 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70500cb860 0x7f70501f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.562 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70500cc8c0 0x7f70501f07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.563 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 --2- 172.21.15.2:0/2005958440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70500cb480 0x7f705014a2a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.563 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 >> 172.21.15.2:0/2005958440 conn(0x7f7050067ec0 msgr2=0x7f70500cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:27.563 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 shutdown_connections 2024-09-19T05:42:27.563 INFO:teuthology.misc.health.smithi002.stderr:2024-09-19T05:42:27.560+0000 7f7057877640 1 -- 172.21.15.2:0/2005958440 wait complete. 2024-09-19T05:42:27.571 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-19T05:42:27.571 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-19T05:42:27.572 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T05:42:27.572 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T05:42:27.574 INFO:tasks.ceph.mds.a:Started 2024-09-19T05:42:27.574 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T05:42:27.574 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T05:42:27.576 INFO:tasks.ceph.mds.c:Started 2024-09-19T05:42:27.576 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-19T05:42:27.576 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-19T05:42:27.578 INFO:tasks.ceph.mds.e:Started 2024-09-19T05:42:27.578 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T05:42:27.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T05:42:27.581 INFO:tasks.ceph.mds.b:Started 2024-09-19T05:42:27.582 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T05:42:27.582 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T05:42:27.584 INFO:tasks.ceph.mds.d:Started 2024-09-19T05:42:27.584 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-19T05:42:27.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-19T05:42:27.586 INFO:tasks.ceph.mds.f:Started 2024-09-19T05:42:27.587 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-19T05:42:27.587 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:42:27.653 INFO:tasks.ceph.mds.a.smithi002.stdout:starting mds.a at 2024-09-19T05:42:27.654 INFO:tasks.ceph.mds.c.smithi002.stdout:starting mds.c at 2024-09-19T05:42:27.656 INFO:tasks.ceph.mds.e.smithi002.stdout:starting mds.e at 2024-09-19T05:42:27.669 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2024-09-19T05:42:27.669 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2024-09-19T05:42:27.670 INFO:tasks.ceph.mds.f.smithi123.stdout:starting mds.f at 2024-09-19T05:42:27.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.695+0000 7f3b5f4f1640 1 Processor -- start 2024-09-19T05:42:27.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.695+0000 7f3b5f4f1640 1 -- start start 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5f4f1640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3b58073220 0x7f3b5806bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5f4f1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58155940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5f4f1640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b58158960 con 0x7f3b58072e40 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5f4f1640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3b580c7b40 con 0x7f3b58073220 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5d28d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3b58072e40 0x7f3b580722c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46384/0 (socket says 172.21.15.2:46384) 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5d28d640 1 -- 172.21.15.2:0/733595911 learned_addr learned my addr 172.21.15.2:0/733595911 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/733595911 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3b580c7e20 con 0x7f3b5815caa0 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5da8e640 1 --2- 172.21.15.2:0/733595911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5ca8c640 1 --2- 172.21.15.2:0/733595911 >> v2:172.21.15.123:3300/0 conn(0x7f3b58073220 0x7f3b5806bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b4ffff640 1 -- 172.21.15.2:0/733595911 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525114407 0 0) 0x7f3b58158960 con 0x7f3b58072e40 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b4ffff640 1 -- 172.21.15.2:0/733595911 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b44003520 con 0x7f3b58072e40 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5da8e640 1 -- 172.21.15.2:0/733595911 >> v2:172.21.15.123:3300/0 conn(0x7f3b58073220 msgr2=0x7f3b5806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:27.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5da8e640 1 --2- 172.21.15.2:0/733595911 >> v2:172.21.15.123:3300/0 conn(0x7f3b58073220 0x7f3b5806bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b5da8e640 1 -- 172.21.15.2:0/733595911 >> v1:172.21.15.2:6789/0 conn(0x7f3b58072e40 legacy=0x7f3b580722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.696+0000 7f3b4ffff640 1 -- 172.21.15.2:0/733595911 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1969381981 0 0) 0x7f3b44003520 con 0x7f3b58072e40 2024-09-19T05:42:27.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.697+0000 7f3b5da8e640 1 -- 172.21.15.2:0/733595911 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b58158c20 con 0x7f3b5815caa0 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.697+0000 7f3b5da8e640 1 --2- 172.21.15.2:0/733595911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58155940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009d90 tx=0x7f3b5002a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b23c70d0f336536e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.697+0000 7f3b4ffff640 1 -- 172.21.15.2:0/733595911 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54001670 con 0x7f3b5815caa0 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.697+0000 7f3b4ffff640 1 -- 172.21.15.2:0/733595911 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b50007540 con 0x7f3b5815caa0 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b4ffff640 1 -- 172.21.15.2:0/733595911 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50007910 con 0x7f3b5815caa0 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/733595911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 msgr2=0x7f3b58155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/733595911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58155940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009d90 tx=0x7f3b5002a560 comp rx=0 tx=0).stop 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/733595911 shutdown_connections 2024-09-19T05:42:27.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/733595911 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58155940 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/733595911 >> v2:172.21.15.123:3300/0 conn(0x7f3b58073220 0x7f3b5806bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/733595911 >> 172.21.15.2:0/733595911 conn(0x7f3b5806e450 msgr2=0x7f3b5806e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:27.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/733595911 shutdown_connections 2024-09-19T05:42:27.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.698+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/733595911 wait complete. 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.700+0000 7f3b5f4f1640 1 Processor -- start 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.700+0000 7f3b5f4f1640 1 -- start start 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.700+0000 7f3b5f4f1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b58072e40 0x7f3b5814edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.700+0000 7f3b5f4f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b58073220 0x7f3b58157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5f4f1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5f4f1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5806c150 con 0x7f3b58072e40 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b58072e40 0x7f3b5814edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5f4f1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b581591e0 con 0x7f3b58073220 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b58072e40 0x7f3b5814edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57112/0 (socket says 172.21.15.2:57112) 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 -- 172.21.15.2:0/2238680204 learned_addr learned my addr 172.21.15.2:0/2238680204 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3b580c77e0 con 0x7f3b5815caa0 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5ca8c640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b58073220 0x7f3b58157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5da8e640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 -- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 msgr2=0x7f3b58152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 -- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b58073220 msgr2=0x7f3b58157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b58073220 0x7f3b58157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 -- 172.21.15.2:0/2238680204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b58157ca0 con 0x7f3b58072e40 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5d28d640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b58072e40 0x7f3b5814edd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400c700 tx=0x7f3b5400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b4dffb640 1 -- 172.21.15.2:0/2238680204 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54017070 con 0x7f3b58072e40 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b58157f30 con 0x7f3b58072e40 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b4dffb640 1 -- 172.21.15.2:0/2238680204 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b54005ad0 con 0x7f3b58072e40 2024-09-19T05:42:27.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.701+0000 7f3b4dffb640 1 -- 172.21.15.2:0/2238680204 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54007070 con 0x7f3b58072e40 2024-09-19T05:42:27.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.702+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b581eeea0 con 0x7f3b58072e40 2024-09-19T05:42:27.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.703+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b58154e40 con 0x7f3b58072e40 2024-09-19T05:42:27.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.703+0000 7f3b4dffb640 1 -- 172.21.15.2:0/2238680204 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3b5401a070 con 0x7f3b58072e40 2024-09-19T05:42:27.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.703+0000 7f3b4dffb640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3b3c06c2b0 0x7f3b3c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:27.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.703+0000 7f3b4dffb640 1 -- 172.21.15.2:0/2238680204 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f3b5408a4a0 con 0x7f3b58072e40 2024-09-19T05:42:27.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.704+0000 7f3b5ca8c640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3b3c06c2b0 0x7f3b3c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:27.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.704+0000 7f3b5ca8c640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3b3c06c2b0 0x7f3b3c06e770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3b5814a3e0 tx=0x7f3b48009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:27.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.708+0000 7f3b4dffb640 1 -- 172.21.15.2:0/2238680204 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3b58154e40 con 0x7f3b58072e40 2024-09-19T05:42:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.917+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b58159700 con 0x7f3b58072e40 2024-09-19T05:42:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.917+0000 7f3b4dffb640 1 -- 172.21.15.2:0/2238680204 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f3b54058910 con 0x7f3b58072e40 2024-09-19T05:42:27.920 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:27.920 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":2,"btime":"2024-09-19T05:42:27:656967+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T05:42:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 2 2024-09-19T05:42:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3b3c06c2b0 msgr2=0x7f3b3c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3b3c06c2b0 0x7f3b3c06e770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3b5814a3e0 tx=0x7f3b48009190 comp rx=0 tx=0).stop 2024-09-19T05:42:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b58072e40 msgr2=0x7f3b5814edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b58072e40 0x7f3b5814edd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400c700 tx=0x7f3b5400cbc0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 shutdown_connections 2024-09-19T05:42:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3b3c06c2b0 0x7f3b3c06e770 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b58073220 0x7f3b58157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b5815caa0 0x7f3b58152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 --2- 172.21.15.2:0/2238680204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3b58072e40 0x7f3b5814edd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.919+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 >> 172.21.15.2:0/2238680204 conn(0x7f3b5806e450 msgr2=0x7f3b580caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.920+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 shutdown_connections 2024-09-19T05:42:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:27.920+0000 7f3b5f4f1640 1 -- 172.21.15.2:0/2238680204 wait complete. 2024-09-19T05:42:29.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:42:30.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.045+0000 7f45a7ea6640 1 Processor -- start 2024-09-19T05:42:30.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.045+0000 7f45a7ea6640 1 -- start start 2024-09-19T05:42:30.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.046+0000 7f45a7ea6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f45a00d06e0 0x7f45a00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.046+0000 7f45a7ea6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f45a015eb50 0x7f45a00cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.046+0000 7f45a7ea6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45a00cb400 con 0x7f45a00d0ac0 2024-09-19T05:42:30.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.046+0000 7f45a7ea6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f45a00cb6e0 con 0x7f45a00d06e0 2024-09-19T05:42:30.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.046+0000 7f45a7ea6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f45a00cc970 con 0x7f45a015eb50 2024-09-19T05:42:30.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.046+0000 7f45a5c42640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f45a00d06e0 0x7f45a00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.047+0000 7f45a5c42640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f45a00d06e0 0x7f45a00cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57118/0 (socket says 172.21.15.2:57118) 2024-09-19T05:42:30.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.047+0000 7f45a5c42640 1 -- 172.21.15.2:0/184055301 learned_addr learned my addr 172.21.15.2:0/184055301 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:30.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.047+0000 7f45a6443640 1 --2- 172.21.15.2:0/184055301 >> v2:172.21.15.123:3300/0 conn(0x7f45a015eb50 0x7f45a00cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.047+0000 7f45a5c42640 1 -- 172.21.15.2:0/184055301 >> v2:172.21.15.123:3300/0 conn(0x7f45a015eb50 msgr2=0x7f45a00cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:30.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.047+0000 7f45a5c42640 1 --2- 172.21.15.2:0/184055301 >> v2:172.21.15.123:3300/0 conn(0x7f45a015eb50 0x7f45a00cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.047+0000 7f45a5c42640 1 -- 172.21.15.2:0/184055301 >> v1:172.21.15.123:6789/0 conn(0x7f45a00d0ac0 legacy=0x7f45a0154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:30.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.047+0000 7f45a5c42640 1 -- 172.21.15.2:0/184055301 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a00ccc50 con 0x7f45a00d06e0 2024-09-19T05:42:30.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a5c42640 1 --2- 172.21.15.2:0/184055301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00cbb40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45a00ce390 tx=0x7f459002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacf199a39fae718 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a4c40640 1 -- 172.21.15.2:0/184055301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590036070 con 0x7f45a00d06e0 2024-09-19T05:42:30.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a4c40640 1 -- 172.21.15.2:0/184055301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4590031600 con 0x7f45a00d06e0 2024-09-19T05:42:30.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a4c40640 1 -- 172.21.15.2:0/184055301 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45900319d0 con 0x7f45a00d06e0 2024-09-19T05:42:30.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 -- 172.21.15.2:0/184055301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 msgr2=0x7f45a00cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/184055301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00cbb40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45a00ce390 tx=0x7f459002a730 comp rx=0 tx=0).stop 2024-09-19T05:42:30.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 -- 172.21.15.2:0/184055301 shutdown_connections 2024-09-19T05:42:30.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/184055301 >> v2:172.21.15.123:3300/0 conn(0x7f45a015eb50 0x7f45a00cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/184055301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00cbb40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 -- 172.21.15.2:0/184055301 >> 172.21.15.2:0/184055301 conn(0x7f45a006d740 msgr2=0x7f45a0073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:30.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 -- 172.21.15.2:0/184055301 shutdown_connections 2024-09-19T05:42:30.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.048+0000 7f45a7ea6640 1 -- 172.21.15.2:0/184055301 wait complete. 2024-09-19T05:42:30.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 Processor -- start 2024-09-19T05:42:30.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 -- start start 2024-09-19T05:42:30.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45a00d0ac0 0x7f45a0153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a015eb50 0x7f45a0155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f45a00c5830 con 0x7f45a00d06e0 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45a015efc0 con 0x7f45a015eb50 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a7ea6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f45a0155200 con 0x7f45a00d0ac0 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.050+0000 7f45a5c42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57128/0 (socket says 172.21.15.2:57128) 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 -- 172.21.15.2:0/1604154951 learned_addr learned my addr 172.21.15.2:0/1604154951 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a6443640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a015eb50 0x7f45a0155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 -- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45a00d0ac0 msgr2=0x7f45a0153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45a00d0ac0 0x7f45a0153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 -- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a015eb50 msgr2=0x7f45a0155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a015eb50 0x7f45a0155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 -- 172.21.15.2:0/1604154951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a01f4880 con 0x7f45a00d06e0 2024-09-19T05:42:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a5c42640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00c9f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4590009f40 tx=0x7f4590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f459e7fc640 1 -- 172.21.15.2:0/1604154951 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459003f020 con 0x7f45a00d06e0 2024-09-19T05:42:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f459e7fc640 1 -- 172.21.15.2:0/1604154951 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4590037040 con 0x7f45a00d06e0 2024-09-19T05:42:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a00c8ae0 con 0x7f45a00d06e0 2024-09-19T05:42:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f459e7fc640 1 -- 172.21.15.2:0/1604154951 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590036070 con 0x7f45a00d06e0 2024-09-19T05:42:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.051+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a00c90a0 con 0x7f45a00d06e0 2024-09-19T05:42:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.052+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a00cbb40 con 0x7f45a00d06e0 2024-09-19T05:42:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.053+0000 7f459e7fc640 1 -- 172.21.15.2:0/1604154951 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4590042020 con 0x7f45a00d06e0 2024-09-19T05:42:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.053+0000 7f459e7fc640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f458c06c2e0 0x7f458c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.054+0000 7f459e7fc640 1 -- 172.21.15.2:0/1604154951 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f45900ab490 con 0x7f45a00d06e0 2024-09-19T05:42:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.054+0000 7f45a5441640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f458c06c2e0 0x7f458c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.054+0000 7f45a5441640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f458c06c2e0 0x7f458c06e7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4594000f60 tx=0x7f45940079d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.058+0000 7f459e7fc640 1 -- 172.21.15.2:0/1604154951 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4590049030 con 0x7f45a00d06e0 2024-09-19T05:42:30.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.269+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45a00c5f30 con 0x7f45a00d06e0 2024-09-19T05:42:30.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.269+0000 7f459e7fc640 1 -- 172.21.15.2:0/1604154951 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5551 (secure 0 0 0) 0x7f459007ee20 con 0x7f45a00d06e0 2024-09-19T05:42:30.272 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:30.272 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"btime":"2024-09-19T05:42:28:658592+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/121690198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":121690198},{"type":"v1","addr":"172.21.15.2:6835","nonce":121690198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1118046326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1118046326},{"type":"v1","addr":"172.21.15.123:6837","nonce":1118046326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3264935286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3264935286},{"type":"v1","addr":"172.21.15.2:6837","nonce":3264935286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/4246839966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4246839966},{"type":"v1","addr":"172.21.15.123:6836","nonce":4246839966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4312,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/4093916922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4093916922},{"type":"v1","addr":"172.21.15.123:6839","nonce":4093916922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T05:42:30.272 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2024-09-19T05:42:30.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.271+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f458c06c2e0 msgr2=0x7f458c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.271+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f458c06c2e0 0x7f458c06e7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4594000f60 tx=0x7f45940079d0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.271+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 msgr2=0x7f45a00c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.271+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00c9f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4590009f40 tx=0x7f4590004a40 comp rx=0 tx=0).stop 2024-09-19T05:42:30.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 shutdown_connections 2024-09-19T05:42:30.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f458c06c2e0 0x7f458c06e7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45a015eb50 0x7f45a0155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45a00d0ac0 0x7f45a0153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 --2- 172.21.15.2:0/1604154951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45a00d06e0 0x7f45a00c9f00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 >> 172.21.15.2:0/1604154951 conn(0x7f45a006d740 msgr2=0x7f45a0073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:30.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 shutdown_connections 2024-09-19T05:42:30.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.272+0000 7f45a7ea6640 1 -- 172.21.15.2:0/1604154951 wait complete. 2024-09-19T05:42:30.285 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T05:42:30.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.385+0000 7f5e275c4640 1 Processor -- start 2024-09-19T05:42:30.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.385+0000 7f5e275c4640 1 -- start start 2024-09-19T05:42:30.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e275c4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5e20154cb0 0x7f5e2015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e275c4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e275c4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5e200c55f0 0x7f5e200cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e275c4640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f5e200cdd00 con 0x7f5e200c55f0 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e275c4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5e200cd330 con 0x7f5e20154cb0 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e275c4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5e200734c0 con 0x7f5e200cc5f0 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5e200c55f0 0x7f5e200cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5e200c55f0 0x7f5e200cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57136/0 (socket says 172.21.15.2:57136) 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 -- 172.21.15.2:0/388505157 learned_addr learned my addr 172.21.15.2:0/388505157 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25360640 1 --2- 172.21.15.2:0/388505157 >> v2:172.21.15.123:3300/0 conn(0x7f5e20154cb0 0x7f5e2015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e24b5f640 1 --2- 172.21.15.2:0/388505157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 -- 172.21.15.2:0/388505157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 msgr2=0x7f5e200cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:30.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 --2- 172.21.15.2:0/388505157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 -- 172.21.15.2:0/388505157 >> v2:172.21.15.123:3300/0 conn(0x7f5e20154cb0 msgr2=0x7f5e2015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 --2- 172.21.15.2:0/388505157 >> v2:172.21.15.123:3300/0 conn(0x7f5e20154cb0 0x7f5e2015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.386+0000 7f5e25b61640 1 -- 172.21.15.2:0/388505157 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e20073780 con 0x7f5e200c55f0 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.387+0000 7f5e25b61640 1 --2- 172.21.15.2:0/388505157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e200c55f0 0x7f5e200cadc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e200d0980 tx=0x7f5e1802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22f8f5cf81f5f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.387+0000 7f5e17fff640 1 -- 172.21.15.2:0/388505157 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18036070 con 0x7f5e200c55f0 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.387+0000 7f5e17fff640 1 -- 172.21.15.2:0/388505157 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e18031600 con 0x7f5e200c55f0 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.387+0000 7f5e17fff640 1 -- 172.21.15.2:0/388505157 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e180319d0 con 0x7f5e200c55f0 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 -- 172.21.15.2:0/388505157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e200c55f0 msgr2=0x7f5e200cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 --2- 172.21.15.2:0/388505157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e200c55f0 0x7f5e200cadc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e200d0980 tx=0x7f5e1802a730 comp rx=0 tx=0).stop 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 -- 172.21.15.2:0/388505157 shutdown_connections 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 --2- 172.21.15.2:0/388505157 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e200c55f0 0x7f5e200cadc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 --2- 172.21.15.2:0/388505157 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 --2- 172.21.15.2:0/388505157 >> v2:172.21.15.123:3300/0 conn(0x7f5e20154cb0 0x7f5e2015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 -- 172.21.15.2:0/388505157 >> 172.21.15.2:0/388505157 conn(0x7f5e2006d0c0 msgr2=0x7f5e2006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:30.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 -- 172.21.15.2:0/388505157 shutdown_connections 2024-09-19T05:42:30.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.388+0000 7f5e275c4640 1 -- 172.21.15.2:0/388505157 wait complete. 2024-09-19T05:42:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.389+0000 7f5e275c4640 1 Processor -- start 2024-09-19T05:42:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.389+0000 7f5e275c4640 1 -- start start 2024-09-19T05:42:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e275c4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e200c55f0 0x7f5e201f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e275c4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e275c4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e20154cb0 0x7f5e201f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e275c4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5e200b30d0 con 0x7f5e20154cb0 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e275c4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e200ccfc0 con 0x7f5e200c55f0 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e275c4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5e200c5a80 con 0x7f5e200cc5f0 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e20154cb0 0x7f5e201f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e20154cb0 0x7f5e201f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57150/0 (socket says 172.21.15.2:57150) 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 -- 172.21.15.2:0/3283436084 learned_addr learned my addr 172.21.15.2:0/3283436084 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25360640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e200c55f0 0x7f5e201f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e24b5f640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 -- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 msgr2=0x7f5e200cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 -- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e200c55f0 msgr2=0x7f5e201f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e200c55f0 0x7f5e201f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 -- 172.21.15.2:0/3283436084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e200cf280 con 0x7f5e20154cb0 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e24b5f640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e25b61640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e20154cb0 0x7f5e201f1090 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e1802ad20 tx=0x7f5e18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e15ffb640 1 -- 172.21.15.2:0/3283436084 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18041020 con 0x7f5e20154cb0 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e15ffb640 1 -- 172.21.15.2:0/3283436084 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e180077a0 con 0x7f5e20154cb0 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e15ffb640 1 -- 172.21.15.2:0/3283436084 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e18036040 con 0x7f5e20154cb0 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.390+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e200cf660 con 0x7f5e20154cb0 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.391+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e200cfbe0 con 0x7f5e20154cb0 2024-09-19T05:42:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.392+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e200cb8b0 con 0x7f5e20154cb0 2024-09-19T05:42:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.392+0000 7f5e15ffb640 1 -- 172.21.15.2:0/3283436084 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5e18044020 con 0x7f5e20154cb0 2024-09-19T05:42:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.392+0000 7f5e15ffb640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5e0406c2b0 0x7f5e0406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.393+0000 7f5e15ffb640 1 -- 172.21.15.2:0/3283436084 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f5e180abf10 con 0x7f5e20154cb0 2024-09-19T05:42:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.393+0000 7f5e25360640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5e0406c2b0 0x7f5e0406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.393+0000 7f5e25360640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5e0406c2b0 0x7f5e0406e770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009aa0 tx=0x7f5e1c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.397+0000 7f5e15ffb640 1 -- 172.21.15.2:0/3283436084 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5e1804b020 con 0x7f5e20154cb0 2024-09-19T05:42:30.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.574+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5e201538a0 con 0x7f5e20154cb0 2024-09-19T05:42:30.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.574+0000 7f5e15ffb640 1 -- 172.21.15.2:0/3283436084 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f5e1807a310 con 0x7f5e20154cb0 2024-09-19T05:42:30.576 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:30.577 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2024-09-19T05:42:30.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:30.577 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2024-09-19T05:42:30.577 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2024-09-19T05:42:30.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:30.577 INFO:teuthology.orchestra.run.smithi002.stdout:] 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5e0406c2b0 msgr2=0x7f5e0406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5e0406c2b0 0x7f5e0406e770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009aa0 tx=0x7f5e1c009190 comp rx=0 tx=0).stop 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e20154cb0 msgr2=0x7f5e201f1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e20154cb0 0x7f5e201f1090 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e1802ad20 tx=0x7f5e18004a40 comp rx=0 tx=0).stop 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 shutdown_connections 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5e0406c2b0 0x7f5e0406e770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e20154cb0 0x7f5e201f1090 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5e200cc5f0 0x7f5e200cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 --2- 172.21.15.2:0/3283436084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e200c55f0 0x7f5e201f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 >> 172.21.15.2:0/3283436084 conn(0x7f5e2006d0c0 msgr2=0x7f5e20151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 shutdown_connections 2024-09-19T05:42:30.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.576+0000 7f5e275c4640 1 -- 172.21.15.2:0/3283436084 wait complete. 2024-09-19T05:42:30.589 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T05:42:30.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T05:42:30.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.693+0000 7fa08cc7b640 1 Processor -- start 2024-09-19T05:42:30.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.693+0000 7fa08cc7b640 1 -- start start 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa08cc7b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa088154dc0 0x7fa08815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa08cc7b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0880cb620 0x7fa08806d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa08cc7b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa08815b050 con 0x7fa0880cbf40 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa08cc7b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa08806cdf0 con 0x7fa0880cb620 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa08cc7b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa08815b2c0 con 0x7fa088154dc0 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa086d9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa0880cbf40 0x7fa08815e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46392/0 (socket says 172.21.15.2:46392) 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa086d9d640 1 -- 172.21.15.2:0/1824299224 learned_addr learned my addr 172.21.15.2:0/1824299224 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:30.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa08659c640 1 --2- 172.21.15.2:0/1824299224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa088154dc0 0x7fa08815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.694+0000 7fa085d9b640 1 --2- 172.21.15.2:0/1824299224 >> v2:172.21.15.123:3300/0 conn(0x7fa0880cb620 0x7fa08806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.695+0000 7fa08559a640 1 -- 172.21.15.2:0/1824299224 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620606696 0 0) 0x7fa08815b050 con 0x7fa0880cbf40 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.695+0000 7fa08559a640 1 -- 172.21.15.2:0/1824299224 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa074003520 con 0x7fa0880cbf40 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.695+0000 7fa08659c640 1 -- 172.21.15.2:0/1824299224 >> v2:172.21.15.123:3300/0 conn(0x7fa0880cb620 msgr2=0x7fa08806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.695+0000 7fa08659c640 1 --2- 172.21.15.2:0/1824299224 >> v2:172.21.15.123:3300/0 conn(0x7fa0880cb620 0x7fa08806d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.695+0000 7fa08659c640 1 -- 172.21.15.2:0/1824299224 >> v1:172.21.15.2:6789/0 conn(0x7fa0880cbf40 legacy=0x7fa08815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.695+0000 7fa08559a640 1 -- 172.21.15.2:0/1824299224 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3929765448 0 0) 0x7fa074003520 con 0x7fa0880cbf40 2024-09-19T05:42:30.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.695+0000 7fa08659c640 1 -- 172.21.15.2:0/1824299224 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08815b5a0 con 0x7fa088154dc0 2024-09-19T05:42:30.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08659c640 1 --2- 172.21.15.2:0/1824299224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa088154dc0 0x7fa08815ab10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa078009d90 tx=0x7fa07802a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d273a2540357a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08559a640 1 -- 172.21.15.2:0/1824299224 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080001670 con 0x7fa088154dc0 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08559a640 1 -- 172.21.15.2:0/1824299224 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa078007540 con 0x7fa088154dc0 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08559a640 1 -- 172.21.15.2:0/1824299224 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0780078f0 con 0x7fa088154dc0 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1824299224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa088154dc0 msgr2=0x7fa08815ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1824299224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa088154dc0 0x7fa08815ab10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa078009d90 tx=0x7fa07802a560 comp rx=0 tx=0).stop 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1824299224 shutdown_connections 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1824299224 >> v2:172.21.15.123:3300/0 conn(0x7fa0880cb620 0x7fa08806d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1824299224 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa088154dc0 0x7fa08815ab10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1824299224 >> 172.21.15.2:0/1824299224 conn(0x7fa088067950 msgr2=0x7fa088067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1824299224 shutdown_connections 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.696+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1824299224 wait complete. 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 Processor -- start 2024-09-19T05:42:30.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 -- start start 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0880cb620 0x7fa0881589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0880cbf40 0x7fa0880c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa088154dc0 0x7fa0880ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa0880b34e0 con 0x7fa0880cbf40 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa08806ca80 con 0x7fa088154dc0 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa08cc7b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa08806d7c0 con 0x7fa0880cb620 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa085d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0880cbf40 0x7fa0880c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.698+0000 7fa085d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0880cbf40 0x7fa0880c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57164/0 (socket says 172.21.15.2:57164) 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa085d9b640 1 -- 172.21.15.2:0/1377464393 learned_addr learned my addr 172.21.15.2:0/1377464393 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:30.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa086d9d640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa088154dc0 0x7fa0880ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa08659c640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0880cb620 0x7fa0881589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa085d9b640 1 -- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0880cb620 msgr2=0x7fa0881589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa085d9b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0880cb620 0x7fa0881589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa085d9b640 1 -- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa088154dc0 msgr2=0x7fa0880ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa085d9b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa088154dc0 0x7fa0880ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa085d9b640 1 -- 172.21.15.2:0/1377464393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0881f1140 con 0x7fa0880cbf40 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa085d9b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0880cbf40 0x7fa0880c9f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e9e0 tx=0x7fa07000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa07effd640 1 -- 172.21.15.2:0/1377464393 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070017070 con 0x7fa0880cbf40 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa07effd640 1 -- 172.21.15.2:0/1377464393 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07000c9d0 con 0x7fa0880cbf40 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa088153d90 con 0x7fa0880cbf40 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa07effd640 1 -- 172.21.15.2:0/1377464393 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000cdc0 con 0x7fa0880cbf40 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.699+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0880c7b30 con 0x7fa0880cbf40 2024-09-19T05:42:30.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.700+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0881551a0 con 0x7fa0880cbf40 2024-09-19T05:42:30.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.701+0000 7fa07effd640 1 -- 172.21.15.2:0/1377464393 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa07001a070 con 0x7fa0880cbf40 2024-09-19T05:42:30.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.701+0000 7fa07effd640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa06806c2e0 0x7fa06806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:30.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.701+0000 7fa07effd640 1 -- 172.21.15.2:0/1377464393 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fa07008b070 con 0x7fa0880cbf40 2024-09-19T05:42:30.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.701+0000 7fa08659c640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa06806c2e0 0x7fa06806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:30.706 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.702+0000 7fa08659c640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa06806c2e0 0x7fa06806e7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa078034000 tx=0x7fa07802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:30.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.705+0000 7fa07effd640 1 -- 172.21.15.2:0/1377464393 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa0700594e0 con 0x7fa0880cbf40 2024-09-19T05:42:30.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.881+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa0880c5350 con 0x7fa0880cbf40 2024-09-19T05:42:30.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.963+0000 7fa07effd640 1 -- 172.21.15.2:0/1377464393 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fa07005d7e0 con 0x7fa0880cbf40 2024-09-19T05:42:30.966 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.967+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa06806c2e0 msgr2=0x7fa06806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.967+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa06806c2e0 0x7fa06806e7a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa078034000 tx=0x7fa07802f000 comp rx=0 tx=0).stop 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.967+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0880cbf40 msgr2=0x7fa0880c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.967+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0880cbf40 0x7fa0880c9f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e9e0 tx=0x7fa07000eea0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.968+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 shutdown_connections 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.968+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa06806c2e0 0x7fa06806e7a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.968+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa088154dc0 0x7fa0880ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.968+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0880cbf40 0x7fa0880c9f40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.968+0000 7fa08cc7b640 1 --2- 172.21.15.2:0/1377464393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa0880cb620 0x7fa0881589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:30.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.968+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 >> 172.21.15.2:0/1377464393 conn(0x7fa088067950 msgr2=0x7fa0881e4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.969+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 shutdown_connections 2024-09-19T05:42:30.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:30.969+0000 7fa08cc7b640 1 -- 172.21.15.2:0/1377464393 wait complete. 2024-09-19T05:42:30.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T05:42:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.092+0000 7f789302c640 1 Processor -- start 2024-09-19T05:42:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.092+0000 7f789302c640 1 -- start start 2024-09-19T05:42:31.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.092+0000 7f789302c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 0x7f788c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.092+0000 7f789302c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f788c0ccbc0 con 0x7f788c0caf40 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.092+0000 7f789302c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f788c0cc600 con 0x7f788c155000 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.092+0000 7f789302c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f788c0cc8e0 con 0x7f788c154c20 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f78915c9640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f788c0caf40 0x7f788c15aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46396/0 (socket says 172.21.15.2:46396) 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f78915c9640 1 -- 172.21.15.2:0/3978984712 learned_addr learned my addr 172.21.15.2:0/3978984712 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1095648644 0 0) 0x7f788c0ccbc0 con 0x7f788c0caf40 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f7890dc8640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 0x7f788c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f787c003520 con 0x7f788c0caf40 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2138250157 0 0) 0x7f787c003520 con 0x7f788c0caf40 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 msgr2=0x7f788c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 0x7f788c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 >> v1:172.21.15.123:6789/0 conn(0x7f788c155000 legacy=0x7f788c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:31.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c0cbc60 con 0x7f788c0caf40 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f78840019f0 con 0x7f788c0caf40 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 >> v1:172.21.15.2:6789/0 conn(0x7f788c0caf40 legacy=0x7f788c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 0x7f787c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f787c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c004d40 0x7f787c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f788c0cc8e0 con 0x7f787c003f20 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f787c00ba20 con 0x7f787c004d40 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f787c00bc20 con 0x7f788c154c20 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.093+0000 7f788bfff640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f787c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f7890dc8640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 0x7f787c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f78915c9640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c004d40 0x7f787c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f788bfff640 1 -- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 msgr2=0x7f787c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f788bfff640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 0x7f787c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f788bfff640 1 -- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c004d40 msgr2=0x7f787c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f788bfff640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f787c004d40 0x7f787c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f788bfff640 1 -- 172.21.15.2:0/3978984712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787c003520 con 0x7f787c003f20 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f7890dc8640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c154c20 0x7f787c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.094+0000 7f788bfff640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f787c004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f788c1553e0 tx=0x7f787c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830749361a941d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878037070 con 0x7f787c003f20 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787802aae0 con 0x7f787c003f20 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f788b7fe640 1 -- 172.21.15.2:0/3978984712 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878001480 con 0x7f787c003f20 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f789302c640 1 -- 172.21.15.2:0/3978984712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 msgr2=0x7f787c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f789302c640 1 --2- 172.21.15.2:0/3978984712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f787c004800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f788c1553e0 tx=0x7f787c005830 comp rx=0 tx=0).stop 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f78915c9640 1 -- 172.21.15.2:0/3978984712 reap_dead start 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f789302c640 1 -- 172.21.15.2:0/3978984712 shutdown_connections 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f789302c640 1 -- 172.21.15.2:0/3978984712 >> 172.21.15.2:0/3978984712 conn(0x7f788c06d0e0 msgr2=0x7f788c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:31.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f789302c640 1 -- 172.21.15.2:0/3978984712 shutdown_connections 2024-09-19T05:42:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.095+0000 7f789302c640 1 -- 172.21.15.2:0/3978984712 wait complete. 2024-09-19T05:42:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 Processor -- start 2024-09-19T05:42:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 -- start start 2024-09-19T05:42:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f788c1e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c1f14c0 0x7f788c0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c0b5c20 0x7f788c1f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f788c073a50 con 0x7f787c003f20 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f788c0cb430 con 0x7f788c1f14c0 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f789302c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f788c073bd0 con 0x7f788c0b5c20 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f7890dc8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f788c1e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f7890dc8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f788c1e1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57178/0 (socket says 172.21.15.2:57178) 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.097+0000 7f7890dc8640 1 -- 172.21.15.2:0/590620113 learned_addr learned my addr 172.21.15.2:0/590620113 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f788bfff640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c1f14c0 0x7f788c0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f78915c9640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c0b5c20 0x7f788c1f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f7890dc8640 1 -- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c0b5c20 msgr2=0x7f788c1f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f7890dc8640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c0b5c20 0x7f788c1f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f7890dc8640 1 -- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c1f14c0 msgr2=0x7f788c0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f7890dc8640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c1f14c0 0x7f788c0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f7890dc8640 1 -- 172.21.15.2:0/590620113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c1e1240 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f7890dc8640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f788c1e1e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7880011ca0 tx=0x7f788000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f78897fa640 1 -- 172.21.15.2:0/590620113 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880011cd0 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f78897fa640 1 -- 172.21.15.2:0/590620113 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7880011e70 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c1f09e0 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f78897fa640 1 -- 172.21.15.2:0/590620113 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880014660 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.098+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c1f0fa0 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.099+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788c1f07d0 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.100+0000 7f78897fa640 1 -- 172.21.15.2:0/590620113 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7880012040 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.100+0000 7f78897fa640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f787006c2b0 0x7f787006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.100+0000 7f78897fa640 1 -- 172.21.15.2:0/590620113 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f788008ba50 con 0x7f787c003f20 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.101+0000 7f788bfff640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f787006c2b0 0x7f787006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:31.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.101+0000 7f788bfff640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f787006c2b0 0x7f787006e770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7878034000 tx=0x7f787802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:31.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.105+0000 7f78897fa640 1 -- 172.21.15.2:0/590620113 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7880019030 con 0x7f787c003f20 2024-09-19T05:42:31.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.281+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f788c0c5770 con 0x7f787c003f20 2024-09-19T05:42:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.967+0000 7f78897fa640 1 -- 172.21.15.2:0/590620113 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f788005a310 con 0x7f787c003f20 2024-09-19T05:42:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2024-09-19T05:42:31.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.970+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f787006c2b0 msgr2=0x7f787006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:31.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.970+0000 7f789302c640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f787006c2b0 0x7f787006e770 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7878034000 tx=0x7f787802e000 comp rx=0 tx=0).stop 2024-09-19T05:42:31.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.971+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 msgr2=0x7f788c1e1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:31.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.971+0000 7f789302c640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f788c1e1e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7880011ca0 tx=0x7f788000f610 comp rx=0 tx=0).stop 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 shutdown_connections 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f787006c2b0 0x7f787006e770 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c1f14c0 0x7f788c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f788c0b5c20 0x7f788c1f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 --2- 172.21.15.2:0/590620113 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f787c003f20 0x7f788c1e1e40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 >> 172.21.15.2:0/590620113 conn(0x7f788c06d0e0 msgr2=0x7f788c0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 shutdown_connections 2024-09-19T05:42:31.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:31.972+0000 7f789302c640 1 -- 172.21.15.2:0/590620113 wait complete. 2024-09-19T05:42:32.001 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T05:42:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.109+0000 7f04832af640 1 Processor -- start 2024-09-19T05:42:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.109+0000 7f04832af640 1 -- start start 2024-09-19T05:42:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f04832af640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f047c155000 0x7f047c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f04832af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c0caf40 0x7f047c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f04832af640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f047c0ccbc0 con 0x7f047c154c20 2024-09-19T05:42:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f04832af640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f047c0cc600 con 0x7f047c155000 2024-09-19T05:42:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f04832af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f047c0cc8e0 con 0x7f047c0caf40 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f048084a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f047c155000 0x7f047c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f048104b640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f047c154c20 0x7f047c073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:42836/0 (socket says 172.21.15.2:42836) 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f048104b640 1 -- 172.21.15.2:0/644817234 learned_addr learned my addr 172.21.15.2:0/644817234 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.110+0000 7f048184c640 1 --2- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c0caf40 0x7f047c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.111+0000 7f048084a640 1 -- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c0caf40 msgr2=0x7f047c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.111+0000 7f048084a640 1 --2- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c0caf40 0x7f047c15aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.111+0000 7f048084a640 1 -- 172.21.15.2:0/644817234 >> v1:172.21.15.123:6789/0 conn(0x7f047c154c20 legacy=0x7f047c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.111+0000 7f048084a640 1 -- 172.21.15.2:0/644817234 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c0cbc60 con 0x7f047c155000 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.111+0000 7f048084a640 1 --2- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c155000 0x7f047c0cd9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0470009790 tx=0x7f047002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6685368c4d6d19df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f047bfff640 1 -- 172.21.15.2:0/644817234 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470037070 con 0x7f047c155000 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f047bfff640 1 -- 172.21.15.2:0/644817234 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0470032440 con 0x7f047c155000 2024-09-19T05:42:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f047bfff640 1 -- 172.21.15.2:0/644817234 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470032810 con 0x7f047c155000 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 -- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c155000 msgr2=0x7f047c0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 --2- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c155000 0x7f047c0cd9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0470009790 tx=0x7f047002a180 comp rx=0 tx=0).stop 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 -- 172.21.15.2:0/644817234 shutdown_connections 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 --2- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c0caf40 0x7f047c15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 --2- 172.21.15.2:0/644817234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c155000 0x7f047c0cd9e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 -- 172.21.15.2:0/644817234 >> 172.21.15.2:0/644817234 conn(0x7f047c06d0e0 msgr2=0x7f047c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 -- 172.21.15.2:0/644817234 shutdown_connections 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.112+0000 7f04832af640 1 -- 172.21.15.2:0/644817234 wait complete. 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.113+0000 7f04832af640 1 Processor -- start 2024-09-19T05:42:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f04832af640 1 -- start start 2024-09-19T05:42:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f04832af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c0caf40 0x7f047c150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f04832af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c154c20 0x7f047c0b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f04832af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c155000 0x7f047c1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f04832af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f047c073a50 con 0x7f047c0caf40 2024-09-19T05:42:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f04832af640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f047c0cb430 con 0x7f047c154c20 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f04832af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f047c073bd0 con 0x7f047c155000 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f048104b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c0caf40 0x7f047c150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f048084a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c154c20 0x7f047c0b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f048104b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c0caf40 0x7f047c150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37342/0 (socket says 172.21.15.2:37342) 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f048084a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c154c20 0x7f047c0b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43746/0 (socket says 172.21.15.2:43746) 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f048104b640 1 -- 172.21.15.2:0/3293159018 learned_addr learned my addr 172.21.15.2:0/3293159018 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.114+0000 7f048184c640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c155000 0x7f047c1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f048104b640 1 -- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c155000 msgr2=0x7f047c1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f048104b640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c155000 0x7f047c1575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f048104b640 1 -- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c154c20 msgr2=0x7f047c0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f048104b640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c154c20 0x7f047c0b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f048104b640 1 -- 172.21.15.2:0/3293159018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c151540 con 0x7f047c0caf40 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f048104b640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c0caf40 0x7f047c150830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f046c00cb40 tx=0x7f046c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f0479ffb640 1 -- 172.21.15.2:0/3293159018 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c017070 con 0x7f047c0caf40 2024-09-19T05:42:32.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f0479ffb640 1 -- 172.21.15.2:0/3293159018 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c00aca0 con 0x7f047c0caf40 2024-09-19T05:42:32.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c150d70 con 0x7f047c0caf40 2024-09-19T05:42:32.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f0479ffb640 1 -- 172.21.15.2:0/3293159018 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c012620 con 0x7f047c0caf40 2024-09-19T05:42:32.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.115+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c151ac0 con 0x7f047c0caf40 2024-09-19T05:42:32.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.116+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047c0cac30 con 0x7f047c0caf40 2024-09-19T05:42:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.117+0000 7f0479ffb640 1 -- 172.21.15.2:0/3293159018 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f046c01a070 con 0x7f047c0caf40 2024-09-19T05:42:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.117+0000 7f0479ffb640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f046806c2e0 0x7f046806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.118+0000 7f0479ffb640 1 -- 172.21.15.2:0/3293159018 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f046c08b140 con 0x7f047c0caf40 2024-09-19T05:42:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.118+0000 7f048084a640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f046806c2e0 0x7f046806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.118+0000 7f048084a640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f046806c2e0 0x7f046806e7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0470034000 tx=0x7f047002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.122+0000 7f0479ffb640 1 -- 172.21.15.2:0/3293159018 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f047c0cac30 con 0x7f047c0caf40 2024-09-19T05:42:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.332+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f047c154760 con 0x7f047c0caf40 2024-09-19T05:42:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.969+0000 7f0479ffb640 1 -- 172.21.15.2:0/3293159018 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f046c059220 con 0x7f047c0caf40 2024-09-19T05:42:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f046806c2e0 msgr2=0x7f046806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f046806c2e0 0x7f046806e7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0470034000 tx=0x7f047002e000 comp rx=0 tx=0).stop 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c0caf40 msgr2=0x7f047c150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c0caf40 0x7f047c150830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f046c00cb40 tx=0x7f046c00a060 comp rx=0 tx=0).stop 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 shutdown_connections 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f046806c2e0 0x7f046806e7a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f047c154c20 0x7f047c0b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f047c155000 0x7f047c1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 --2- 172.21.15.2:0/3293159018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f047c0caf40 0x7f047c150830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.971+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 >> 172.21.15.2:0/3293159018 conn(0x7f047c06d0e0 msgr2=0x7f047c1e74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.972+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 shutdown_connections 2024-09-19T05:42:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:32.972+0000 7f04832af640 1 -- 172.21.15.2:0/3293159018 wait complete. 2024-09-19T05:42:32.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:33.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.092+0000 7ff0f219f640 1 Processor -- start 2024-09-19T05:42:33.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.093+0000 7ff0f219f640 1 -- start start 2024-09-19T05:42:33.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.093+0000 7ff0f219f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec154a00 0x7ff0ec06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.093+0000 7ff0f219f640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ec06c1d0 con 0x7ff0ec15b150 2024-09-19T05:42:33.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.093+0000 7ff0f219f640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ec06c470 con 0x7ff0ec0c53f0 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.093+0000 7ff0f219f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec154de0 con 0x7ff0ec154a00 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0f099c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff0ec15b150 0x7ff0ec0ccc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51474/0 (socket says 172.21.15.2:51474) 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0f099c640 1 -- 172.21.15.2:0/1070566916 learned_addr learned my addr 172.21.15.2:0/1070566916 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0f119d640 1 --2- 172.21.15.2:0/1070566916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec154a00 0x7ff0ec06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 823114398 0 0) 0x7ff0ec06c1d0 con 0x7ff0ec15b150 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0d8003520 con 0x7ff0ec15b150 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667797619 0 0) 0x7ff0ec06c470 con 0x7ff0ec0c53f0 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0ec06c1d0 con 0x7ff0ec0c53f0 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0f119d640 1 -- 172.21.15.2:0/1070566916 >> v1:172.21.15.123:6789/0 conn(0x7ff0ec0c53f0 legacy=0x7ff0ec0c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4214684659 0 0) 0x7ff0d8003520 con 0x7ff0ec15b150 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0f119d640 1 -- 172.21.15.2:0/1070566916 >> v1:172.21.15.2:6789/0 conn(0x7ff0ec15b150 legacy=0x7ff0ec0ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.094+0000 7ff0f119d640 1 -- 172.21.15.2:0/1070566916 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec06c730 con 0x7ff0ec154a00 2024-09-19T05:42:33.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.095+0000 7ff0f119d640 1 --2- 172.21.15.2:0/1070566916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec154a00 0x7ff0ec06d730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80097d0 tx=0x7ff0e802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3bae3a036840582 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.095+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8037070 con 0x7ff0ec154a00 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.095+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e802ac30 con 0x7ff0ec154a00 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.095+0000 7ff0e3fff640 1 -- 172.21.15.2:0/1070566916 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc001670 con 0x7ff0ec154a00 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.096+0000 7ff0f219f640 1 -- 172.21.15.2:0/1070566916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec154a00 msgr2=0x7ff0ec06d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.096+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1070566916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec154a00 0x7ff0ec06d730 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80097d0 tx=0x7ff0e802a1a0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.096+0000 7ff0f219f640 1 -- 172.21.15.2:0/1070566916 shutdown_connections 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.096+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1070566916 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec154a00 0x7ff0ec06d730 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.096+0000 7ff0f219f640 1 -- 172.21.15.2:0/1070566916 >> 172.21.15.2:0/1070566916 conn(0x7ff0ec067950 msgr2=0x7ff0ec067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.096+0000 7ff0f219f640 1 -- 172.21.15.2:0/1070566916 shutdown_connections 2024-09-19T05:42:33.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.096+0000 7ff0f219f640 1 -- 172.21.15.2:0/1070566916 wait complete. 2024-09-19T05:42:33.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.097+0000 7ff0f219f640 1 Processor -- start 2024-09-19T05:42:33.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.097+0000 7ff0f219f640 1 -- start start 2024-09-19T05:42:33.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f219f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec0c53f0 0x7ff0ec14f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f219f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0ec154a00 0x7ff0ec1e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f219f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0ec15b150 0x7ff0ec1e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f219f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec06cfd0 con 0x7ff0ec154a00 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f219f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec06ccf0 con 0x7ff0ec15b150 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f219f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ec0cae80 con 0x7ff0ec0c53f0 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0ec154a00 0x7ff0ec1e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0ec154a00 0x7ff0ec1e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37352/0 (socket says 172.21.15.2:37352) 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 -- 172.21.15.2:0/1492662573 learned_addr learned my addr 172.21.15.2:0/1492662573 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f119d640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec0c53f0 0x7ff0ec14f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f199e640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0ec15b150 0x7ff0ec1e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 -- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec0c53f0 msgr2=0x7ff0ec14f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec0c53f0 0x7ff0ec14f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 -- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0ec15b150 msgr2=0x7ff0ec1e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0ec15b150 0x7ff0ec1e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 -- 172.21.15.2:0/1492662573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec1f0570 con 0x7ff0ec154a00 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0f099c640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0ec154a00 0x7ff0ec1e60a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00c520 tx=0x7ff0dc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.098+0000 7ff0e1ffb640 1 -- 172.21.15.2:0/1492662573 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc017070 con 0x7ff0ec154a00 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.099+0000 7ff0e1ffb640 1 -- 172.21.15.2:0/1492662573 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0dc005ce0 con 0x7ff0ec154a00 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.099+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec1f0850 con 0x7ff0ec154a00 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.099+0000 7ff0e1ffb640 1 -- 172.21.15.2:0/1492662573 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc0072b0 con 0x7ff0ec154a00 2024-09-19T05:42:33.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.099+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec1f0e10 con 0x7ff0ec154a00 2024-09-19T05:42:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.100+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ec06dbb0 con 0x7ff0ec154a00 2024-09-19T05:42:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.100+0000 7ff0e1ffb640 1 -- 172.21.15.2:0/1492662573 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff0dc01a070 con 0x7ff0ec154a00 2024-09-19T05:42:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.101+0000 7ff0e1ffb640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff0d006c2b0 0x7ff0d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.101+0000 7ff0e1ffb640 1 -- 172.21.15.2:0/1492662573 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff0dc08a940 con 0x7ff0ec154a00 2024-09-19T05:42:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.101+0000 7ff0f119d640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff0d006c2b0 0x7ff0d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.101+0000 7ff0f119d640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff0d006c2b0 0x7ff0d006e770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8034000 tx=0x7ff0e802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.105+0000 7ff0e1ffb640 1 -- 172.21.15.2:0/1492662573 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff0ec06dbb0 con 0x7ff0ec154a00 2024-09-19T05:42:33.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:42:33.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:42:33.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.281+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0ec15b530 con 0x7ff0ec154a00 2024-09-19T05:42:33.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.282+0000 7ff0e1ffb640 1 -- 172.21.15.2:0/1492662573 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16634 (secure 0 0 0) 0x7ff0dc059020 con 0x7ff0ec154a00 2024-09-19T05:42:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:33.284 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":21,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:32.334355+0000","last_up_change":"2024-09-19T05:42:20.842106+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:33.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff0d006c2b0 msgr2=0x7ff0d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff0d006c2b0 0x7ff0d006e770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8034000 tx=0x7ff0e802e000 comp rx=0 tx=0).stop 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0ec154a00 msgr2=0x7ff0ec1e60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0ec154a00 0x7ff0ec1e60a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00c520 tx=0x7ff0dc00c9e0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 shutdown_connections 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff0d006c2b0 0x7ff0d006e770 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0ec15b150 0x7ff0ec1e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0ec154a00 0x7ff0ec1e60a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 --2- 172.21.15.2:0/1492662573 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0ec0c53f0 0x7ff0ec14f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 >> 172.21.15.2:0/1492662573 conn(0x7ff0ec067950 msgr2=0x7ff0ec0ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 shutdown_connections 2024-09-19T05:42:33.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.284+0000 7ff0f219f640 1 -- 172.21.15.2:0/1492662573 wait complete. 2024-09-19T05:42:33.297 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:33.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.438+0000 7fd485359640 1 Processor -- start 2024-09-19T05:42:33.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.438+0000 7fd485359640 1 -- start start 2024-09-19T05:42:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.438+0000 7fd485359640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd4800d0ac0 0x7fd4800cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.438+0000 7fd485359640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd48015eb50 0x7fd4800cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.438+0000 7fd485359640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4800cb400 con 0x7fd4800d06e0 2024-09-19T05:42:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.438+0000 7fd485359640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd4800cb6e0 con 0x7fd48015eb50 2024-09-19T05:42:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.438+0000 7fd485359640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd4800cc970 con 0x7fd4800d0ac0 2024-09-19T05:42:33.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47f7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd48015eb50 0x7fd4800cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47f7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd48015eb50 0x7fd4800cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37360/0 (socket says 172.21.15.2:37360) 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47f7fe640 1 -- 172.21.15.2:0/10101389 learned_addr learned my addr 172.21.15.2:0/10101389 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47effd640 1 --1- 172.21.15.2:0/10101389 >> v1:172.21.15.2:6789/0 conn(0x7fd4800d06e0 0x7fd480154af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51484/0 (socket says 172.21.15.2:51484) 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47e7fc640 1 --2- 172.21.15.2:0/10101389 >> v2:172.21.15.123:3300/0 conn(0x7fd4800d0ac0 0x7fd4800cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47f7fe640 1 -- 172.21.15.2:0/10101389 >> v2:172.21.15.123:3300/0 conn(0x7fd4800d0ac0 msgr2=0x7fd4800cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47f7fe640 1 --2- 172.21.15.2:0/10101389 >> v2:172.21.15.123:3300/0 conn(0x7fd4800d0ac0 0x7fd4800cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47f7fe640 1 -- 172.21.15.2:0/10101389 >> v1:172.21.15.2:6789/0 conn(0x7fd4800d06e0 legacy=0x7fd480154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47dffb640 1 -- 172.21.15.2:0/10101389 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734176293 0 0) 0x7fd4800cb400 con 0x7fd4800d06e0 2024-09-19T05:42:33.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.439+0000 7fd47f7fe640 1 -- 172.21.15.2:0/10101389 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4800ccc50 con 0x7fd48015eb50 2024-09-19T05:42:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd47f7fe640 1 --2- 172.21.15.2:0/10101389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48015eb50 0x7fd4800cd450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4800b68e0 tx=0x7fd47002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270492011e09629c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd47dffb640 1 -- 172.21.15.2:0/10101389 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470036070 con 0x7fd48015eb50 2024-09-19T05:42:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd47dffb640 1 -- 172.21.15.2:0/10101389 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd470031600 con 0x7fd48015eb50 2024-09-19T05:42:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd47dffb640 1 -- 172.21.15.2:0/10101389 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4700319d0 con 0x7fd48015eb50 2024-09-19T05:42:33.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 -- 172.21.15.2:0/10101389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48015eb50 msgr2=0x7fd4800cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 --2- 172.21.15.2:0/10101389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48015eb50 0x7fd4800cd450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd4800b68e0 tx=0x7fd47002a730 comp rx=0 tx=0).stop 2024-09-19T05:42:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 -- 172.21.15.2:0/10101389 shutdown_connections 2024-09-19T05:42:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 --2- 172.21.15.2:0/10101389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd48015eb50 0x7fd4800cd450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 --2- 172.21.15.2:0/10101389 >> v2:172.21.15.123:3300/0 conn(0x7fd4800d0ac0 0x7fd4800cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 -- 172.21.15.2:0/10101389 >> 172.21.15.2:0/10101389 conn(0x7fd48006d740 msgr2=0x7fd480073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 -- 172.21.15.2:0/10101389 shutdown_connections 2024-09-19T05:42:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.440+0000 7fd485359640 1 -- 172.21.15.2:0/10101389 wait complete. 2024-09-19T05:42:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 Processor -- start 2024-09-19T05:42:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 -- start start 2024-09-19T05:42:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4800d06e0 0x7fd4801f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4800d0ac0 0x7fd48014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48015eb50 0x7fd48014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800c5830 con 0x7fd4800d06e0 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd48015efc0 con 0x7fd48015eb50 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd485359640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd480155200 con 0x7fd4800d0ac0 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4800d06e0 0x7fd4801f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.442+0000 7fd47effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4800d06e0 0x7fd4801f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37374/0 (socket says 172.21.15.2:37374) 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47effd640 1 -- 172.21.15.2:0/1147523012 learned_addr learned my addr 172.21.15.2:0/1147523012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47f7fe640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48015eb50 0x7fd48014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47e7fc640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4800d0ac0 0x7fd48014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47effd640 1 -- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4800d0ac0 msgr2=0x7fd48014afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47effd640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4800d0ac0 0x7fd48014afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47effd640 1 -- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48015eb50 msgr2=0x7fd48014b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47effd640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48015eb50 0x7fd48014b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47effd640 1 -- 172.21.15.2:0/1147523012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801f1140 con 0x7fd4800d06e0 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd47effd640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4800d06e0 0x7fd4801f0c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c500 tx=0x7fd47800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd4777fe640 1 -- 172.21.15.2:0/1147523012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478018070 con 0x7fd4800d06e0 2024-09-19T05:42:33.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd4777fe640 1 -- 172.21.15.2:0/1147523012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd478007580 con 0x7fd4800d06e0 2024-09-19T05:42:33.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd4777fe640 1 -- 172.21.15.2:0/1147523012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478004a20 con 0x7fd4800d06e0 2024-09-19T05:42:33.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4801557e0 con 0x7fd4800d06e0 2024-09-19T05:42:33.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.443+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd480155da0 con 0x7fd4800d06e0 2024-09-19T05:42:33.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.444+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4800cbb40 con 0x7fd4800d06e0 2024-09-19T05:42:33.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.445+0000 7fd4777fe640 1 -- 172.21.15.2:0/1147523012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd478004bc0 con 0x7fd4800d06e0 2024-09-19T05:42:33.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.445+0000 7fd4777fe640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd46006c2b0 0x7fd46006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.445+0000 7fd4777fe640 1 -- 172.21.15.2:0/1147523012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd47808a690 con 0x7fd4800d06e0 2024-09-19T05:42:33.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.445+0000 7fd47e7fc640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd46006c2b0 0x7fd46006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.446+0000 7fd47e7fc640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd46006c2b0 0x7fd46006e770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4800cae40 tx=0x7fd468008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.449+0000 7fd4777fe640 1 -- 172.21.15.2:0/1147523012 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd478058e90 con 0x7fd4800d06e0 2024-09-19T05:42:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.627+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4800734e0 con 0x7fd4800d06e0 2024-09-19T05:42:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.628+0000 7fd4777fe640 1 -- 172.21.15.2:0/1147523012 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16634 (secure 0 0 0) 0x7fd478058790 con 0x7fd4800d06e0 2024-09-19T05:42:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:33.631 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":21,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:32.334355+0000","last_up_change":"2024-09-19T05:42:20.842106+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd46006c2b0 msgr2=0x7fd46006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd46006c2b0 0x7fd46006e770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4800cae40 tx=0x7fd468008040 comp rx=0 tx=0).stop 2024-09-19T05:42:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4800d06e0 msgr2=0x7fd4801f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4800d06e0 0x7fd4801f0c00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c500 tx=0x7fd47800c9c0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 shutdown_connections 2024-09-19T05:42:33.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd46006c2b0 0x7fd46006e770 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd48015eb50 0x7fd48014b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4800d0ac0 0x7fd48014afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 --2- 172.21.15.2:0/1147523012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd4800d06e0 0x7fd4801f0c00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 >> 172.21.15.2:0/1147523012 conn(0x7fd48006d740 msgr2=0x7fd480150f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:33.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 shutdown_connections 2024-09-19T05:42:33.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.630+0000 7fd485359640 1 -- 172.21.15.2:0/1147523012 wait complete. 2024-09-19T05:42:33.644 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T05:42:33.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.746+0000 7ff2c734b640 1 Processor -- start 2024-09-19T05:42:33.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.747+0000 7ff2c734b640 1 -- start start 2024-09-19T05:42:33.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.747+0000 7ff2c734b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff2c0152b80 0x7ff2c01591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.747+0000 7ff2c734b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c9000 0x7ff2c00ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.747+0000 7ff2c734b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00ca3a0 con 0x7ff2c0071110 2024-09-19T05:42:33.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c734b640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c00c94f0 con 0x7ff2c0152b80 2024-09-19T05:42:33.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c734b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c00c3750 con 0x7ff2c00c9000 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c50e7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff2c0152b80 0x7ff2c01591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c58e8640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff2c0071110 0x7ff2c015beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51498/0 (socket says 172.21.15.2:51498) 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c58e8640 1 -- 172.21.15.2:0/739208144 learned_addr learned my addr 172.21.15.2:0/739208144 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c48e6640 1 --2- 172.21.15.2:0/739208144 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c9000 0x7ff2c00ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c50e7640 1 -- 172.21.15.2:0/739208144 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c9000 msgr2=0x7ff2c00ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c50e7640 1 --2- 172.21.15.2:0/739208144 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c9000 0x7ff2c00ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c50e7640 1 -- 172.21.15.2:0/739208144 >> v1:172.21.15.2:6789/0 conn(0x7ff2c0071110 legacy=0x7ff2c015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2c50e7640 1 -- 172.21.15.2:0/739208144 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c00c3a30 con 0x7ff2c0152b80 2024-09-19T05:42:33.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.748+0000 7ff2b7fff640 1 -- 172.21.15.2:0/739208144 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1111511446 0 0) 0x7ff2c00ca3a0 con 0x7ff2c0071110 2024-09-19T05:42:33.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2c50e7640 1 --2- 172.21.15.2:0/739208144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0152b80 0x7ff2c01591d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00cf560 tx=0x7ff2bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd5a1ec719f414e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2b7fff640 1 -- 172.21.15.2:0/739208144 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc037070 con 0x7ff2c0152b80 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2b7fff640 1 -- 172.21.15.2:0/739208144 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2bc032440 con 0x7ff2c0152b80 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2b7fff640 1 -- 172.21.15.2:0/739208144 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc032810 con 0x7ff2c0152b80 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2c734b640 1 -- 172.21.15.2:0/739208144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0152b80 msgr2=0x7ff2c01591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2c734b640 1 --2- 172.21.15.2:0/739208144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0152b80 0x7ff2c01591d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00cf560 tx=0x7ff2bc02a1a0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2c734b640 1 -- 172.21.15.2:0/739208144 shutdown_connections 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2c734b640 1 --2- 172.21.15.2:0/739208144 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c9000 0x7ff2c00ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2c734b640 1 --2- 172.21.15.2:0/739208144 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0152b80 0x7ff2c01591d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.749+0000 7ff2c734b640 1 -- 172.21.15.2:0/739208144 >> 172.21.15.2:0/739208144 conn(0x7ff2c006c240 msgr2=0x7ff2c0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.750+0000 7ff2c734b640 1 -- 172.21.15.2:0/739208144 shutdown_connections 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.750+0000 7ff2c734b640 1 -- 172.21.15.2:0/739208144 wait complete. 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 Processor -- start 2024-09-19T05:42:33.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 -- start start 2024-09-19T05:42:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0071110 0x7ff2c01f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c9000 0x7ff2c01f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0152b80 0x7ff2c0147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c0158cb0 con 0x7ff2c0071110 2024-09-19T05:42:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c01589d0 con 0x7ff2c0152b80 2024-09-19T05:42:33.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c734b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c0152f60 con 0x7ff2c00c9000 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c50e7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0071110 0x7ff2c01f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.751+0000 7ff2c50e7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0071110 0x7ff2c01f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37394/0 (socket says 172.21.15.2:37394) 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c50e7640 1 -- 172.21.15.2:0/2133403314 learned_addr learned my addr 172.21.15.2:0/2133403314 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c48e6640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c9000 0x7ff2c01f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c58e8640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0152b80 0x7ff2c0147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c50e7640 1 -- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c9000 msgr2=0x7ff2c01f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c50e7640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c9000 0x7ff2c01f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c50e7640 1 -- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0152b80 msgr2=0x7ff2c0147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c50e7640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0152b80 0x7ff2c0147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c50e7640 1 -- 172.21.15.2:0/2133403314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c01f2760 con 0x7ff2c0071110 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c58e8640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0152b80 0x7ff2c0147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c50e7640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0071110 0x7ff2c01f1c00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc000c00 tx=0x7ff2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2b5ffb640 1 -- 172.21.15.2:0/2133403314 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc037040 con 0x7ff2c0071110 2024-09-19T05:42:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2b5ffb640 1 -- 172.21.15.2:0/2133403314 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2bc042070 con 0x7ff2c0071110 2024-09-19T05:42:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c01572a0 con 0x7ff2c0071110 2024-09-19T05:42:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2b5ffb640 1 -- 172.21.15.2:0/2133403314 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc030bd0 con 0x7ff2c0071110 2024-09-19T05:42:33.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.752+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c0146fc0 con 0x7ff2c0071110 2024-09-19T05:42:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.754+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c00cb740 con 0x7ff2c0071110 2024-09-19T05:42:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.754+0000 7ff2b5ffb640 1 -- 172.21.15.2:0/2133403314 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff2bc047460 con 0x7ff2c0071110 2024-09-19T05:42:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.754+0000 7ff2b5ffb640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a406c2b0 0x7ff2a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.755+0000 7ff2b5ffb640 1 -- 172.21.15.2:0/2133403314 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff2bc0ac080 con 0x7ff2c0071110 2024-09-19T05:42:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.755+0000 7ff2c48e6640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a406c2b0 0x7ff2a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:33.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.755+0000 7ff2c48e6640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a406c2b0 0x7ff2a406e770 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff2c014ed60 tx=0x7ff2b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:33.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.759+0000 7ff2b5ffb640 1 -- 172.21.15.2:0/2133403314 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff2bc032e20 con 0x7ff2c0071110 2024-09-19T05:42:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:33.977+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff2c0070ef0 con 0x7ff2c0071110 2024-09-19T05:42:34.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.971+0000 7ff2b5ffb640 1 -- 172.21.15.2:0/2133403314 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7ff2bc07a7f0 con 0x7ff2c0071110 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a406c2b0 msgr2=0x7ff2a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a406c2b0 0x7ff2a406e770 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff2c014ed60 tx=0x7ff2b0008040 comp rx=0 tx=0).stop 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0071110 msgr2=0x7ff2c01f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0071110 0x7ff2c01f1c00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc000c00 tx=0x7ff2bc0046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 shutdown_connections 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a406c2b0 0x7ff2a406e770 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c0152b80 0x7ff2c0147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c9000 0x7ff2c01f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 --2- 172.21.15.2:0/2133403314 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c0071110 0x7ff2c01f1c00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:34.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 >> 172.21.15.2:0/2133403314 conn(0x7ff2c006c240 msgr2=0x7ff2c014def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 shutdown_connections 2024-09-19T05:42:34.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:34.974+0000 7ff2c734b640 1 -- 172.21.15.2:0/2133403314 wait complete. 2024-09-19T05:42:34.989 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-09-19T05:42:34.990 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T05:42:35.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.099+0000 7f4b9353a640 1 Processor -- start 2024-09-19T05:42:35.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.099+0000 7f4b9353a640 1 -- start start 2024-09-19T05:42:35.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b9353a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4b8c15ede0 0x7f4b8c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:35.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b9353a640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b8c15ab40 con 0x7f4b8c0cba20 2024-09-19T05:42:35.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b9353a640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b8c15ade0 con 0x7f4b8c0cb640 2024-09-19T05:42:35.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b9353a640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c0c5ef0 con 0x7f4b8c15ede0 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b90ad5640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4b8c0cba20 0x7f4b8c1549e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51510/0 (socket says 172.21.15.2:51510) 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b90ad5640 1 -- 172.21.15.2:0/197812519 learned_addr learned my addr 172.21.15.2:0/197812519 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b91ad7640 1 --2- 172.21.15.2:0/197812519 >> v2:172.21.15.2:3300/0 conn(0x7f4b8c15ede0 0x7f4b8c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b83fff640 1 -- 172.21.15.2:0/197812519 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979381335 0 0) 0x7f4b8c15ab40 con 0x7f4b8c0cba20 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b83fff640 1 -- 172.21.15.2:0/197812519 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b78003520 con 0x7f4b8c0cba20 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b91ad7640 1 -- 172.21.15.2:0/197812519 >> v1:172.21.15.123:6789/0 conn(0x7f4b8c0cb640 legacy=0x7f4b8c0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b91ad7640 1 -- 172.21.15.2:0/197812519 >> v1:172.21.15.2:6789/0 conn(0x7f4b8c0cba20 legacy=0x7f4b8c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:35.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.100+0000 7f4b91ad7640 1 -- 172.21.15.2:0/197812519 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c15b0c0 con 0x7f4b8c15ede0 2024-09-19T05:42:35.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.101+0000 7f4b91ad7640 1 --2- 172.21.15.2:0/197812519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c15ede0 0x7f4b8c0c6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009d90 tx=0x7f4b8402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94539b35a899a1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:35.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.101+0000 7f4b83fff640 1 -- 172.21.15.2:0/197812519 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84037070 con 0x7f4b8c15ede0 2024-09-19T05:42:35.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.101+0000 7f4b83fff640 1 -- 172.21.15.2:0/197812519 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b84007740 con 0x7f4b8c15ede0 2024-09-19T05:42:35.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.101+0000 7f4b83fff640 1 -- 172.21.15.2:0/197812519 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84007b10 con 0x7f4b8c15ede0 2024-09-19T05:42:35.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.102+0000 7f4b9353a640 1 -- 172.21.15.2:0/197812519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c15ede0 msgr2=0x7f4b8c0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:35.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.102+0000 7f4b9353a640 1 --2- 172.21.15.2:0/197812519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c15ede0 0x7f4b8c0c6380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009d90 tx=0x7f4b8402a780 comp rx=0 tx=0).stop 2024-09-19T05:42:35.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.102+0000 7f4b9353a640 1 -- 172.21.15.2:0/197812519 shutdown_connections 2024-09-19T05:42:35.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.102+0000 7f4b9353a640 1 --2- 172.21.15.2:0/197812519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c15ede0 0x7f4b8c0c6380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.102+0000 7f4b9353a640 1 -- 172.21.15.2:0/197812519 >> 172.21.15.2:0/197812519 conn(0x7f4b8c067950 msgr2=0x7f4b8c067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:35.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.102+0000 7f4b9353a640 1 -- 172.21.15.2:0/197812519 shutdown_connections 2024-09-19T05:42:35.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.102+0000 7f4b9353a640 1 -- 172.21.15.2:0/197812519 wait complete. 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.103+0000 7f4b9353a640 1 Processor -- start 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.103+0000 7f4b9353a640 1 -- start start 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b9353a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b8c0cb640 0x7f4b8c0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b9353a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c0cba20 0x7f4b8c14eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b9353a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c15ede0 0x7f4b8c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b9353a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c15b960 con 0x7f4b8c0cba20 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b9353a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c15b680 con 0x7f4b8c15ede0 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b9353a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c0c5bd0 con 0x7f4b8c0cb640 2024-09-19T05:42:35.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c0cba20 0x7f4b8c14eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c0cba20 0x7f4b8c14eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37424/0 (socket says 172.21.15.2:37424) 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 -- 172.21.15.2:0/853841251 learned_addr learned my addr 172.21.15.2:0/853841251 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b912d6640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b8c0cb640 0x7f4b8c0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b91ad7640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c15ede0 0x7f4b8c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 -- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b8c0cb640 msgr2=0x7f4b8c0ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b8c0cb640 0x7f4b8c0ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 -- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c15ede0 msgr2=0x7f4b8c0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c15ede0 0x7f4b8c0b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 -- 172.21.15.2:0/853841251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c0c7c90 con 0x7f4b8c0cba20 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b90ad5640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c0cba20 0x7f4b8c14eed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c500 tx=0x7f4b7c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b81ffb640 1 -- 172.21.15.2:0/853841251 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c017070 con 0x7f4b8c0cba20 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b81ffb640 1 -- 172.21.15.2:0/853841251 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c005ce0 con 0x7f4b8c0cba20 2024-09-19T05:42:35.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c0c7f70 con 0x7f4b8c0cba20 2024-09-19T05:42:35.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.104+0000 7f4b81ffb640 1 -- 172.21.15.2:0/853841251 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c007310 con 0x7f4b8c0cba20 2024-09-19T05:42:35.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.105+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1f07d0 con 0x7f4b8c0cba20 2024-09-19T05:42:35.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.106+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c0c54c0 con 0x7f4b8c0cba20 2024-09-19T05:42:35.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.106+0000 7f4b81ffb640 1 -- 172.21.15.2:0/853841251 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4b7c01a070 con 0x7f4b8c0cba20 2024-09-19T05:42:35.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.107+0000 7f4b81ffb640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4b7006c2b0 0x7f4b7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:35.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.107+0000 7f4b81ffb640 1 -- 172.21.15.2:0/853841251 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4b7c08a770 con 0x7f4b8c0cba20 2024-09-19T05:42:35.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.107+0000 7f4b912d6640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4b7006c2b0 0x7f4b7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:35.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.107+0000 7f4b912d6640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4b7006c2b0 0x7f4b7006e770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000f60 tx=0x7f4b88009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:35.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.111+0000 7f4b81ffb640 1 -- 172.21.15.2:0/853841251 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4b7c0589e0 con 0x7f4b8c0cba20 2024-09-19T05:42:35.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.323+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4b8c0cbf10 con 0x7f4b8c0cba20 2024-09-19T05:42:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.976+0000 7f4b81ffb640 1 -- 172.21.15.2:0/853841251 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7f4b7c05cce0 con 0x7f4b8c0cba20 2024-09-19T05:42:35.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4b7006c2b0 msgr2=0x7f4b7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:35.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4b7006c2b0 0x7f4b7006e770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000f60 tx=0x7f4b88009040 comp rx=0 tx=0).stop 2024-09-19T05:42:35.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c0cba20 msgr2=0x7f4b8c14eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:35.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c0cba20 0x7f4b8c14eed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c500 tx=0x7f4b7c00c9c0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 shutdown_connections 2024-09-19T05:42:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4b7006c2b0 0x7f4b7006e770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b8c15ede0 0x7f4b8c0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b8c0cba20 0x7f4b8c14eed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 --2- 172.21.15.2:0/853841251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b8c0cb640 0x7f4b8c0ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 >> 172.21.15.2:0/853841251 conn(0x7f4b8c067950 msgr2=0x7f4b8c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 shutdown_connections 2024-09-19T05:42:35.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:35.979+0000 7f4b9353a640 1 -- 172.21.15.2:0/853841251 wait complete. 2024-09-19T05:42:35.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-19T05:42:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.104+0000 7feafd7d6640 1 Processor -- start 2024-09-19T05:42:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.104+0000 7feafd7d6640 1 -- start start 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.104+0000 7feafd7d6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feaf8154c20 0x7feaf80cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.104+0000 7feafd7d6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8155000 0x7feaf8073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.104+0000 7feafd7d6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaf80ccbc0 con 0x7feaf80caf40 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feafd7d6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7feaf80cc600 con 0x7feaf8154c20 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feafd7d6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feaf80cc8e0 con 0x7feaf8155000 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf6ffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feaf8154c20 0x7feaf80cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf6ffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feaf8154c20 0x7feaf80cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37428/0 (socket says 172.21.15.2:37428) 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf6ffd640 1 -- 172.21.15.2:0/626507 learned_addr learned my addr 172.21.15.2:0/626507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf67fc640 1 --2- 172.21.15.2:0/626507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8155000 0x7feaf8073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf6ffd640 1 -- 172.21.15.2:0/626507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8155000 msgr2=0x7feaf8073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf6ffd640 1 --2- 172.21.15.2:0/626507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8155000 0x7feaf8073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf6ffd640 1 -- 172.21.15.2:0/626507 >> v1:172.21.15.123:6789/0 conn(0x7feaf80caf40 legacy=0x7feaf815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.105+0000 7feaf6ffd640 1 -- 172.21.15.2:0/626507 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf80cbc60 con 0x7feaf8154c20 2024-09-19T05:42:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feaf6ffd640 1 --2- 172.21.15.2:0/626507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8154c20 0x7feaf80cd9e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feae8009870 tx=0x7feae802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfaf3b7b5420c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feaf5ffb640 1 -- 172.21.15.2:0/626507 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8037070 con 0x7feaf8154c20 2024-09-19T05:42:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feaf5ffb640 1 -- 172.21.15.2:0/626507 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae8032440 con 0x7feaf8154c20 2024-09-19T05:42:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feaf5ffb640 1 -- 172.21.15.2:0/626507 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8032810 con 0x7feaf8154c20 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 -- 172.21.15.2:0/626507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8154c20 msgr2=0x7feaf80cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 --2- 172.21.15.2:0/626507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8154c20 0x7feaf80cd9e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feae8009870 tx=0x7feae802a210 comp rx=0 tx=0).stop 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 -- 172.21.15.2:0/626507 shutdown_connections 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 --2- 172.21.15.2:0/626507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8155000 0x7feaf8073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 --2- 172.21.15.2:0/626507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8154c20 0x7feaf80cd9e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 -- 172.21.15.2:0/626507 >> 172.21.15.2:0/626507 conn(0x7feaf806d0e0 msgr2=0x7feaf806d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 -- 172.21.15.2:0/626507 shutdown_connections 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.106+0000 7feafd7d6640 1 -- 172.21.15.2:0/626507 wait complete. 2024-09-19T05:42:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 Processor -- start 2024-09-19T05:42:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 -- start start 2024-09-19T05:42:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf80caf40 0x7feaf81e4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8154c20 0x7feaf81e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8155000 0x7feaf81e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7feaf815b1f0 con 0x7feaf8155000 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feaf80cb430 con 0x7feaf80caf40 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.108+0000 7feafd7d6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feaf815b370 con 0x7feaf8154c20 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8155000 0x7feaf81e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8155000 0x7feaf81e4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37436/0 (socket says 172.21.15.2:37436) 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 -- 172.21.15.2:0/2592025571 learned_addr learned my addr 172.21.15.2:0/2592025571 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf67fc640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8154c20 0x7feaf81e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf6ffd640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf80caf40 0x7feaf81e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 -- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8154c20 msgr2=0x7feaf81e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8154c20 0x7feaf81e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 -- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf80caf40 msgr2=0x7feaf81e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf80caf40 0x7feaf81e4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 -- 172.21.15.2:0/2592025571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf80cf260 con 0x7feaf8155000 2024-09-19T05:42:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feaf77fe640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8155000 0x7feaf81e4c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c780 tx=0x7feaf000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feadf7fe640 1 -- 172.21.15.2:0/2592025571 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0004b60 con 0x7feaf8155000 2024-09-19T05:42:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feadf7fe640 1 -- 172.21.15.2:0/2592025571 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf0004d00 con 0x7feaf8155000 2024-09-19T05:42:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf80cf500 con 0x7feaf8155000 2024-09-19T05:42:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feadf7fe640 1 -- 172.21.15.2:0/2592025571 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf000aa90 con 0x7feaf8155000 2024-09-19T05:42:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.109+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf80cfac0 con 0x7feaf8155000 2024-09-19T05:42:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.110+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf80d1000 con 0x7feaf8155000 2024-09-19T05:42:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.111+0000 7feadf7fe640 1 -- 172.21.15.2:0/2592025571 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feaf000ac30 con 0x7feaf8155000 2024-09-19T05:42:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.111+0000 7feadf7fe640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feae406c2b0 0x7feae406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.111+0000 7feadf7fe640 1 -- 172.21.15.2:0/2592025571 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feaf008b5d0 con 0x7feaf8155000 2024-09-19T05:42:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.111+0000 7feaf6ffd640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feae406c2b0 0x7feae406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.112+0000 7feaf6ffd640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feae406c2b0 0x7feae406e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feae8034000 tx=0x7feae802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.116+0000 7feadf7fe640 1 -- 172.21.15.2:0/2592025571 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7feaf80d1000 con 0x7feaf8155000 2024-09-19T05:42:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.328+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7feaf80c5770 con 0x7feaf8155000 2024-09-19T05:42:36.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.981+0000 7feadf7fe640 1 -- 172.21.15.2:0/2592025571 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v7) ==== 115+0+0 (secure 0 0 0) 0x7feaf0020070 con 0x7feaf8155000 2024-09-19T05:42:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feae406c2b0 msgr2=0x7feae406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feae406c2b0 0x7feae406e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feae8034000 tx=0x7feae802e000 comp rx=0 tx=0).stop 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8155000 msgr2=0x7feaf81e4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8155000 0x7feaf81e4c90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c780 tx=0x7feaf000cc40 comp rx=0 tx=0).stop 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 shutdown_connections 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feae406c2b0 0x7feae406e770 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feaf8155000 0x7feaf81e4c90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feaf8154c20 0x7feaf81e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 --2- 172.21.15.2:0/2592025571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaf80caf40 0x7feaf81e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 >> 172.21.15.2:0/2592025571 conn(0x7feaf806d0e0 msgr2=0x7feaf81e35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 shutdown_connections 2024-09-19T05:42:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:36.984+0000 7feafd7d6640 1 -- 172.21.15.2:0/2592025571 wait complete. 2024-09-19T05:42:37.000 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T05:42:37.000 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:42:37.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.106+0000 7f46ec25e640 1 Processor -- start 2024-09-19T05:42:37.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.106+0000 7f46ec25e640 1 -- start start 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46ec25e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d06e0 0x7f46e40cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46ec25e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f46e40d0ac0 0x7f46e415eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46ec25e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e40cb400 con 0x7f46e4154af0 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46ec25e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f46e415f090 con 0x7f46e40d0ac0 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46ec25e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f46e40cb6e0 con 0x7f46e40d06e0 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46e97f9640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f46e40d0ac0 0x7f46e415eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46e97f9640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f46e40d0ac0 0x7f46e415eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37446/0 (socket says 172.21.15.2:37446) 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.107+0000 7f46e97f9640 1 -- 172.21.15.2:0/1265289105 learned_addr learned my addr 172.21.15.2:0/1265289105 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:37.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.108+0000 7f46e9ffa640 1 --2- 172.21.15.2:0/1265289105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d06e0 0x7f46e40cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.108+0000 7f46e97f9640 1 -- 172.21.15.2:0/1265289105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d06e0 msgr2=0x7f46e40cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:37.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.108+0000 7f46e97f9640 1 --2- 172.21.15.2:0/1265289105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d06e0 0x7f46e40cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.108+0000 7f46e97f9640 1 -- 172.21.15.2:0/1265289105 >> v1:172.21.15.123:6789/0 conn(0x7f46e4154af0 legacy=0x7f46e40cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.108+0000 7f46e97f9640 1 -- 172.21.15.2:0/1265289105 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e40cc970 con 0x7f46e40d0ac0 2024-09-19T05:42:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.108+0000 7f46e97f9640 1 --2- 172.21.15.2:0/1265289105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d0ac0 0x7f46e415eb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009870 tx=0x7f46d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e260ee370d8aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46e8ff8640 1 -- 172.21.15.2:0/1265289105 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8037070 con 0x7f46e40d0ac0 2024-09-19T05:42:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46e8ff8640 1 -- 172.21.15.2:0/1265289105 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8032440 con 0x7f46e40d0ac0 2024-09-19T05:42:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46e8ff8640 1 -- 172.21.15.2:0/1265289105 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8032810 con 0x7f46e40d0ac0 2024-09-19T05:42:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 -- 172.21.15.2:0/1265289105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d0ac0 msgr2=0x7f46e415eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 --2- 172.21.15.2:0/1265289105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d0ac0 0x7f46e415eb50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009870 tx=0x7f46d802a270 comp rx=0 tx=0).stop 2024-09-19T05:42:37.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 -- 172.21.15.2:0/1265289105 shutdown_connections 2024-09-19T05:42:37.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 --2- 172.21.15.2:0/1265289105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d0ac0 0x7f46e415eb50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 --2- 172.21.15.2:0/1265289105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d06e0 0x7f46e40cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 -- 172.21.15.2:0/1265289105 >> 172.21.15.2:0/1265289105 conn(0x7f46e406d740 msgr2=0x7f46e4073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:37.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 -- 172.21.15.2:0/1265289105 shutdown_connections 2024-09-19T05:42:37.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.109+0000 7f46ec25e640 1 -- 172.21.15.2:0/1265289105 wait complete. 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 Processor -- start 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 -- start start 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d06e0 0x7f46e414d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d0ac0 0x7f46e41f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4154af0 0x7f46e414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46e40c5830 con 0x7f46e40d06e0 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46e40ccf30 con 0x7f46e4154af0 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46ec25e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f46e40cdb60 con 0x7f46e40d0ac0 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46e9ffa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d06e0 0x7f46e414d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46e9ffa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d06e0 0x7f46e414d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37448/0 (socket says 172.21.15.2:37448) 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.111+0000 7f46e9ffa640 1 -- 172.21.15.2:0/3310843582 learned_addr learned my addr 172.21.15.2:0/3310843582 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e97f9640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d0ac0 0x7f46e41f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46ea7fb640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4154af0 0x7f46e414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e9ffa640 1 -- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d0ac0 msgr2=0x7f46e41f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e9ffa640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d0ac0 0x7f46e41f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e9ffa640 1 -- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4154af0 msgr2=0x7f46e414dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e9ffa640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4154af0 0x7f46e414dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e9ffa640 1 -- 172.21.15.2:0/3310843582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e414c580 con 0x7f46e40d06e0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e9ffa640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d06e0 0x7f46e414d220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009db0 tx=0x7f46d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e27fc640 1 -- 172.21.15.2:0/3310843582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4018020 con 0x7f46e40d06e0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e27fc640 1 -- 172.21.15.2:0/3310843582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d4014480 con 0x7f46e40d06e0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46e27fc640 1 -- 172.21.15.2:0/3310843582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4014850 con 0x7f46e40d06e0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e414bce0 con 0x7f46e40d06e0 2024-09-19T05:42:37.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.112+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e414c2a0 con 0x7f46e40d06e0 2024-09-19T05:42:37.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.113+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e40cbb40 con 0x7f46e40d06e0 2024-09-19T05:42:37.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.114+0000 7f46e27fc640 1 -- 172.21.15.2:0/3310843582 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f46d4005d10 con 0x7f46e40d06e0 2024-09-19T05:42:37.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.114+0000 7f46e27fc640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f46d006c2e0 0x7f46d006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.114+0000 7f46e27fc640 1 -- 172.21.15.2:0/3310843582 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f46d408c120 con 0x7f46e40d06e0 2024-09-19T05:42:37.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.114+0000 7f46e97f9640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f46d006c2e0 0x7f46d006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.115+0000 7f46e97f9640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f46d006c2e0 0x7f46d006e7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46d8034000 tx=0x7f46d802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.118+0000 7f46e27fc640 1 -- 172.21.15.2:0/3310843582 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f46d4022030 con 0x7f46e40d06e0 2024-09-19T05:42:37.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.331+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46e40b6380 con 0x7f46e40d06e0 2024-09-19T05:42:37.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.332+0000 7f46e27fc640 1 -- 172.21.15.2:0/3310843582 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+6858 (secure 0 0 0) 0x7f46d4010040 con 0x7f46e40d06e0 2024-09-19T05:42:37.335 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:37.335 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":7,"btime":"2024-09-19T05:42:36:978066+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/121690198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":121690198},{"type":"v1","addr":"172.21.15.2:6835","nonce":121690198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1118046326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1118046326},{"type":"v1","addr":"172.21.15.123:6837","nonce":1118046326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3264935286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3264935286},{"type":"v1","addr":"172.21.15.2:6837","nonce":3264935286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/4246839966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4246839966},{"type":"v1","addr":"172.21.15.123:6836","nonce":4246839966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:42:32.334667+0000","modified":"2024-09-19T05:42:36.330385+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"f","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.123:6839/4093916922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4093916922},{"type":"v1","addr":"172.21.15.123:6839","nonce":4093916922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-19T05:42:37.335 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 7 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f46d006c2e0 msgr2=0x7f46d006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f46d006c2e0 0x7f46d006e7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f46d8034000 tx=0x7f46d802e000 comp rx=0 tx=0).stop 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d06e0 msgr2=0x7f46e414d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d06e0 0x7f46e414d220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009db0 tx=0x7f46d400c490 comp rx=0 tx=0).stop 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 shutdown_connections 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f46d006c2e0 0x7f46d006e7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46e4154af0 0x7f46e414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46e40d0ac0 0x7f46e41f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 --2- 172.21.15.2:0/3310843582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46e40d06e0 0x7f46e414d220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 >> 172.21.15.2:0/3310843582 conn(0x7f46e406d740 msgr2=0x7f46e40c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:37.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 shutdown_connections 2024-09-19T05:42:37.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.334+0000 7f46ec25e640 1 -- 172.21.15.2:0/3310843582 wait complete. 2024-09-19T05:42:37.347 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:37.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.447+0000 7f475ebd6640 1 Processor -- start 2024-09-19T05:42:37.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.447+0000 7f475ebd6640 1 -- start start 2024-09-19T05:42:37.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475ebd6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47580d06e0 0x7f47580cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475ebd6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47580cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475ebd6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47580cb400 con 0x7f47580d0ac0 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475ebd6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f47580cb6e0 con 0x7f47580d06e0 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475ebd6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f47580cc970 con 0x7f475815eb50 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475c972640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47580d06e0 0x7f47580cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475d173640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47580cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475c972640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47580d06e0 0x7f47580cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43842/0 (socket says 172.21.15.2:43842) 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f4757fff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f47580d0ac0 0x7f4758154af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:42896/0 (socket says 172.21.15.2:42896) 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475c972640 1 -- 172.21.15.2:0/3258736188 learned_addr learned my addr 172.21.15.2:0/3258736188 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475d173640 1 -- 172.21.15.2:0/3258736188 >> v2:172.21.15.123:3300/0 conn(0x7f47580d06e0 msgr2=0x7f47580cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475d173640 1 --2- 172.21.15.2:0/3258736188 >> v2:172.21.15.123:3300/0 conn(0x7f47580d06e0 0x7f47580cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475d173640 1 -- 172.21.15.2:0/3258736188 >> v1:172.21.15.123:6789/0 conn(0x7f47580d0ac0 legacy=0x7f4758154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475d173640 1 -- 172.21.15.2:0/3258736188 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47580ccc50 con 0x7f475815eb50 2024-09-19T05:42:37.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f47577fe640 1 -- 172.21.15.2:0/3258736188 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023087151 0 0) 0x7f47580cb400 con 0x7f47580d0ac0 2024-09-19T05:42:37.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.448+0000 7f475c972640 1 --2- 172.21.15.2:0/3258736188 >> v2:172.21.15.123:3300/0 conn(0x7f47580d06e0 0x7f47580cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:37.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.449+0000 7f475d173640 1 --2- 172.21.15.2:0/3258736188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47580cd450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4750009d90 tx=0x7f475002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4bcb218cdb296c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.449+0000 7f47577fe640 1 -- 172.21.15.2:0/3258736188 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750007740 con 0x7f475815eb50 2024-09-19T05:42:37.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.449+0000 7f47577fe640 1 -- 172.21.15.2:0/3258736188 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47500078e0 con 0x7f475815eb50 2024-09-19T05:42:37.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.449+0000 7f47577fe640 1 -- 172.21.15.2:0/3258736188 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750007cb0 con 0x7f475815eb50 2024-09-19T05:42:37.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 -- 172.21.15.2:0/3258736188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 msgr2=0x7f47580cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 --2- 172.21.15.2:0/3258736188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47580cd450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4750009d90 tx=0x7f475002a660 comp rx=0 tx=0).stop 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 -- 172.21.15.2:0/3258736188 shutdown_connections 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 --2- 172.21.15.2:0/3258736188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47580cd450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 --2- 172.21.15.2:0/3258736188 >> v2:172.21.15.123:3300/0 conn(0x7f47580d06e0 0x7f47580cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 -- 172.21.15.2:0/3258736188 >> 172.21.15.2:0/3258736188 conn(0x7f475806d740 msgr2=0x7f4758073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 -- 172.21.15.2:0/3258736188 shutdown_connections 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.450+0000 7f475ebd6640 1 -- 172.21.15.2:0/3258736188 wait complete. 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.451+0000 7f475ebd6640 1 Processor -- start 2024-09-19T05:42:37.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.451+0000 7f475ebd6640 1 -- start start 2024-09-19T05:42:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475ebd6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47580d06e0 0x7f47581571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475ebd6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47580d0ac0 0x7f4758157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475ebd6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47581f0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475ebd6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f47580c5830 con 0x7f47580d06e0 2024-09-19T05:42:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475ebd6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f475815f000 con 0x7f47580d0ac0 2024-09-19T05:42:37.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475ebd6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4758155200 con 0x7f475815eb50 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47580d06e0 0x7f47581571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47580d06e0 0x7f47581571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37452/0 (socket says 172.21.15.2:37452) 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 -- 172.21.15.2:0/2180852131 learned_addr learned my addr 172.21.15.2:0/2180852131 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f4757fff640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47580d0ac0 0x7f4758157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475d173640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47581f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 -- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 msgr2=0x7f47581f0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47581f0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 -- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47580d0ac0 msgr2=0x7f4758157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47580d0ac0 0x7f4758157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.452+0000 7f475c972640 1 -- 172.21.15.2:0/2180852131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4758152010 con 0x7f47580d06e0 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.453+0000 7f475c972640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47580d06e0 0x7f47581571c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9d0 tx=0x7f474c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.453+0000 7f47557fa640 1 -- 172.21.15.2:0/2180852131 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c017070 con 0x7f47580d06e0 2024-09-19T05:42:37.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.453+0000 7f47557fa640 1 -- 172.21.15.2:0/2180852131 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f474c00c9a0 con 0x7f47580d06e0 2024-09-19T05:42:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.453+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47581f45e0 con 0x7f47580d06e0 2024-09-19T05:42:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.453+0000 7f47557fa640 1 -- 172.21.15.2:0/2180852131 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c00cd50 con 0x7f47580d06e0 2024-09-19T05:42:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.453+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4758147670 con 0x7f47580d06e0 2024-09-19T05:42:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.454+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47580cbb40 con 0x7f47580d06e0 2024-09-19T05:42:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.454+0000 7f47557fa640 1 -- 172.21.15.2:0/2180852131 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f474c01a070 con 0x7f47580d06e0 2024-09-19T05:42:37.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.455+0000 7f47557fa640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f473c06c2b0 0x7f473c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.455+0000 7f47557fa640 1 -- 172.21.15.2:0/2180852131 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f474c08b570 con 0x7f47580d06e0 2024-09-19T05:42:37.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.455+0000 7f4757fff640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f473c06c2b0 0x7f473c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.455+0000 7f4757fff640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f473c06c2b0 0x7f473c06e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f47580caa90 tx=0x7f4744007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.459+0000 7f47557fa640 1 -- 172.21.15.2:0/2180852131 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f474c059720 con 0x7f47580d06e0 2024-09-19T05:42:37.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.634+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4758157700 con 0x7f47580d06e0 2024-09-19T05:42:37.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.635+0000 7f47557fa640 1 -- 172.21.15.2:0/2180852131 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16621 (secure 0 0 0) 0x7f474c05da20 con 0x7f47580d06e0 2024-09-19T05:42:37.637 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:37.638 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:33.917577+0000","last_up_change":"2024-09-19T05:42:20.842106+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f473c06c2b0 msgr2=0x7f473c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f473c06c2b0 0x7f473c06e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f47580caa90 tx=0x7f4744007d50 comp rx=0 tx=0).stop 2024-09-19T05:42:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47580d06e0 msgr2=0x7f47581571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47580d06e0 0x7f47581571c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f474c00e9d0 tx=0x7f474c00ee90 comp rx=0 tx=0).stop 2024-09-19T05:42:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 shutdown_connections 2024-09-19T05:42:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f473c06c2b0 0x7f473c06e770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47580d0ac0 0x7f4758157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f475815eb50 0x7f47581f0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 --2- 172.21.15.2:0/2180852131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47580d06e0 0x7f47581571c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 >> 172.21.15.2:0/2180852131 conn(0x7f475806d740 msgr2=0x7f4758152580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 shutdown_connections 2024-09-19T05:42:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.637+0000 7f475ebd6640 1 -- 172.21.15.2:0/2180852131 wait complete. 2024-09-19T05:42:37.652 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T05:42:37.653 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:42:37.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.752+0000 7f1561386640 1 Processor -- start 2024-09-19T05:42:37.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.752+0000 7f1561386640 1 -- start start 2024-09-19T05:42:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f1561386640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f155c072e40 0x7f155c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f1561386640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c073220 0x7f155c06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f1561386640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155c158960 con 0x7f155c15caa0 2024-09-19T05:42:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f1561386640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f155c0c7b40 con 0x7f155c072e40 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f1561386640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f155c0c7e20 con 0x7f155c073220 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f155affd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f155c072e40 0x7f155c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f155affd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f155c072e40 0x7f155c0722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37456/0 (socket says 172.21.15.2:37456) 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.753+0000 7f155affd640 1 -- 172.21.15.2:0/1584327136 learned_addr learned my addr 172.21.15.2:0/1584327136 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.754+0000 7f155a7fc640 1 --2- 172.21.15.2:0/1584327136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c073220 0x7f155c06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.754+0000 7f155affd640 1 -- 172.21.15.2:0/1584327136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c073220 msgr2=0x7f155c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.754+0000 7f155affd640 1 --2- 172.21.15.2:0/1584327136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c073220 0x7f155c06bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.754+0000 7f155affd640 1 -- 172.21.15.2:0/1584327136 >> v1:172.21.15.123:6789/0 conn(0x7f155c15caa0 legacy=0x7f155c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.754+0000 7f155affd640 1 -- 172.21.15.2:0/1584327136 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c158c20 con 0x7f155c072e40 2024-09-19T05:42:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.754+0000 7f155affd640 1 --2- 172.21.15.2:0/1584327136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c072e40 0x7f155c0722c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1554009870 tx=0x7f155402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c724a53fd12a9f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1559ffb640 1 -- 172.21.15.2:0/1584327136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554037070 con 0x7f155c072e40 2024-09-19T05:42:37.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1559ffb640 1 -- 172.21.15.2:0/1584327136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f155402ae50 con 0x7f155c072e40 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1559ffb640 1 -- 172.21.15.2:0/1584327136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554032860 con 0x7f155c072e40 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 -- 172.21.15.2:0/1584327136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c072e40 msgr2=0x7f155c0722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 --2- 172.21.15.2:0/1584327136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c072e40 0x7f155c0722c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1554009870 tx=0x7f155402a300 comp rx=0 tx=0).stop 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 -- 172.21.15.2:0/1584327136 shutdown_connections 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 --2- 172.21.15.2:0/1584327136 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c073220 0x7f155c06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 --2- 172.21.15.2:0/1584327136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c072e40 0x7f155c0722c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 -- 172.21.15.2:0/1584327136 >> 172.21.15.2:0/1584327136 conn(0x7f155c06e450 msgr2=0x7f155c06e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 -- 172.21.15.2:0/1584327136 shutdown_connections 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.755+0000 7f1561386640 1 -- 172.21.15.2:0/1584327136 wait complete. 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.756+0000 7f1561386640 1 Processor -- start 2024-09-19T05:42:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f1561386640 1 -- start start 2024-09-19T05:42:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f1561386640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c072e40 0x7f155c0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f1561386640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c073220 0x7f155c0c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f1561386640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c15caa0 0x7f155c1f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f1561386640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f155c072800 con 0x7f155c15caa0 2024-09-19T05:42:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f1561386640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f155c1591e0 con 0x7f155c073220 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f1561386640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f155c06c150 con 0x7f155c072e40 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f155b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c15caa0 0x7f155c1f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f155b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c15caa0 0x7f155c1f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37460/0 (socket says 172.21.15.2:37460) 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f155b7fe640 1 -- 172.21.15.2:0/1129538375 learned_addr learned my addr 172.21.15.2:0/1129538375 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f155affd640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c072e40 0x7f155c0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.757+0000 7f155a7fc640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c073220 0x7f155c0c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f155b7fe640 1 -- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c072e40 msgr2=0x7f155c0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f155b7fe640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c072e40 0x7f155c0cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f155b7fe640 1 -- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c073220 msgr2=0x7f155c0c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f155b7fe640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c073220 0x7f155c0c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f155b7fe640 1 -- 172.21.15.2:0/1129538375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c1f0ef0 con 0x7f155c15caa0 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f155b7fe640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c15caa0 0x7f155c1f39e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f154c00c6b0 tx=0x7f154c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f15537fe640 1 -- 172.21.15.2:0/1129538375 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c003b70 con 0x7f155c15caa0 2024-09-19T05:42:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f15537fe640 1 -- 172.21.15.2:0/1129538375 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f154c003d30 con 0x7f155c15caa0 2024-09-19T05:42:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c1f11b0 con 0x7f155c15caa0 2024-09-19T05:42:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f15537fe640 1 -- 172.21.15.2:0/1129538375 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c005310 con 0x7f155c15caa0 2024-09-19T05:42:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.758+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c15a7a0 con 0x7f155c15caa0 2024-09-19T05:42:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.759+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155c155120 con 0x7f155c15caa0 2024-09-19T05:42:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.760+0000 7f15537fe640 1 -- 172.21.15.2:0/1129538375 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f154c0054b0 con 0x7f155c15caa0 2024-09-19T05:42:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.760+0000 7f15537fe640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f153c06c2b0 0x7f153c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.760+0000 7f15537fe640 1 -- 172.21.15.2:0/1129538375 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f154c08b0e0 con 0x7f155c15caa0 2024-09-19T05:42:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.760+0000 7f155affd640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f153c06c2b0 0x7f153c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.761+0000 7f155affd640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f153c06c2b0 0x7f153c06e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1554034000 tx=0x7f155402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:37.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.765+0000 7f15537fe640 1 -- 172.21.15.2:0/1129538375 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f155c155120 con 0x7f155c15caa0 2024-09-19T05:42:37.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.937+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f155c0c77e0 con 0x7f153c06c2b0 2024-09-19T05:42:37.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.944+0000 7f15537fe640 1 -- 172.21.15.2:0/1129538375 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f155c0c77e0 con 0x7f153c06c2b0 2024-09-19T05:42:37.950 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:37.951 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:42:37.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f153c06c2b0 msgr2=0x7f153c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f153c06c2b0 0x7f153c06e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1554034000 tx=0x7f155402e000 comp rx=0 tx=0).stop 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c15caa0 msgr2=0x7f155c1f39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c15caa0 0x7f155c1f39e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f154c00c6b0 tx=0x7f154c00cb70 comp rx=0 tx=0).stop 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 shutdown_connections 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f153c06c2b0 0x7f153c06e770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f155c073220 0x7f155c0c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f155c15caa0 0x7f155c1f39e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 --2- 172.21.15.2:0/1129538375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f155c072e40 0x7f155c0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 >> 172.21.15.2:0/1129538375 conn(0x7f155c06e450 msgr2=0x7f155c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 shutdown_connections 2024-09-19T05:42:37.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:37.948+0000 7f1561386640 1 -- 172.21.15.2:0/1129538375 wait complete. 2024-09-19T05:42:37.961 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-19T05:42:37.820549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234796,"kb_used_data":1876,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1921024,"data_stored":1807364,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001541"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058235+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:42:34.058235+0000","last_peered":"2024-09-19T05:42:34.058235+0000","last_clean":"2024-09-19T05:42:34.058235+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:42:34.058235+0000","last_undegraded":"2024-09-19T05:42:34.058235+0000","last_fullsized":"2024-09-19T05:42:34.058235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058494+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:42:34.058494+0000","last_peered":"2024-09-19T05:42:34.058494+0000","last_clean":"2024-09-19T05:42:34.058494+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:42:34.058494+0000","last_undegraded":"2024-09-19T05:42:34.058494+0000","last_fullsized":"2024-09-19T05:42:34.058494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058017+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:42:34.058017+0000","last_peered":"2024-09-19T05:42:34.058017+0000","last_clean":"2024-09-19T05:42:34.058017+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:42:34.058017+0000","last_undegraded":"2024-09-19T05:42:34.058017+0000","last_fullsized":"2024-09-19T05:42:34.058017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923708+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:42:33.923708+0000","last_peered":"2024-09-19T05:42:33.923708+0000","last_clean":"2024-09-19T05:42:33.923708+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:42:33.923708+0000","last_undegraded":"2024-09-19T05:42:33.923708+0000","last_fullsized":"2024-09-19T05:42:33.923708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057846+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:42:34.057846+0000","last_peered":"2024-09-19T05:42:34.057846+0000","last_clean":"2024-09-19T05:42:34.057846+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:42:34.057846+0000","last_undegraded":"2024-09-19T05:42:34.057846+0000","last_fullsized":"2024-09-19T05:42:34.057846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923679+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:42:33.923679+0000","last_peered":"2024-09-19T05:42:33.923679+0000","last_clean":"2024-09-19T05:42:33.923679+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:42:33.923679+0000","last_undegraded":"2024-09-19T05:42:33.923679+0000","last_fullsized":"2024-09-19T05:42:33.923679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057764+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:42:34.057764+0000","last_peered":"2024-09-19T05:42:34.057764+0000","last_clean":"2024-09-19T05:42:34.057764+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:42:34.057764+0000","last_undegraded":"2024-09-19T05:42:34.057764+0000","last_fullsized":"2024-09-19T05:42:34.057764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058259+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:42:34.058259+0000","last_peered":"2024-09-19T05:42:34.058259+0000","last_clean":"2024-09-19T05:42:34.058259+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:42:34.058259+0000","last_undegraded":"2024-09-19T05:42:34.058259+0000","last_fullsized":"2024-09-19T05:42:34.058259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058756+0000","last_change":"2024-09-19T05:42:32.945872+0000","last_active":"2024-09-19T05:42:34.058756+0000","last_peered":"2024-09-19T05:42:34.058756+0000","last_clean":"2024-09-19T05:42:34.058756+0000","last_became_active":"2024-09-19T05:42:32.945545+0000","last_became_peered":"2024-09-19T05:42:32.945545+0000","last_unstale":"2024-09-19T05:42:34.058756+0000","last_undegraded":"2024-09-19T05:42:34.058756+0000","last_fullsized":"2024-09-19T05:42:34.058756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058073+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:42:34.058073+0000","last_peered":"2024-09-19T05:42:34.058073+0000","last_clean":"2024-09-19T05:42:34.058073+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:42:34.058073+0000","last_undegraded":"2024-09-19T05:42:34.058073+0000","last_fullsized":"2024-09-19T05:42:34.058073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058096+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:42:34.058096+0000","last_peered":"2024-09-19T05:42:34.058096+0000","last_clean":"2024-09-19T05:42:34.058096+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:42:34.058096+0000","last_undegraded":"2024-09-19T05:42:34.058096+0000","last_fullsized":"2024-09-19T05:42:34.058096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058167+0000","last_change":"2024-09-19T05:42:32.941601+0000","last_active":"2024-09-19T05:42:34.058167+0000","last_peered":"2024-09-19T05:42:34.058167+0000","last_clean":"2024-09-19T05:42:34.058167+0000","last_became_active":"2024-09-19T05:42:32.941288+0000","last_became_peered":"2024-09-19T05:42:32.941288+0000","last_unstale":"2024-09-19T05:42:34.058167+0000","last_undegraded":"2024-09-19T05:42:34.058167+0000","last_fullsized":"2024-09-19T05:42:34.058167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:35.670719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058583+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:42:34.058583+0000","last_peered":"2024-09-19T05:42:34.058583+0000","last_clean":"2024-09-19T05:42:34.058583+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:42:34.058583+0000","last_undegraded":"2024-09-19T05:42:34.058583+0000","last_fullsized":"2024-09-19T05:42:34.058583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058482+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:42:34.058482+0000","last_peered":"2024-09-19T05:42:34.058482+0000","last_clean":"2024-09-19T05:42:34.058482+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:42:34.058482+0000","last_undegraded":"2024-09-19T05:42:34.058482+0000","last_fullsized":"2024-09-19T05:42:34.058482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058261+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:42:34.058261+0000","last_peered":"2024-09-19T05:42:34.058261+0000","last_clean":"2024-09-19T05:42:34.058261+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:42:34.058261+0000","last_undegraded":"2024-09-19T05:42:34.058261+0000","last_fullsized":"2024-09-19T05:42:34.058261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057516+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:42:34.057516+0000","last_peered":"2024-09-19T05:42:34.057516+0000","last_clean":"2024-09-19T05:42:34.057516+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:42:34.057516+0000","last_undegraded":"2024-09-19T05:42:34.057516+0000","last_fullsized":"2024-09-19T05:42:34.057516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059660+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:42:34.059660+0000","last_peered":"2024-09-19T05:42:34.059660+0000","last_clean":"2024-09-19T05:42:34.059660+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:42:34.059660+0000","last_undegraded":"2024-09-19T05:42:34.059660+0000","last_fullsized":"2024-09-19T05:42:34.059660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059534+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:42:34.059534+0000","last_peered":"2024-09-19T05:42:34.059534+0000","last_clean":"2024-09-19T05:42:34.059534+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:42:34.059534+0000","last_undegraded":"2024-09-19T05:42:34.059534+0000","last_fullsized":"2024-09-19T05:42:34.059534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058089+0000","last_change":"2024-09-19T05:42:32.940627+0000","last_active":"2024-09-19T05:42:34.058089+0000","last_peered":"2024-09-19T05:42:34.058089+0000","last_clean":"2024-09-19T05:42:34.058089+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058089+0000","last_undegraded":"2024-09-19T05:42:34.058089+0000","last_fullsized":"2024-09-19T05:42:34.058089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.616840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925755+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:42:33.925755+0000","last_peered":"2024-09-19T05:42:33.925755+0000","last_clean":"2024-09-19T05:42:33.925755+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:42:33.925755+0000","last_undegraded":"2024-09-19T05:42:33.925755+0000","last_fullsized":"2024-09-19T05:42:33.925755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058013+0000","last_change":"2024-09-19T05:42:32.940425+0000","last_active":"2024-09-19T05:42:34.058013+0000","last_peered":"2024-09-19T05:42:34.058013+0000","last_clean":"2024-09-19T05:42:34.058013+0000","last_became_active":"2024-09-19T05:42:32.940059+0000","last_became_peered":"2024-09-19T05:42:32.940059+0000","last_unstale":"2024-09-19T05:42:34.058013+0000","last_undegraded":"2024-09-19T05:42:34.058013+0000","last_fullsized":"2024-09-19T05:42:34.058013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058206+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:42:34.058206+0000","last_peered":"2024-09-19T05:42:34.058206+0000","last_clean":"2024-09-19T05:42:34.058206+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:42:34.058206+0000","last_undegraded":"2024-09-19T05:42:34.058206+0000","last_fullsized":"2024-09-19T05:42:34.058206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057512+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:42:34.057512+0000","last_peered":"2024-09-19T05:42:34.057512+0000","last_clean":"2024-09-19T05:42:34.057512+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:42:34.057512+0000","last_undegraded":"2024-09-19T05:42:34.057512+0000","last_fullsized":"2024-09-19T05:42:34.057512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058955+0000","last_change":"2024-09-19T05:42:32.941414+0000","last_active":"2024-09-19T05:42:34.058955+0000","last_peered":"2024-09-19T05:42:34.058955+0000","last_clean":"2024-09-19T05:42:34.058955+0000","last_became_active":"2024-09-19T05:42:32.941076+0000","last_became_peered":"2024-09-19T05:42:32.941076+0000","last_unstale":"2024-09-19T05:42:34.058955+0000","last_undegraded":"2024-09-19T05:42:34.058955+0000","last_fullsized":"2024-09-19T05:42:34.058955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:13.371046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923839+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:42:33.923839+0000","last_peered":"2024-09-19T05:42:33.923839+0000","last_clean":"2024-09-19T05:42:33.923839+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:42:33.923839+0000","last_undegraded":"2024-09-19T05:42:33.923839+0000","last_fullsized":"2024-09-19T05:42:33.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058867+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:42:34.058867+0000","last_peered":"2024-09-19T05:42:34.058867+0000","last_clean":"2024-09-19T05:42:34.058867+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:42:34.058867+0000","last_undegraded":"2024-09-19T05:42:34.058867+0000","last_fullsized":"2024-09-19T05:42:34.058867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058377+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:42:34.058377+0000","last_peered":"2024-09-19T05:42:34.058377+0000","last_clean":"2024-09-19T05:42:34.058377+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:42:34.058377+0000","last_undegraded":"2024-09-19T05:42:34.058377+0000","last_fullsized":"2024-09-19T05:42:34.058377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059114+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:42:34.059114+0000","last_peered":"2024-09-19T05:42:34.059114+0000","last_clean":"2024-09-19T05:42:34.059114+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:42:34.059114+0000","last_undegraded":"2024-09-19T05:42:34.059114+0000","last_fullsized":"2024-09-19T05:42:34.059114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924874+0000","last_change":"2024-09-19T05:42:32.927184+0000","last_active":"2024-09-19T05:42:33.924874+0000","last_peered":"2024-09-19T05:42:33.924874+0000","last_clean":"2024-09-19T05:42:33.924874+0000","last_became_active":"2024-09-19T05:42:32.926952+0000","last_became_peered":"2024-09-19T05:42:32.926952+0000","last_unstale":"2024-09-19T05:42:33.924874+0000","last_undegraded":"2024-09-19T05:42:33.924874+0000","last_fullsized":"2024-09-19T05:42:33.924874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923475+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:42:33.923475+0000","last_peered":"2024-09-19T05:42:33.923475+0000","last_clean":"2024-09-19T05:42:33.923475+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:42:33.923475+0000","last_undegraded":"2024-09-19T05:42:33.923475+0000","last_fullsized":"2024-09-19T05:42:33.923475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058161+0000","last_change":"2024-09-19T05:42:32.942584+0000","last_active":"2024-09-19T05:42:34.058161+0000","last_peered":"2024-09-19T05:42:34.058161+0000","last_clean":"2024-09-19T05:42:34.058161+0000","last_became_active":"2024-09-19T05:42:32.942274+0000","last_became_peered":"2024-09-19T05:42:32.942274+0000","last_unstale":"2024-09-19T05:42:34.058161+0000","last_undegraded":"2024-09-19T05:42:34.058161+0000","last_fullsized":"2024-09-19T05:42:34.058161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058203+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:42:34.058203+0000","last_peered":"2024-09-19T05:42:34.058203+0000","last_clean":"2024-09-19T05:42:34.058203+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:42:34.058203+0000","last_undegraded":"2024-09-19T05:42:34.058203+0000","last_fullsized":"2024-09-19T05:42:34.058203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925748+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:42:33.925748+0000","last_peered":"2024-09-19T05:42:33.925748+0000","last_clean":"2024-09-19T05:42:33.925748+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:42:33.925748+0000","last_undegraded":"2024-09-19T05:42:33.925748+0000","last_fullsized":"2024-09-19T05:42:33.925748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059210+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:42:34.059210+0000","last_peered":"2024-09-19T05:42:34.059210+0000","last_clean":"2024-09-19T05:42:34.059210+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:42:34.059210+0000","last_undegraded":"2024-09-19T05:42:34.059210+0000","last_fullsized":"2024-09-19T05:42:34.059210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058211+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:42:34.058211+0000","last_peered":"2024-09-19T05:42:34.058211+0000","last_clean":"2024-09-19T05:42:34.058211+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:42:34.058211+0000","last_undegraded":"2024-09-19T05:42:34.058211+0000","last_fullsized":"2024-09-19T05:42:34.058211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058257+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:42:34.058257+0000","last_peered":"2024-09-19T05:42:34.058257+0000","last_clean":"2024-09-19T05:42:34.058257+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058257+0000","last_undegraded":"2024-09-19T05:42:34.058257+0000","last_fullsized":"2024-09-19T05:42:34.058257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057643+0000","last_change":"2024-09-19T05:42:31.995072+0000","last_active":"2024-09-19T05:42:34.057643+0000","last_peered":"2024-09-19T05:42:34.057643+0000","last_clean":"2024-09-19T05:42:34.057643+0000","last_became_active":"2024-09-19T05:42:31.994586+0000","last_became_peered":"2024-09-19T05:42:31.994586+0000","last_unstale":"2024-09-19T05:42:34.057643+0000","last_undegraded":"2024-09-19T05:42:34.057643+0000","last_fullsized":"2024-09-19T05:42:34.057643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925971+0000","last_change":"2024-09-19T05:42:32.928695+0000","last_active":"2024-09-19T05:42:33.925971+0000","last_peered":"2024-09-19T05:42:33.925971+0000","last_clean":"2024-09-19T05:42:33.925971+0000","last_became_active":"2024-09-19T05:42:32.928437+0000","last_became_peered":"2024-09-19T05:42:32.928437+0000","last_unstale":"2024-09-19T05:42:33.925971+0000","last_undegraded":"2024-09-19T05:42:33.925971+0000","last_fullsized":"2024-09-19T05:42:33.925971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058846+0000","last_change":"2024-09-19T05:42:31.985393+0000","last_active":"2024-09-19T05:42:34.058846+0000","last_peered":"2024-09-19T05:42:34.058846+0000","last_clean":"2024-09-19T05:42:34.058846+0000","last_became_active":"2024-09-19T05:42:31.985034+0000","last_became_peered":"2024-09-19T05:42:31.985034+0000","last_unstale":"2024-09-19T05:42:34.058846+0000","last_undegraded":"2024-09-19T05:42:34.058846+0000","last_fullsized":"2024-09-19T05:42:34.058846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:11.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923861+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:42:33.923861+0000","last_peered":"2024-09-19T05:42:33.923861+0000","last_clean":"2024-09-19T05:42:33.923861+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:42:33.923861+0000","last_undegraded":"2024-09-19T05:42:33.923861+0000","last_fullsized":"2024-09-19T05:42:33.923861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058193+0000","last_change":"2024-09-19T05:42:31.987634+0000","last_active":"2024-09-19T05:42:34.058193+0000","last_peered":"2024-09-19T05:42:34.058193+0000","last_clean":"2024-09-19T05:42:34.058193+0000","last_became_active":"2024-09-19T05:42:31.986452+0000","last_became_peered":"2024-09-19T05:42:31.986452+0000","last_unstale":"2024-09-19T05:42:34.058193+0000","last_undegraded":"2024-09-19T05:42:34.058193+0000","last_fullsized":"2024-09-19T05:42:34.058193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925095+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:42:33.925095+0000","last_peered":"2024-09-19T05:42:33.925095+0000","last_clean":"2024-09-19T05:42:33.925095+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:42:33.925095+0000","last_undegraded":"2024-09-19T05:42:33.925095+0000","last_fullsized":"2024-09-19T05:42:33.925095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059270+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:42:34.059270+0000","last_peered":"2024-09-19T05:42:34.059270+0000","last_clean":"2024-09-19T05:42:34.059270+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:42:34.059270+0000","last_undegraded":"2024-09-19T05:42:34.059270+0000","last_fullsized":"2024-09-19T05:42:34.059270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923690+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:42:33.923690+0000","last_peered":"2024-09-19T05:42:33.923690+0000","last_clean":"2024-09-19T05:42:33.923690+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:42:33.923690+0000","last_undegraded":"2024-09-19T05:42:33.923690+0000","last_fullsized":"2024-09-19T05:42:33.923690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058283+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:42:34.058283+0000","last_peered":"2024-09-19T05:42:34.058283+0000","last_clean":"2024-09-19T05:42:34.058283+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:42:34.058283+0000","last_undegraded":"2024-09-19T05:42:34.058283+0000","last_fullsized":"2024-09-19T05:42:34.058283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925200+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:42:33.925200+0000","last_peered":"2024-09-19T05:42:33.925200+0000","last_clean":"2024-09-19T05:42:33.925200+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:42:33.925200+0000","last_undegraded":"2024-09-19T05:42:33.925200+0000","last_fullsized":"2024-09-19T05:42:33.925200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058589+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:42:34.058589+0000","last_peered":"2024-09-19T05:42:34.058589+0000","last_clean":"2024-09-19T05:42:34.058589+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:42:34.058589+0000","last_undegraded":"2024-09-19T05:42:34.058589+0000","last_fullsized":"2024-09-19T05:42:34.058589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059325+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:42:34.059325+0000","last_peered":"2024-09-19T05:42:34.059325+0000","last_clean":"2024-09-19T05:42:34.059325+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:42:34.059325+0000","last_undegraded":"2024-09-19T05:42:34.059325+0000","last_fullsized":"2024-09-19T05:42:34.059325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059177+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:42:34.059177+0000","last_peered":"2024-09-19T05:42:34.059177+0000","last_clean":"2024-09-19T05:42:34.059177+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:42:34.059177+0000","last_undegraded":"2024-09-19T05:42:34.059177+0000","last_fullsized":"2024-09-19T05:42:34.059177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057760+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:42:34.057760+0000","last_peered":"2024-09-19T05:42:34.057760+0000","last_clean":"2024-09-19T05:42:34.057760+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:42:34.057760+0000","last_undegraded":"2024-09-19T05:42:34.057760+0000","last_fullsized":"2024-09-19T05:42:34.057760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058006+0000","last_change":"2024-09-19T05:42:32.938549+0000","last_active":"2024-09-19T05:42:34.058006+0000","last_peered":"2024-09-19T05:42:34.058006+0000","last_clean":"2024-09-19T05:42:34.058006+0000","last_became_active":"2024-09-19T05:42:32.938113+0000","last_became_peered":"2024-09-19T05:42:32.938113+0000","last_unstale":"2024-09-19T05:42:34.058006+0000","last_undegraded":"2024-09-19T05:42:34.058006+0000","last_fullsized":"2024-09-19T05:42:34.058006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:15.787698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925541+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:42:33.925541+0000","last_peered":"2024-09-19T05:42:33.925541+0000","last_clean":"2024-09-19T05:42:33.925541+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:42:33.925541+0000","last_undegraded":"2024-09-19T05:42:33.925541+0000","last_fullsized":"2024-09-19T05:42:33.925541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058325+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:42:34.058325+0000","last_peered":"2024-09-19T05:42:34.058325+0000","last_clean":"2024-09-19T05:42:34.058325+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:42:34.058325+0000","last_undegraded":"2024-09-19T05:42:34.058325+0000","last_fullsized":"2024-09-19T05:42:34.058325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923969+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:42:33.923969+0000","last_peered":"2024-09-19T05:42:33.923969+0000","last_clean":"2024-09-19T05:42:33.923969+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:42:33.923969+0000","last_undegraded":"2024-09-19T05:42:33.923969+0000","last_fullsized":"2024-09-19T05:42:33.923969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925826+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:42:33.925826+0000","last_peered":"2024-09-19T05:42:33.925826+0000","last_clean":"2024-09-19T05:42:33.925826+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:42:33.925826+0000","last_undegraded":"2024-09-19T05:42:33.925826+0000","last_fullsized":"2024-09-19T05:42:33.925826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058299+0000","last_change":"2024-09-19T05:42:32.945685+0000","last_active":"2024-09-19T05:42:34.058299+0000","last_peered":"2024-09-19T05:42:34.058299+0000","last_clean":"2024-09-19T05:42:34.058299+0000","last_became_active":"2024-09-19T05:42:32.945337+0000","last_became_peered":"2024-09-19T05:42:32.945337+0000","last_unstale":"2024-09-19T05:42:34.058299+0000","last_undegraded":"2024-09-19T05:42:34.058299+0000","last_fullsized":"2024-09-19T05:42:34.058299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058153+0000","last_change":"2024-09-19T05:42:31.991430+0000","last_active":"2024-09-19T05:42:34.058153+0000","last_peered":"2024-09-19T05:42:34.058153+0000","last_clean":"2024-09-19T05:42:34.058153+0000","last_became_active":"2024-09-19T05:42:31.991057+0000","last_became_peered":"2024-09-19T05:42:31.991057+0000","last_unstale":"2024-09-19T05:42:34.058153+0000","last_undegraded":"2024-09-19T05:42:34.058153+0000","last_fullsized":"2024-09-19T05:42:34.058153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:58.247495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058366+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:42:34.058366+0000","last_peered":"2024-09-19T05:42:34.058366+0000","last_clean":"2024-09-19T05:42:34.058366+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:42:34.058366+0000","last_undegraded":"2024-09-19T05:42:34.058366+0000","last_fullsized":"2024-09-19T05:42:34.058366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058851+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:42:34.058851+0000","last_peered":"2024-09-19T05:42:34.058851+0000","last_clean":"2024-09-19T05:42:34.058851+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:42:34.058851+0000","last_undegraded":"2024-09-19T05:42:34.058851+0000","last_fullsized":"2024-09-19T05:42:34.058851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059399+0000","last_change":"2024-09-19T05:42:32.941889+0000","last_active":"2024-09-19T05:42:34.059399+0000","last_peered":"2024-09-19T05:42:34.059399+0000","last_clean":"2024-09-19T05:42:34.059399+0000","last_became_active":"2024-09-19T05:42:32.941254+0000","last_became_peered":"2024-09-19T05:42:32.941254+0000","last_unstale":"2024-09-19T05:42:34.059399+0000","last_undegraded":"2024-09-19T05:42:34.059399+0000","last_fullsized":"2024-09-19T05:42:34.059399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.771973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057837+0000","last_change":"2024-09-19T05:42:23.458936+0000","last_active":"2024-09-19T05:42:34.057837+0000","last_peered":"2024-09-19T05:42:34.057837+0000","last_clean":"2024-09-19T05:42:34.057837+0000","last_became_active":"2024-09-19T05:42:23.457990+0000","last_became_peered":"2024-09-19T05:42:23.457990+0000","last_unstale":"2024-09-19T05:42:34.057837+0000","last_undegraded":"2024-09-19T05:42:34.057837+0000","last_fullsized":"2024-09-19T05:42:34.057837+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:45.259241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059347+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:42:34.059347+0000","last_peered":"2024-09-19T05:42:34.059347+0000","last_clean":"2024-09-19T05:42:34.059347+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:42:34.059347+0000","last_undegraded":"2024-09-19T05:42:34.059347+0000","last_fullsized":"2024-09-19T05:42:34.059347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925215+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:42:33.925215+0000","last_peered":"2024-09-19T05:42:33.925215+0000","last_clean":"2024-09-19T05:42:33.925215+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:42:33.925215+0000","last_undegraded":"2024-09-19T05:42:33.925215+0000","last_fullsized":"2024-09-19T05:42:33.925215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057735+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:42:34.057735+0000","last_peered":"2024-09-19T05:42:34.057735+0000","last_clean":"2024-09-19T05:42:34.057735+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:42:34.057735+0000","last_undegraded":"2024-09-19T05:42:34.057735+0000","last_fullsized":"2024-09-19T05:42:34.057735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058126+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:42:34.058126+0000","last_peered":"2024-09-19T05:42:34.058126+0000","last_clean":"2024-09-19T05:42:34.058126+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:42:34.058126+0000","last_undegraded":"2024-09-19T05:42:34.058126+0000","last_fullsized":"2024-09-19T05:42:34.058126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924861+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:42:33.924861+0000","last_peered":"2024-09-19T05:42:33.924861+0000","last_clean":"2024-09-19T05:42:33.924861+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:42:33.924861+0000","last_undegraded":"2024-09-19T05:42:33.924861+0000","last_fullsized":"2024-09-19T05:42:33.924861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057949+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:42:34.057949+0000","last_peered":"2024-09-19T05:42:34.057949+0000","last_clean":"2024-09-19T05:42:34.057949+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:42:34.057949+0000","last_undegraded":"2024-09-19T05:42:34.057949+0000","last_fullsized":"2024-09-19T05:42:34.057949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059345+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:42:34.059345+0000","last_peered":"2024-09-19T05:42:34.059345+0000","last_clean":"2024-09-19T05:42:34.059345+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:42:34.059345+0000","last_undegraded":"2024-09-19T05:42:34.059345+0000","last_fullsized":"2024-09-19T05:42:34.059345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057870+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:42:34.057870+0000","last_peered":"2024-09-19T05:42:34.057870+0000","last_clean":"2024-09-19T05:42:34.057870+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:42:34.057870+0000","last_undegraded":"2024-09-19T05:42:34.057870+0000","last_fullsized":"2024-09-19T05:42:34.057870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058164+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:42:34.058164+0000","last_peered":"2024-09-19T05:42:34.058164+0000","last_clean":"2024-09-19T05:42:34.058164+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:42:34.058164+0000","last_undegraded":"2024-09-19T05:42:34.058164+0000","last_fullsized":"2024-09-19T05:42:34.058164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923659+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:42:33.923659+0000","last_peered":"2024-09-19T05:42:33.923659+0000","last_clean":"2024-09-19T05:42:33.923659+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:42:33.923659+0000","last_undegraded":"2024-09-19T05:42:33.923659+0000","last_fullsized":"2024-09-19T05:42:33.923659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059484+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:42:34.059484+0000","last_peered":"2024-09-19T05:42:34.059484+0000","last_clean":"2024-09-19T05:42:34.059484+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:42:34.059484+0000","last_undegraded":"2024-09-19T05:42:34.059484+0000","last_fullsized":"2024-09-19T05:42:34.059484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058726+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:42:34.058726+0000","last_peered":"2024-09-19T05:42:34.058726+0000","last_clean":"2024-09-19T05:42:34.058726+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:42:34.058726+0000","last_undegraded":"2024-09-19T05:42:34.058726+0000","last_fullsized":"2024-09-19T05:42:34.058726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925942+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:42:33.925942+0000","last_peered":"2024-09-19T05:42:33.925942+0000","last_clean":"2024-09-19T05:42:33.925942+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:42:33.925942+0000","last_undegraded":"2024-09-19T05:42:33.925942+0000","last_fullsized":"2024-09-19T05:42:33.925942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925132+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:42:33.925132+0000","last_peered":"2024-09-19T05:42:33.925132+0000","last_clean":"2024-09-19T05:42:33.925132+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:42:33.925132+0000","last_undegraded":"2024-09-19T05:42:33.925132+0000","last_fullsized":"2024-09-19T05:42:33.925132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059606+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:42:34.059606+0000","last_peered":"2024-09-19T05:42:34.059606+0000","last_clean":"2024-09-19T05:42:34.059606+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:42:34.059606+0000","last_undegraded":"2024-09-19T05:42:34.059606+0000","last_fullsized":"2024-09-19T05:42:34.059606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925887+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:42:33.925887+0000","last_peered":"2024-09-19T05:42:33.925887+0000","last_clean":"2024-09-19T05:42:33.925887+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:42:33.925887+0000","last_undegraded":"2024-09-19T05:42:33.925887+0000","last_fullsized":"2024-09-19T05:42:33.925887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057349+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:42:34.057349+0000","last_peered":"2024-09-19T05:42:34.057349+0000","last_clean":"2024-09-19T05:42:34.057349+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:42:34.057349+0000","last_undegraded":"2024-09-19T05:42:34.057349+0000","last_fullsized":"2024-09-19T05:42:34.057349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059062+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:42:34.059062+0000","last_peered":"2024-09-19T05:42:34.059062+0000","last_clean":"2024-09-19T05:42:34.059062+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:42:34.059062+0000","last_undegraded":"2024-09-19T05:42:34.059062+0000","last_fullsized":"2024-09-19T05:42:34.059062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925905+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:42:33.925905+0000","last_peered":"2024-09-19T05:42:33.925905+0000","last_clean":"2024-09-19T05:42:33.925905+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:42:33.925905+0000","last_undegraded":"2024-09-19T05:42:33.925905+0000","last_fullsized":"2024-09-19T05:42:33.925905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925556+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:42:33.925556+0000","last_peered":"2024-09-19T05:42:33.925556+0000","last_clean":"2024-09-19T05:42:33.925556+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:42:33.925556+0000","last_undegraded":"2024-09-19T05:42:33.925556+0000","last_fullsized":"2024-09-19T05:42:33.925556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058603+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:42:34.058603+0000","last_peered":"2024-09-19T05:42:34.058603+0000","last_clean":"2024-09-19T05:42:34.058603+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:42:34.058603+0000","last_undegraded":"2024-09-19T05:42:34.058603+0000","last_fullsized":"2024-09-19T05:42:34.058603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925165+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:42:33.925165+0000","last_peered":"2024-09-19T05:42:33.925165+0000","last_clean":"2024-09-19T05:42:33.925165+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:42:33.925165+0000","last_undegraded":"2024-09-19T05:42:33.925165+0000","last_fullsized":"2024-09-19T05:42:33.925165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059115+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:42:34.059115+0000","last_peered":"2024-09-19T05:42:34.059115+0000","last_clean":"2024-09-19T05:42:34.059115+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:42:34.059115+0000","last_undegraded":"2024-09-19T05:42:34.059115+0000","last_fullsized":"2024-09-19T05:42:34.059115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059198+0000","last_change":"2024-09-19T05:42:32.941131+0000","last_active":"2024-09-19T05:42:34.059198+0000","last_peered":"2024-09-19T05:42:34.059198+0000","last_clean":"2024-09-19T05:42:34.059198+0000","last_became_active":"2024-09-19T05:42:32.940596+0000","last_became_peered":"2024-09-19T05:42:32.940596+0000","last_unstale":"2024-09-19T05:42:34.059198+0000","last_undegraded":"2024-09-19T05:42:34.059198+0000","last_fullsized":"2024-09-19T05:42:34.059198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:00.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059376+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:42:34.059376+0000","last_peered":"2024-09-19T05:42:34.059376+0000","last_clean":"2024-09-19T05:42:34.059376+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:42:34.059376+0000","last_undegraded":"2024-09-19T05:42:34.059376+0000","last_fullsized":"2024-09-19T05:42:34.059376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058312+0000","last_change":"2024-09-19T05:42:31.988393+0000","last_active":"2024-09-19T05:42:34.058312+0000","last_peered":"2024-09-19T05:42:34.058312+0000","last_clean":"2024-09-19T05:42:34.058312+0000","last_became_active":"2024-09-19T05:42:31.986557+0000","last_became_peered":"2024-09-19T05:42:31.986557+0000","last_unstale":"2024-09-19T05:42:34.058312+0000","last_undegraded":"2024-09-19T05:42:34.058312+0000","last_fullsized":"2024-09-19T05:42:34.058312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058853+0000","last_change":"2024-09-19T05:42:32.929140+0000","last_active":"2024-09-19T05:42:34.058853+0000","last_peered":"2024-09-19T05:42:34.058853+0000","last_clean":"2024-09-19T05:42:34.058853+0000","last_became_active":"2024-09-19T05:42:32.928811+0000","last_became_peered":"2024-09-19T05:42:32.928811+0000","last_unstale":"2024-09-19T05:42:34.058853+0000","last_undegraded":"2024-09-19T05:42:34.058853+0000","last_fullsized":"2024-09-19T05:42:34.058853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:29.306206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058372+0000","last_change":"2024-09-19T05:42:31.988083+0000","last_active":"2024-09-19T05:42:34.058372+0000","last_peered":"2024-09-19T05:42:34.058372+0000","last_clean":"2024-09-19T05:42:34.058372+0000","last_became_active":"2024-09-19T05:42:31.987088+0000","last_became_peered":"2024-09-19T05:42:31.987088+0000","last_unstale":"2024-09-19T05:42:34.058372+0000","last_undegraded":"2024-09-19T05:42:34.058372+0000","last_fullsized":"2024-09-19T05:42:34.058372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.941310+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.940997+0000","last_became_peered":"2024-09-19T05:42:32.940997+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:58.755996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058706+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:42:34.058706+0000","last_peered":"2024-09-19T05:42:34.058706+0000","last_clean":"2024-09-19T05:42:34.058706+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:42:34.058706+0000","last_undegraded":"2024-09-19T05:42:34.058706+0000","last_fullsized":"2024-09-19T05:42:34.058706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058604+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:42:34.058604+0000","last_peered":"2024-09-19T05:42:34.058604+0000","last_clean":"2024-09-19T05:42:34.058604+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:42:34.058604+0000","last_undegraded":"2024-09-19T05:42:34.058604+0000","last_fullsized":"2024-09-19T05:42:34.058604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":290816,"data_stored":669484,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":663184,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:42:37.965 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:42:38.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.101+0000 7f53f6a37640 1 Processor -- start 2024-09-19T05:42:38.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.101+0000 7f53f6a37640 1 -- start start 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f6a37640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f53f0154cb0 0x7f53f015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f6a37640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53f00cd7c0 0x7f53f00cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f6a37640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f00734c0 con 0x7f53f00cc5f0 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f6a37640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f53f00cd330 con 0x7f53f00cd7c0 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f6a37640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f53f0073780 con 0x7f53f0154cb0 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f4fd4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53f00cd7c0 0x7f53f00cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53ef7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f53f00cc5f0 0x7f53f00c55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51514/0 (socket says 172.21.15.2:51514) 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53ef7fe640 1 -- 172.21.15.2:0/146700384 learned_addr learned my addr 172.21.15.2:0/146700384 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53effff640 1 --2- 172.21.15.2:0/146700384 >> v2:172.21.15.123:3300/0 conn(0x7f53f0154cb0 0x7f53f015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f4fd4640 1 -- 172.21.15.2:0/146700384 >> v2:172.21.15.123:3300/0 conn(0x7f53f0154cb0 msgr2=0x7f53f015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:38.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f4fd4640 1 --2- 172.21.15.2:0/146700384 >> v2:172.21.15.123:3300/0 conn(0x7f53f0154cb0 0x7f53f015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f4fd4640 1 -- 172.21.15.2:0/146700384 >> v1:172.21.15.2:6789/0 conn(0x7f53f00cc5f0 legacy=0x7f53f00c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.102+0000 7f53f4fd4640 1 -- 172.21.15.2:0/146700384 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0073a60 con 0x7f53f00cd7c0 2024-09-19T05:42:38.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.103+0000 7f53effff640 1 --2- 172.21.15.2:0/146700384 >> v2:172.21.15.123:3300/0 conn(0x7f53f0154cb0 0x7f53f015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:38.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.103+0000 7f53f4fd4640 1 --2- 172.21.15.2:0/146700384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cd7c0 0x7f53f00cadc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53f00d0980 tx=0x7f53e802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7136c0b0be70028a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.103+0000 7f53eeffd640 1 -- 172.21.15.2:0/146700384 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8007740 con 0x7f53f00cd7c0 2024-09-19T05:42:38.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.103+0000 7f53eeffd640 1 -- 172.21.15.2:0/146700384 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e80078e0 con 0x7f53f00cd7c0 2024-09-19T05:42:38.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.103+0000 7f53eeffd640 1 -- 172.21.15.2:0/146700384 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e8007cb0 con 0x7f53f00cd7c0 2024-09-19T05:42:38.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 -- 172.21.15.2:0/146700384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cd7c0 msgr2=0x7f53f00cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 --2- 172.21.15.2:0/146700384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cd7c0 0x7f53f00cadc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f53f00d0980 tx=0x7f53e802a710 comp rx=0 tx=0).stop 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 -- 172.21.15.2:0/146700384 shutdown_connections 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 --2- 172.21.15.2:0/146700384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cd7c0 0x7f53f00cadc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 --2- 172.21.15.2:0/146700384 >> v2:172.21.15.123:3300/0 conn(0x7f53f0154cb0 0x7f53f015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 -- 172.21.15.2:0/146700384 >> 172.21.15.2:0/146700384 conn(0x7f53f006d0c0 msgr2=0x7f53f006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 -- 172.21.15.2:0/146700384 shutdown_connections 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.104+0000 7f53f6a37640 1 -- 172.21.15.2:0/146700384 wait complete. 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.105+0000 7f53f6a37640 1 Processor -- start 2024-09-19T05:42:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.105+0000 7f53f6a37640 1 -- start start 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f6a37640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cc5f0 0x7f53f01f1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f6a37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f00cd7c0 0x7f53f01e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f6a37640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f0154cb0 0x7f53f01f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f6a37640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00cdbf0 con 0x7f53f00cc5f0 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f6a37640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00b30d0 con 0x7f53f00cd7c0 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f6a37640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f53f00cb300 con 0x7f53f0154cb0 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cc5f0 0x7f53f01f1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cc5f0 0x7f53f01f1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37480/0 (socket says 172.21.15.2:37480) 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 -- 172.21.15.2:0/527099695 learned_addr learned my addr 172.21.15.2:0/527099695 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53ef7fe640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f00cd7c0 0x7f53f01e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 -- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f0154cb0 msgr2=0x7f53f01f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f4fd4640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f0154cb0 0x7f53f01f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f0154cb0 0x7f53f01f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 -- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f00cd7c0 msgr2=0x7f53f01e23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f00cd7c0 0x7f53f01e23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53effff640 1 -- 172.21.15.2:0/527099695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f01518f0 con 0x7f53f00cc5f0 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53ef7fe640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f00cd7c0 0x7f53f01e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.106+0000 7f53f4fd4640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f0154cb0 0x7f53f01f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.107+0000 7f53effff640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cc5f0 0x7f53f01f1290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c820 tx=0x7f53e000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.107+0000 7f53ecff9640 1 -- 172.21.15.2:0/527099695 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0015070 con 0x7f53f00cc5f0 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.107+0000 7f53ecff9640 1 -- 172.21.15.2:0/527099695 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e0007a30 con 0x7f53f00cc5f0 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.107+0000 7f53ecff9640 1 -- 172.21.15.2:0/527099695 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0007de0 con 0x7f53f00cc5f0 2024-09-19T05:42:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.107+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f0151bb0 con 0x7f53f00cc5f0 2024-09-19T05:42:38.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.107+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f01f3dc0 con 0x7f53f00cc5f0 2024-09-19T05:42:38.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.108+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f00cadc0 con 0x7f53f00cc5f0 2024-09-19T05:42:38.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.108+0000 7f53ecff9640 1 -- 172.21.15.2:0/527099695 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f53e0018070 con 0x7f53f00cc5f0 2024-09-19T05:42:38.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.109+0000 7f53ecff9640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f53dc06c2b0 0x7f53dc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.109+0000 7f53ecff9640 1 -- 172.21.15.2:0/527099695 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f53e008b390 con 0x7f53f00cc5f0 2024-09-19T05:42:38.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.109+0000 7f53ef7fe640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f53dc06c2b0 0x7f53dc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.110+0000 7f53ef7fe640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f53dc06c2b0 0x7f53dc06e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f53f01e22b0 tx=0x7f53e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.113+0000 7f53ecff9640 1 -- 172.21.15.2:0/527099695 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f53e0059a70 con 0x7f53f00cc5f0 2024-09-19T05:42:38.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.286+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53f00d01e0 con 0x7f53dc06c2b0 2024-09-19T05:42:38.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.293+0000 7f53ecff9640 1 -- 172.21.15.2:0/527099695 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251927 (secure 0 0 0) 0x7f53f00d01e0 con 0x7f53dc06c2b0 2024-09-19T05:42:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:38.299 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:42:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f53dc06c2b0 msgr2=0x7f53dc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f53dc06c2b0 0x7f53dc06e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f53f01e22b0 tx=0x7f53e4009040 comp rx=0 tx=0).stop 2024-09-19T05:42:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cc5f0 msgr2=0x7f53f01f1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cc5f0 0x7f53f01f1290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c820 tx=0x7f53e000cce0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 shutdown_connections 2024-09-19T05:42:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f53dc06c2b0 0x7f53dc06e770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f00cd7c0 0x7f53f01e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f0154cb0 0x7f53f01f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 --2- 172.21.15.2:0/527099695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cc5f0 0x7f53f01f1290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 >> 172.21.15.2:0/527099695 conn(0x7f53f006d0c0 msgr2=0x7f53f00c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 shutdown_connections 2024-09-19T05:42:38.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.296+0000 7f53f6a37640 1 -- 172.21.15.2:0/527099695 wait complete. 2024-09-19T05:42:38.309 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-19T05:42:37.820549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234796,"kb_used_data":1876,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1921024,"data_stored":1807364,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001541"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058235+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:42:34.058235+0000","last_peered":"2024-09-19T05:42:34.058235+0000","last_clean":"2024-09-19T05:42:34.058235+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:42:34.058235+0000","last_undegraded":"2024-09-19T05:42:34.058235+0000","last_fullsized":"2024-09-19T05:42:34.058235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058494+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:42:34.058494+0000","last_peered":"2024-09-19T05:42:34.058494+0000","last_clean":"2024-09-19T05:42:34.058494+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:42:34.058494+0000","last_undegraded":"2024-09-19T05:42:34.058494+0000","last_fullsized":"2024-09-19T05:42:34.058494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058017+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:42:34.058017+0000","last_peered":"2024-09-19T05:42:34.058017+0000","last_clean":"2024-09-19T05:42:34.058017+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:42:34.058017+0000","last_undegraded":"2024-09-19T05:42:34.058017+0000","last_fullsized":"2024-09-19T05:42:34.058017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923708+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:42:33.923708+0000","last_peered":"2024-09-19T05:42:33.923708+0000","last_clean":"2024-09-19T05:42:33.923708+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:42:33.923708+0000","last_undegraded":"2024-09-19T05:42:33.923708+0000","last_fullsized":"2024-09-19T05:42:33.923708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057846+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:42:34.057846+0000","last_peered":"2024-09-19T05:42:34.057846+0000","last_clean":"2024-09-19T05:42:34.057846+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:42:34.057846+0000","last_undegraded":"2024-09-19T05:42:34.057846+0000","last_fullsized":"2024-09-19T05:42:34.057846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923679+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:42:33.923679+0000","last_peered":"2024-09-19T05:42:33.923679+0000","last_clean":"2024-09-19T05:42:33.923679+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:42:33.923679+0000","last_undegraded":"2024-09-19T05:42:33.923679+0000","last_fullsized":"2024-09-19T05:42:33.923679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057764+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:42:34.057764+0000","last_peered":"2024-09-19T05:42:34.057764+0000","last_clean":"2024-09-19T05:42:34.057764+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:42:34.057764+0000","last_undegraded":"2024-09-19T05:42:34.057764+0000","last_fullsized":"2024-09-19T05:42:34.057764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058259+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:42:34.058259+0000","last_peered":"2024-09-19T05:42:34.058259+0000","last_clean":"2024-09-19T05:42:34.058259+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:42:34.058259+0000","last_undegraded":"2024-09-19T05:42:34.058259+0000","last_fullsized":"2024-09-19T05:42:34.058259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058756+0000","last_change":"2024-09-19T05:42:32.945872+0000","last_active":"2024-09-19T05:42:34.058756+0000","last_peered":"2024-09-19T05:42:34.058756+0000","last_clean":"2024-09-19T05:42:34.058756+0000","last_became_active":"2024-09-19T05:42:32.945545+0000","last_became_peered":"2024-09-19T05:42:32.945545+0000","last_unstale":"2024-09-19T05:42:34.058756+0000","last_undegraded":"2024-09-19T05:42:34.058756+0000","last_fullsized":"2024-09-19T05:42:34.058756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058073+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:42:34.058073+0000","last_peered":"2024-09-19T05:42:34.058073+0000","last_clean":"2024-09-19T05:42:34.058073+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:42:34.058073+0000","last_undegraded":"2024-09-19T05:42:34.058073+0000","last_fullsized":"2024-09-19T05:42:34.058073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058096+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:42:34.058096+0000","last_peered":"2024-09-19T05:42:34.058096+0000","last_clean":"2024-09-19T05:42:34.058096+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:42:34.058096+0000","last_undegraded":"2024-09-19T05:42:34.058096+0000","last_fullsized":"2024-09-19T05:42:34.058096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058167+0000","last_change":"2024-09-19T05:42:32.941601+0000","last_active":"2024-09-19T05:42:34.058167+0000","last_peered":"2024-09-19T05:42:34.058167+0000","last_clean":"2024-09-19T05:42:34.058167+0000","last_became_active":"2024-09-19T05:42:32.941288+0000","last_became_peered":"2024-09-19T05:42:32.941288+0000","last_unstale":"2024-09-19T05:42:34.058167+0000","last_undegraded":"2024-09-19T05:42:34.058167+0000","last_fullsized":"2024-09-19T05:42:34.058167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:35.670719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058583+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:42:34.058583+0000","last_peered":"2024-09-19T05:42:34.058583+0000","last_clean":"2024-09-19T05:42:34.058583+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:42:34.058583+0000","last_undegraded":"2024-09-19T05:42:34.058583+0000","last_fullsized":"2024-09-19T05:42:34.058583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058482+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:42:34.058482+0000","last_peered":"2024-09-19T05:42:34.058482+0000","last_clean":"2024-09-19T05:42:34.058482+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:42:34.058482+0000","last_undegraded":"2024-09-19T05:42:34.058482+0000","last_fullsized":"2024-09-19T05:42:34.058482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058261+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:42:34.058261+0000","last_peered":"2024-09-19T05:42:34.058261+0000","last_clean":"2024-09-19T05:42:34.058261+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:42:34.058261+0000","last_undegraded":"2024-09-19T05:42:34.058261+0000","last_fullsized":"2024-09-19T05:42:34.058261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057516+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:42:34.057516+0000","last_peered":"2024-09-19T05:42:34.057516+0000","last_clean":"2024-09-19T05:42:34.057516+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:42:34.057516+0000","last_undegraded":"2024-09-19T05:42:34.057516+0000","last_fullsized":"2024-09-19T05:42:34.057516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059660+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:42:34.059660+0000","last_peered":"2024-09-19T05:42:34.059660+0000","last_clean":"2024-09-19T05:42:34.059660+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:42:34.059660+0000","last_undegraded":"2024-09-19T05:42:34.059660+0000","last_fullsized":"2024-09-19T05:42:34.059660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059534+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:42:34.059534+0000","last_peered":"2024-09-19T05:42:34.059534+0000","last_clean":"2024-09-19T05:42:34.059534+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:42:34.059534+0000","last_undegraded":"2024-09-19T05:42:34.059534+0000","last_fullsized":"2024-09-19T05:42:34.059534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058089+0000","last_change":"2024-09-19T05:42:32.940627+0000","last_active":"2024-09-19T05:42:34.058089+0000","last_peered":"2024-09-19T05:42:34.058089+0000","last_clean":"2024-09-19T05:42:34.058089+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058089+0000","last_undegraded":"2024-09-19T05:42:34.058089+0000","last_fullsized":"2024-09-19T05:42:34.058089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.616840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925755+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:42:33.925755+0000","last_peered":"2024-09-19T05:42:33.925755+0000","last_clean":"2024-09-19T05:42:33.925755+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:42:33.925755+0000","last_undegraded":"2024-09-19T05:42:33.925755+0000","last_fullsized":"2024-09-19T05:42:33.925755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058013+0000","last_change":"2024-09-19T05:42:32.940425+0000","last_active":"2024-09-19T05:42:34.058013+0000","last_peered":"2024-09-19T05:42:34.058013+0000","last_clean":"2024-09-19T05:42:34.058013+0000","last_became_active":"2024-09-19T05:42:32.940059+0000","last_became_peered":"2024-09-19T05:42:32.940059+0000","last_unstale":"2024-09-19T05:42:34.058013+0000","last_undegraded":"2024-09-19T05:42:34.058013+0000","last_fullsized":"2024-09-19T05:42:34.058013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058206+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:42:34.058206+0000","last_peered":"2024-09-19T05:42:34.058206+0000","last_clean":"2024-09-19T05:42:34.058206+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:42:34.058206+0000","last_undegraded":"2024-09-19T05:42:34.058206+0000","last_fullsized":"2024-09-19T05:42:34.058206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057512+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:42:34.057512+0000","last_peered":"2024-09-19T05:42:34.057512+0000","last_clean":"2024-09-19T05:42:34.057512+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:42:34.057512+0000","last_undegraded":"2024-09-19T05:42:34.057512+0000","last_fullsized":"2024-09-19T05:42:34.057512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058955+0000","last_change":"2024-09-19T05:42:32.941414+0000","last_active":"2024-09-19T05:42:34.058955+0000","last_peered":"2024-09-19T05:42:34.058955+0000","last_clean":"2024-09-19T05:42:34.058955+0000","last_became_active":"2024-09-19T05:42:32.941076+0000","last_became_peered":"2024-09-19T05:42:32.941076+0000","last_unstale":"2024-09-19T05:42:34.058955+0000","last_undegraded":"2024-09-19T05:42:34.058955+0000","last_fullsized":"2024-09-19T05:42:34.058955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:13.371046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923839+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:42:33.923839+0000","last_peered":"2024-09-19T05:42:33.923839+0000","last_clean":"2024-09-19T05:42:33.923839+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:42:33.923839+0000","last_undegraded":"2024-09-19T05:42:33.923839+0000","last_fullsized":"2024-09-19T05:42:33.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058867+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:42:34.058867+0000","last_peered":"2024-09-19T05:42:34.058867+0000","last_clean":"2024-09-19T05:42:34.058867+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:42:34.058867+0000","last_undegraded":"2024-09-19T05:42:34.058867+0000","last_fullsized":"2024-09-19T05:42:34.058867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058377+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:42:34.058377+0000","last_peered":"2024-09-19T05:42:34.058377+0000","last_clean":"2024-09-19T05:42:34.058377+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:42:34.058377+0000","last_undegraded":"2024-09-19T05:42:34.058377+0000","last_fullsized":"2024-09-19T05:42:34.058377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059114+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:42:34.059114+0000","last_peered":"2024-09-19T05:42:34.059114+0000","last_clean":"2024-09-19T05:42:34.059114+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:42:34.059114+0000","last_undegraded":"2024-09-19T05:42:34.059114+0000","last_fullsized":"2024-09-19T05:42:34.059114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924874+0000","last_change":"2024-09-19T05:42:32.927184+0000","last_active":"2024-09-19T05:42:33.924874+0000","last_peered":"2024-09-19T05:42:33.924874+0000","last_clean":"2024-09-19T05:42:33.924874+0000","last_became_active":"2024-09-19T05:42:32.926952+0000","last_became_peered":"2024-09-19T05:42:32.926952+0000","last_unstale":"2024-09-19T05:42:33.924874+0000","last_undegraded":"2024-09-19T05:42:33.924874+0000","last_fullsized":"2024-09-19T05:42:33.924874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923475+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:42:33.923475+0000","last_peered":"2024-09-19T05:42:33.923475+0000","last_clean":"2024-09-19T05:42:33.923475+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:42:33.923475+0000","last_undegraded":"2024-09-19T05:42:33.923475+0000","last_fullsized":"2024-09-19T05:42:33.923475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058161+0000","last_change":"2024-09-19T05:42:32.942584+0000","last_active":"2024-09-19T05:42:34.058161+0000","last_peered":"2024-09-19T05:42:34.058161+0000","last_clean":"2024-09-19T05:42:34.058161+0000","last_became_active":"2024-09-19T05:42:32.942274+0000","last_became_peered":"2024-09-19T05:42:32.942274+0000","last_unstale":"2024-09-19T05:42:34.058161+0000","last_undegraded":"2024-09-19T05:42:34.058161+0000","last_fullsized":"2024-09-19T05:42:34.058161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058203+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:42:34.058203+0000","last_peered":"2024-09-19T05:42:34.058203+0000","last_clean":"2024-09-19T05:42:34.058203+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:42:34.058203+0000","last_undegraded":"2024-09-19T05:42:34.058203+0000","last_fullsized":"2024-09-19T05:42:34.058203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925748+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:42:33.925748+0000","last_peered":"2024-09-19T05:42:33.925748+0000","last_clean":"2024-09-19T05:42:33.925748+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:42:33.925748+0000","last_undegraded":"2024-09-19T05:42:33.925748+0000","last_fullsized":"2024-09-19T05:42:33.925748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059210+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:42:34.059210+0000","last_peered":"2024-09-19T05:42:34.059210+0000","last_clean":"2024-09-19T05:42:34.059210+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:42:34.059210+0000","last_undegraded":"2024-09-19T05:42:34.059210+0000","last_fullsized":"2024-09-19T05:42:34.059210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058211+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:42:34.058211+0000","last_peered":"2024-09-19T05:42:34.058211+0000","last_clean":"2024-09-19T05:42:34.058211+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:42:34.058211+0000","last_undegraded":"2024-09-19T05:42:34.058211+0000","last_fullsized":"2024-09-19T05:42:34.058211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058257+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:42:34.058257+0000","last_peered":"2024-09-19T05:42:34.058257+0000","last_clean":"2024-09-19T05:42:34.058257+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058257+0000","last_undegraded":"2024-09-19T05:42:34.058257+0000","last_fullsized":"2024-09-19T05:42:34.058257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057643+0000","last_change":"2024-09-19T05:42:31.995072+0000","last_active":"2024-09-19T05:42:34.057643+0000","last_peered":"2024-09-19T05:42:34.057643+0000","last_clean":"2024-09-19T05:42:34.057643+0000","last_became_active":"2024-09-19T05:42:31.994586+0000","last_became_peered":"2024-09-19T05:42:31.994586+0000","last_unstale":"2024-09-19T05:42:34.057643+0000","last_undegraded":"2024-09-19T05:42:34.057643+0000","last_fullsized":"2024-09-19T05:42:34.057643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925971+0000","last_change":"2024-09-19T05:42:32.928695+0000","last_active":"2024-09-19T05:42:33.925971+0000","last_peered":"2024-09-19T05:42:33.925971+0000","last_clean":"2024-09-19T05:42:33.925971+0000","last_became_active":"2024-09-19T05:42:32.928437+0000","last_became_peered":"2024-09-19T05:42:32.928437+0000","last_unstale":"2024-09-19T05:42:33.925971+0000","last_undegraded":"2024-09-19T05:42:33.925971+0000","last_fullsized":"2024-09-19T05:42:33.925971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058846+0000","last_change":"2024-09-19T05:42:31.985393+0000","last_active":"2024-09-19T05:42:34.058846+0000","last_peered":"2024-09-19T05:42:34.058846+0000","last_clean":"2024-09-19T05:42:34.058846+0000","last_became_active":"2024-09-19T05:42:31.985034+0000","last_became_peered":"2024-09-19T05:42:31.985034+0000","last_unstale":"2024-09-19T05:42:34.058846+0000","last_undegraded":"2024-09-19T05:42:34.058846+0000","last_fullsized":"2024-09-19T05:42:34.058846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:11.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923861+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:42:33.923861+0000","last_peered":"2024-09-19T05:42:33.923861+0000","last_clean":"2024-09-19T05:42:33.923861+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:42:33.923861+0000","last_undegraded":"2024-09-19T05:42:33.923861+0000","last_fullsized":"2024-09-19T05:42:33.923861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058193+0000","last_change":"2024-09-19T05:42:31.987634+0000","last_active":"2024-09-19T05:42:34.058193+0000","last_peered":"2024-09-19T05:42:34.058193+0000","last_clean":"2024-09-19T05:42:34.058193+0000","last_became_active":"2024-09-19T05:42:31.986452+0000","last_became_peered":"2024-09-19T05:42:31.986452+0000","last_unstale":"2024-09-19T05:42:34.058193+0000","last_undegraded":"2024-09-19T05:42:34.058193+0000","last_fullsized":"2024-09-19T05:42:34.058193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925095+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:42:33.925095+0000","last_peered":"2024-09-19T05:42:33.925095+0000","last_clean":"2024-09-19T05:42:33.925095+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:42:33.925095+0000","last_undegraded":"2024-09-19T05:42:33.925095+0000","last_fullsized":"2024-09-19T05:42:33.925095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059270+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:42:34.059270+0000","last_peered":"2024-09-19T05:42:34.059270+0000","last_clean":"2024-09-19T05:42:34.059270+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:42:34.059270+0000","last_undegraded":"2024-09-19T05:42:34.059270+0000","last_fullsized":"2024-09-19T05:42:34.059270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923690+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:42:33.923690+0000","last_peered":"2024-09-19T05:42:33.923690+0000","last_clean":"2024-09-19T05:42:33.923690+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:42:33.923690+0000","last_undegraded":"2024-09-19T05:42:33.923690+0000","last_fullsized":"2024-09-19T05:42:33.923690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058283+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:42:34.058283+0000","last_peered":"2024-09-19T05:42:34.058283+0000","last_clean":"2024-09-19T05:42:34.058283+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:42:34.058283+0000","last_undegraded":"2024-09-19T05:42:34.058283+0000","last_fullsized":"2024-09-19T05:42:34.058283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925200+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:42:33.925200+0000","last_peered":"2024-09-19T05:42:33.925200+0000","last_clean":"2024-09-19T05:42:33.925200+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:42:33.925200+0000","last_undegraded":"2024-09-19T05:42:33.925200+0000","last_fullsized":"2024-09-19T05:42:33.925200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058589+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:42:34.058589+0000","last_peered":"2024-09-19T05:42:34.058589+0000","last_clean":"2024-09-19T05:42:34.058589+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:42:34.058589+0000","last_undegraded":"2024-09-19T05:42:34.058589+0000","last_fullsized":"2024-09-19T05:42:34.058589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059325+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:42:34.059325+0000","last_peered":"2024-09-19T05:42:34.059325+0000","last_clean":"2024-09-19T05:42:34.059325+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:42:34.059325+0000","last_undegraded":"2024-09-19T05:42:34.059325+0000","last_fullsized":"2024-09-19T05:42:34.059325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059177+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:42:34.059177+0000","last_peered":"2024-09-19T05:42:34.059177+0000","last_clean":"2024-09-19T05:42:34.059177+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:42:34.059177+0000","last_undegraded":"2024-09-19T05:42:34.059177+0000","last_fullsized":"2024-09-19T05:42:34.059177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057760+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:42:34.057760+0000","last_peered":"2024-09-19T05:42:34.057760+0000","last_clean":"2024-09-19T05:42:34.057760+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:42:34.057760+0000","last_undegraded":"2024-09-19T05:42:34.057760+0000","last_fullsized":"2024-09-19T05:42:34.057760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058006+0000","last_change":"2024-09-19T05:42:32.938549+0000","last_active":"2024-09-19T05:42:34.058006+0000","last_peered":"2024-09-19T05:42:34.058006+0000","last_clean":"2024-09-19T05:42:34.058006+0000","last_became_active":"2024-09-19T05:42:32.938113+0000","last_became_peered":"2024-09-19T05:42:32.938113+0000","last_unstale":"2024-09-19T05:42:34.058006+0000","last_undegraded":"2024-09-19T05:42:34.058006+0000","last_fullsized":"2024-09-19T05:42:34.058006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:15.787698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925541+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:42:33.925541+0000","last_peered":"2024-09-19T05:42:33.925541+0000","last_clean":"2024-09-19T05:42:33.925541+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:42:33.925541+0000","last_undegraded":"2024-09-19T05:42:33.925541+0000","last_fullsized":"2024-09-19T05:42:33.925541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058325+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:42:34.058325+0000","last_peered":"2024-09-19T05:42:34.058325+0000","last_clean":"2024-09-19T05:42:34.058325+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:42:34.058325+0000","last_undegraded":"2024-09-19T05:42:34.058325+0000","last_fullsized":"2024-09-19T05:42:34.058325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923969+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:42:33.923969+0000","last_peered":"2024-09-19T05:42:33.923969+0000","last_clean":"2024-09-19T05:42:33.923969+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:42:33.923969+0000","last_undegraded":"2024-09-19T05:42:33.923969+0000","last_fullsized":"2024-09-19T05:42:33.923969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925826+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:42:33.925826+0000","last_peered":"2024-09-19T05:42:33.925826+0000","last_clean":"2024-09-19T05:42:33.925826+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:42:33.925826+0000","last_undegraded":"2024-09-19T05:42:33.925826+0000","last_fullsized":"2024-09-19T05:42:33.925826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058299+0000","last_change":"2024-09-19T05:42:32.945685+0000","last_active":"2024-09-19T05:42:34.058299+0000","last_peered":"2024-09-19T05:42:34.058299+0000","last_clean":"2024-09-19T05:42:34.058299+0000","last_became_active":"2024-09-19T05:42:32.945337+0000","last_became_peered":"2024-09-19T05:42:32.945337+0000","last_unstale":"2024-09-19T05:42:34.058299+0000","last_undegraded":"2024-09-19T05:42:34.058299+0000","last_fullsized":"2024-09-19T05:42:34.058299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058153+0000","last_change":"2024-09-19T05:42:31.991430+0000","last_active":"2024-09-19T05:42:34.058153+0000","last_peered":"2024-09-19T05:42:34.058153+0000","last_clean":"2024-09-19T05:42:34.058153+0000","last_became_active":"2024-09-19T05:42:31.991057+0000","last_became_peered":"2024-09-19T05:42:31.991057+0000","last_unstale":"2024-09-19T05:42:34.058153+0000","last_undegraded":"2024-09-19T05:42:34.058153+0000","last_fullsized":"2024-09-19T05:42:34.058153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:58.247495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058366+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:42:34.058366+0000","last_peered":"2024-09-19T05:42:34.058366+0000","last_clean":"2024-09-19T05:42:34.058366+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:42:34.058366+0000","last_undegraded":"2024-09-19T05:42:34.058366+0000","last_fullsized":"2024-09-19T05:42:34.058366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058851+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:42:34.058851+0000","last_peered":"2024-09-19T05:42:34.058851+0000","last_clean":"2024-09-19T05:42:34.058851+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:42:34.058851+0000","last_undegraded":"2024-09-19T05:42:34.058851+0000","last_fullsized":"2024-09-19T05:42:34.058851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059399+0000","last_change":"2024-09-19T05:42:32.941889+0000","last_active":"2024-09-19T05:42:34.059399+0000","last_peered":"2024-09-19T05:42:34.059399+0000","last_clean":"2024-09-19T05:42:34.059399+0000","last_became_active":"2024-09-19T05:42:32.941254+0000","last_became_peered":"2024-09-19T05:42:32.941254+0000","last_unstale":"2024-09-19T05:42:34.059399+0000","last_undegraded":"2024-09-19T05:42:34.059399+0000","last_fullsized":"2024-09-19T05:42:34.059399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.771973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057837+0000","last_change":"2024-09-19T05:42:23.458936+0000","last_active":"2024-09-19T05:42:34.057837+0000","last_peered":"2024-09-19T05:42:34.057837+0000","last_clean":"2024-09-19T05:42:34.057837+0000","last_became_active":"2024-09-19T05:42:23.457990+0000","last_became_peered":"2024-09-19T05:42:23.457990+0000","last_unstale":"2024-09-19T05:42:34.057837+0000","last_undegraded":"2024-09-19T05:42:34.057837+0000","last_fullsized":"2024-09-19T05:42:34.057837+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:45.259241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059347+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:42:34.059347+0000","last_peered":"2024-09-19T05:42:34.059347+0000","last_clean":"2024-09-19T05:42:34.059347+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:42:34.059347+0000","last_undegraded":"2024-09-19T05:42:34.059347+0000","last_fullsized":"2024-09-19T05:42:34.059347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925215+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:42:33.925215+0000","last_peered":"2024-09-19T05:42:33.925215+0000","last_clean":"2024-09-19T05:42:33.925215+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:42:33.925215+0000","last_undegraded":"2024-09-19T05:42:33.925215+0000","last_fullsized":"2024-09-19T05:42:33.925215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057735+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:42:34.057735+0000","last_peered":"2024-09-19T05:42:34.057735+0000","last_clean":"2024-09-19T05:42:34.057735+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:42:34.057735+0000","last_undegraded":"2024-09-19T05:42:34.057735+0000","last_fullsized":"2024-09-19T05:42:34.057735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058126+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:42:34.058126+0000","last_peered":"2024-09-19T05:42:34.058126+0000","last_clean":"2024-09-19T05:42:34.058126+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:42:34.058126+0000","last_undegraded":"2024-09-19T05:42:34.058126+0000","last_fullsized":"2024-09-19T05:42:34.058126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924861+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:42:33.924861+0000","last_peered":"2024-09-19T05:42:33.924861+0000","last_clean":"2024-09-19T05:42:33.924861+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:42:33.924861+0000","last_undegraded":"2024-09-19T05:42:33.924861+0000","last_fullsized":"2024-09-19T05:42:33.924861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057949+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:42:34.057949+0000","last_peered":"2024-09-19T05:42:34.057949+0000","last_clean":"2024-09-19T05:42:34.057949+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:42:34.057949+0000","last_undegraded":"2024-09-19T05:42:34.057949+0000","last_fullsized":"2024-09-19T05:42:34.057949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059345+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:42:34.059345+0000","last_peered":"2024-09-19T05:42:34.059345+0000","last_clean":"2024-09-19T05:42:34.059345+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:42:34.059345+0000","last_undegraded":"2024-09-19T05:42:34.059345+0000","last_fullsized":"2024-09-19T05:42:34.059345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057870+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:42:34.057870+0000","last_peered":"2024-09-19T05:42:34.057870+0000","last_clean":"2024-09-19T05:42:34.057870+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:42:34.057870+0000","last_undegraded":"2024-09-19T05:42:34.057870+0000","last_fullsized":"2024-09-19T05:42:34.057870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058164+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:42:34.058164+0000","last_peered":"2024-09-19T05:42:34.058164+0000","last_clean":"2024-09-19T05:42:34.058164+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:42:34.058164+0000","last_undegraded":"2024-09-19T05:42:34.058164+0000","last_fullsized":"2024-09-19T05:42:34.058164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923659+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:42:33.923659+0000","last_peered":"2024-09-19T05:42:33.923659+0000","last_clean":"2024-09-19T05:42:33.923659+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:42:33.923659+0000","last_undegraded":"2024-09-19T05:42:33.923659+0000","last_fullsized":"2024-09-19T05:42:33.923659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059484+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:42:34.059484+0000","last_peered":"2024-09-19T05:42:34.059484+0000","last_clean":"2024-09-19T05:42:34.059484+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:42:34.059484+0000","last_undegraded":"2024-09-19T05:42:34.059484+0000","last_fullsized":"2024-09-19T05:42:34.059484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058726+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:42:34.058726+0000","last_peered":"2024-09-19T05:42:34.058726+0000","last_clean":"2024-09-19T05:42:34.058726+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:42:34.058726+0000","last_undegraded":"2024-09-19T05:42:34.058726+0000","last_fullsized":"2024-09-19T05:42:34.058726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925942+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:42:33.925942+0000","last_peered":"2024-09-19T05:42:33.925942+0000","last_clean":"2024-09-19T05:42:33.925942+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:42:33.925942+0000","last_undegraded":"2024-09-19T05:42:33.925942+0000","last_fullsized":"2024-09-19T05:42:33.925942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925132+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:42:33.925132+0000","last_peered":"2024-09-19T05:42:33.925132+0000","last_clean":"2024-09-19T05:42:33.925132+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:42:33.925132+0000","last_undegraded":"2024-09-19T05:42:33.925132+0000","last_fullsized":"2024-09-19T05:42:33.925132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059606+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:42:34.059606+0000","last_peered":"2024-09-19T05:42:34.059606+0000","last_clean":"2024-09-19T05:42:34.059606+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:42:34.059606+0000","last_undegraded":"2024-09-19T05:42:34.059606+0000","last_fullsized":"2024-09-19T05:42:34.059606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925887+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:42:33.925887+0000","last_peered":"2024-09-19T05:42:33.925887+0000","last_clean":"2024-09-19T05:42:33.925887+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:42:33.925887+0000","last_undegraded":"2024-09-19T05:42:33.925887+0000","last_fullsized":"2024-09-19T05:42:33.925887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057349+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:42:34.057349+0000","last_peered":"2024-09-19T05:42:34.057349+0000","last_clean":"2024-09-19T05:42:34.057349+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:42:34.057349+0000","last_undegraded":"2024-09-19T05:42:34.057349+0000","last_fullsized":"2024-09-19T05:42:34.057349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059062+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:42:34.059062+0000","last_peered":"2024-09-19T05:42:34.059062+0000","last_clean":"2024-09-19T05:42:34.059062+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:42:34.059062+0000","last_undegraded":"2024-09-19T05:42:34.059062+0000","last_fullsized":"2024-09-19T05:42:34.059062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925905+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:42:33.925905+0000","last_peered":"2024-09-19T05:42:33.925905+0000","last_clean":"2024-09-19T05:42:33.925905+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:42:33.925905+0000","last_undegraded":"2024-09-19T05:42:33.925905+0000","last_fullsized":"2024-09-19T05:42:33.925905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925556+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:42:33.925556+0000","last_peered":"2024-09-19T05:42:33.925556+0000","last_clean":"2024-09-19T05:42:33.925556+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:42:33.925556+0000","last_undegraded":"2024-09-19T05:42:33.925556+0000","last_fullsized":"2024-09-19T05:42:33.925556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058603+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:42:34.058603+0000","last_peered":"2024-09-19T05:42:34.058603+0000","last_clean":"2024-09-19T05:42:34.058603+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:42:34.058603+0000","last_undegraded":"2024-09-19T05:42:34.058603+0000","last_fullsized":"2024-09-19T05:42:34.058603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925165+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:42:33.925165+0000","last_peered":"2024-09-19T05:42:33.925165+0000","last_clean":"2024-09-19T05:42:33.925165+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:42:33.925165+0000","last_undegraded":"2024-09-19T05:42:33.925165+0000","last_fullsized":"2024-09-19T05:42:33.925165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059115+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:42:34.059115+0000","last_peered":"2024-09-19T05:42:34.059115+0000","last_clean":"2024-09-19T05:42:34.059115+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:42:34.059115+0000","last_undegraded":"2024-09-19T05:42:34.059115+0000","last_fullsized":"2024-09-19T05:42:34.059115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059198+0000","last_change":"2024-09-19T05:42:32.941131+0000","last_active":"2024-09-19T05:42:34.059198+0000","last_peered":"2024-09-19T05:42:34.059198+0000","last_clean":"2024-09-19T05:42:34.059198+0000","last_became_active":"2024-09-19T05:42:32.940596+0000","last_became_peered":"2024-09-19T05:42:32.940596+0000","last_unstale":"2024-09-19T05:42:34.059198+0000","last_undegraded":"2024-09-19T05:42:34.059198+0000","last_fullsized":"2024-09-19T05:42:34.059198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:00.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059376+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:42:34.059376+0000","last_peered":"2024-09-19T05:42:34.059376+0000","last_clean":"2024-09-19T05:42:34.059376+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:42:34.059376+0000","last_undegraded":"2024-09-19T05:42:34.059376+0000","last_fullsized":"2024-09-19T05:42:34.059376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058312+0000","last_change":"2024-09-19T05:42:31.988393+0000","last_active":"2024-09-19T05:42:34.058312+0000","last_peered":"2024-09-19T05:42:34.058312+0000","last_clean":"2024-09-19T05:42:34.058312+0000","last_became_active":"2024-09-19T05:42:31.986557+0000","last_became_peered":"2024-09-19T05:42:31.986557+0000","last_unstale":"2024-09-19T05:42:34.058312+0000","last_undegraded":"2024-09-19T05:42:34.058312+0000","last_fullsized":"2024-09-19T05:42:34.058312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058853+0000","last_change":"2024-09-19T05:42:32.929140+0000","last_active":"2024-09-19T05:42:34.058853+0000","last_peered":"2024-09-19T05:42:34.058853+0000","last_clean":"2024-09-19T05:42:34.058853+0000","last_became_active":"2024-09-19T05:42:32.928811+0000","last_became_peered":"2024-09-19T05:42:32.928811+0000","last_unstale":"2024-09-19T05:42:34.058853+0000","last_undegraded":"2024-09-19T05:42:34.058853+0000","last_fullsized":"2024-09-19T05:42:34.058853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:29.306206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058372+0000","last_change":"2024-09-19T05:42:31.988083+0000","last_active":"2024-09-19T05:42:34.058372+0000","last_peered":"2024-09-19T05:42:34.058372+0000","last_clean":"2024-09-19T05:42:34.058372+0000","last_became_active":"2024-09-19T05:42:31.987088+0000","last_became_peered":"2024-09-19T05:42:31.987088+0000","last_unstale":"2024-09-19T05:42:34.058372+0000","last_undegraded":"2024-09-19T05:42:34.058372+0000","last_fullsized":"2024-09-19T05:42:34.058372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.941310+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.940997+0000","last_became_peered":"2024-09-19T05:42:32.940997+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:58.755996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058706+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:42:34.058706+0000","last_peered":"2024-09-19T05:42:34.058706+0000","last_clean":"2024-09-19T05:42:34.058706+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:42:34.058706+0000","last_undegraded":"2024-09-19T05:42:34.058706+0000","last_fullsized":"2024-09-19T05:42:34.058706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058604+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:42:34.058604+0000","last_peered":"2024-09-19T05:42:34.058604+0000","last_clean":"2024-09-19T05:42:34.058604+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:42:34.058604+0000","last_undegraded":"2024-09-19T05:42:34.058604+0000","last_fullsized":"2024-09-19T05:42:34.058604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":290816,"data_stored":669484,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":663184,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:42:38.313 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T05:42:38.313 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T05:42:38.314 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-19T05:42:38.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-19T05:42:38.314 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-19T05:42:38.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.450+0000 7feb8ebb7640 1 Processor -- start 2024-09-19T05:42:38.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.450+0000 7feb8ebb7640 1 -- start start 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.450+0000 7feb8ebb7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feb880d06e0 0x7feb880cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.450+0000 7feb8ebb7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb880d0ac0 0x7feb8815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.450+0000 7feb8ebb7640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb880cb400 con 0x7feb88154af0 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.450+0000 7feb8ebb7640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7feb8815f090 con 0x7feb880d06e0 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.450+0000 7feb8ebb7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feb880cb6e0 con 0x7feb880d0ac0 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb8c953640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feb880d06e0 0x7feb880cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb8c953640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feb880d06e0 0x7feb880cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37488/0 (socket says 172.21.15.2:37488) 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb8c953640 1 -- 172.21.15.2:0/4008525584 learned_addr learned my addr 172.21.15.2:0/4008525584 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:38.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb87fff640 1 --2- 172.21.15.2:0/4008525584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb880d0ac0 0x7feb8815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb8c953640 1 -- 172.21.15.2:0/4008525584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb880d0ac0 msgr2=0x7feb8815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb8c953640 1 --2- 172.21.15.2:0/4008525584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb880d0ac0 0x7feb8815eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb8c953640 1 -- 172.21.15.2:0/4008525584 >> v1:172.21.15.123:6789/0 conn(0x7feb88154af0 legacy=0x7feb880cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.451+0000 7feb8c953640 1 -- 172.21.15.2:0/4008525584 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb880cc970 con 0x7feb880d06e0 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8c953640 1 --2- 172.21.15.2:0/4008525584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d06e0 0x7feb880cbb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb80009870 tx=0x7feb8002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a28d58b151cbe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb877fe640 1 -- 172.21.15.2:0/4008525584 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80037070 con 0x7feb880d06e0 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb877fe640 1 -- 172.21.15.2:0/4008525584 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb80032440 con 0x7feb880d06e0 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb877fe640 1 -- 172.21.15.2:0/4008525584 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80032810 con 0x7feb880d06e0 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4008525584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d06e0 msgr2=0x7feb880cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4008525584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d06e0 0x7feb880cbb40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb80009870 tx=0x7feb8002a210 comp rx=0 tx=0).stop 2024-09-19T05:42:38.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4008525584 shutdown_connections 2024-09-19T05:42:38.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4008525584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb880d0ac0 0x7feb8815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4008525584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d06e0 0x7feb880cbb40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4008525584 >> 172.21.15.2:0/4008525584 conn(0x7feb8806d740 msgr2=0x7feb88073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:38.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4008525584 shutdown_connections 2024-09-19T05:42:38.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.452+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4008525584 wait complete. 2024-09-19T05:42:38.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.454+0000 7feb8ebb7640 1 Processor -- start 2024-09-19T05:42:38.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.454+0000 7feb8ebb7640 1 -- start start 2024-09-19T05:42:38.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.454+0000 7feb8ebb7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb880d06e0 0x7feb881f0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.454+0000 7feb8ebb7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d0ac0 0x7feb88150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.454+0000 7feb8ebb7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb88154af0 0x7feb88157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb8ebb7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7feb880c5830 con 0x7feb880d0ac0 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb8ebb7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb880ccf30 con 0x7feb880d06e0 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb8ebb7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feb880cdb60 con 0x7feb88154af0 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d0ac0 0x7feb88150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d0ac0 0x7feb88150990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37490/0 (socket says 172.21.15.2:37490) 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 -- 172.21.15.2:0/4048823928 learned_addr learned my addr 172.21.15.2:0/4048823928 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb8c953640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb880d06e0 0x7feb881f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb8d154640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb88154af0 0x7feb88157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 -- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb88154af0 msgr2=0x7feb88157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb88154af0 0x7feb88157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 -- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb880d06e0 msgr2=0x7feb881f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb880d06e0 0x7feb881f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 -- 172.21.15.2:0/4048823928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb881f0b80 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb87fff640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d0ac0 0x7feb88150990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb78007ef0 tx=0x7feb7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb857fa640 1 -- 172.21.15.2:0/4048823928 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78014620 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb857fa640 1 -- 172.21.15.2:0/4048823928 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb78019070 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb881f0e40 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb857fa640 1 -- 172.21.15.2:0/4048823928 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78014a00 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.455+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb881f1400 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.456+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb880cbb40 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.457+0000 7feb857fa640 1 -- 172.21.15.2:0/4048823928 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feb78014ba0 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.457+0000 7feb857fa640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb6806c2b0 0x7feb6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.457+0000 7feb857fa640 1 -- 172.21.15.2:0/4048823928 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feb7808bbd0 con 0x7feb880d0ac0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.458+0000 7feb8c953640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb6806c2b0 0x7feb6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.458+0000 7feb8c953640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb6806c2b0 0x7feb6806e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb80034000 tx=0x7feb8002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.462+0000 7feb857fa640 1 -- 172.21.15.2:0/4048823928 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7feb78059cf0 con 0x7feb880d0ac0 2024-09-19T05:42:38.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.683+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7feb88068140 con 0x7feb880d0ac0 2024-09-19T05:42:38.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.686+0000 7feb857fa640 1 -- 172.21.15.2:0/4048823928 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173508 (secure 0 0 0) 0x7feb7805dff0 con 0x7feb880d0ac0 2024-09-19T05:42:38.690 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:38.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb6806c2b0 msgr2=0x7feb6806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb6806c2b0 0x7feb6806e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb80034000 tx=0x7feb8002e000 comp rx=0 tx=0).stop 2024-09-19T05:42:38.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d0ac0 msgr2=0x7feb88150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d0ac0 0x7feb88150990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb78007ef0 tx=0x7feb7800e4b0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 shutdown_connections 2024-09-19T05:42:38.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb6806c2b0 0x7feb6806e770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb88154af0 0x7feb88157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb880d0ac0 0x7feb88150990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 --2- 172.21.15.2:0/4048823928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb880d06e0 0x7feb881f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 >> 172.21.15.2:0/4048823928 conn(0x7feb8806d740 msgr2=0x7feb880cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:38.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 shutdown_connections 2024-09-19T05:42:38.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.689+0000 7feb8ebb7640 1 -- 172.21.15.2:0/4048823928 wait complete. 2024-09-19T05:42:38.702 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":302582025},{"type":"v1","addr":"172.21.15.123:6833","nonce":302582025}]},"active_addr":"172.21.15.123:6833/302582025","active_change":"2024-09-19T05:42:13.809176+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4098,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2937210004}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":311463449}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":4136187285}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1444224951}]}]} 2024-09-19T05:42:38.707 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-19T05:42:38.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T05:42:38.707 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.807+0000 7faf9b3a6640 1 Processor -- start 2024-09-19T05:42:38.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.807+0000 7faf9b3a6640 1 -- start start 2024-09-19T05:42:38.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf9b3a6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf94154d50 0x7faf9415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf9b3a6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7faf940cdb00 0x7faf940c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf9b3a6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf940c8f50 con 0x7faf940cb6c0 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf9b3a6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7faf940cd670 con 0x7faf940cdb00 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf9b3a6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7faf9415b8f0 con 0x7faf94154d50 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf98941640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7faf940cdb00 0x7faf940c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf98941640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7faf940cdb00 0x7faf940c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37502/0 (socket says 172.21.15.2:37502) 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf98941640 1 -- 172.21.15.2:0/1922619470 learned_addr learned my addr 172.21.15.2:0/1922619470 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf99142640 1 --2- 172.21.15.2:0/1922619470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf94154d50 0x7faf9415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf98941640 1 -- 172.21.15.2:0/1922619470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf94154d50 msgr2=0x7faf9415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf98941640 1 --2- 172.21.15.2:0/1922619470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf94154d50 0x7faf9415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf98941640 1 -- 172.21.15.2:0/1922619470 >> v1:172.21.15.123:6789/0 conn(0x7faf940cb6c0 legacy=0x7faf940d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.808+0000 7faf98941640 1 -- 172.21.15.2:0/1922619470 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf940cc780 con 0x7faf940cdb00 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.809+0000 7faf98941640 1 --2- 172.21.15.2:0/1922619470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf940cdb00 0x7faf940c8a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf84009870 tx=0x7faf8402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5dc8fe9afba4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.809+0000 7faf8bfff640 1 -- 172.21.15.2:0/1922619470 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84037070 con 0x7faf940cdb00 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.809+0000 7faf8bfff640 1 -- 172.21.15.2:0/1922619470 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf8402ae50 con 0x7faf940cdb00 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.809+0000 7faf8bfff640 1 -- 172.21.15.2:0/1922619470 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84032860 con 0x7faf940cdb00 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 -- 172.21.15.2:0/1922619470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf940cdb00 msgr2=0x7faf940c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/1922619470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf940cdb00 0x7faf940c8a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf84009870 tx=0x7faf8402a300 comp rx=0 tx=0).stop 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 -- 172.21.15.2:0/1922619470 shutdown_connections 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/1922619470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf940cdb00 0x7faf940c8a10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/1922619470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf94154d50 0x7faf9415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 -- 172.21.15.2:0/1922619470 >> 172.21.15.2:0/1922619470 conn(0x7faf9406d560 msgr2=0x7faf9406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:38.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 -- 172.21.15.2:0/1922619470 shutdown_connections 2024-09-19T05:42:38.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.810+0000 7faf9b3a6640 1 -- 172.21.15.2:0/1922619470 wait complete. 2024-09-19T05:42:38.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.811+0000 7faf9b3a6640 1 Processor -- start 2024-09-19T05:42:38.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.811+0000 7faf9b3a6640 1 -- start start 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.811+0000 7faf9b3a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf940cb6c0 0x7faf941f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf9b3a6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf940cdb00 0x7faf94067bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf9b3a6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf94154d50 0x7faf9414d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf9b3a6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf940cd020 con 0x7faf94154d50 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf9b3a6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faf940ccd40 con 0x7faf940cb6c0 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf9b3a6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7faf940cd300 con 0x7faf940cdb00 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf94154d50 0x7faf9414d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf94154d50 0x7faf9414d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37508/0 (socket says 172.21.15.2:37508) 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 -- 172.21.15.2:0/2508277298 learned_addr learned my addr 172.21.15.2:0/2508277298 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99142640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf940cb6c0 0x7faf941f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf98941640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf940cdb00 0x7faf94067bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 -- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf940cdb00 msgr2=0x7faf94067bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf940cdb00 0x7faf94067bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 -- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf940cb6c0 msgr2=0x7faf941f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf940cb6c0 0x7faf941f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 -- 172.21.15.2:0/2508277298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9414bdc0 con 0x7faf94154d50 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf99943640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf94154d50 0x7faf9414d930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c7d0 tx=0x7faf8c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf89ffb640 1 -- 172.21.15.2:0/2508277298 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c003b70 con 0x7faf94154d50 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf89ffb640 1 -- 172.21.15.2:0/2508277298 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf8c003d30 con 0x7faf94154d50 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf89ffb640 1 -- 172.21.15.2:0/2508277298 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00aa90 con 0x7faf94154d50 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.812+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9414c0a0 con 0x7faf94154d50 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.813+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9414c660 con 0x7faf94154d50 2024-09-19T05:42:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.814+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf940cbb70 con 0x7faf94154d50 2024-09-19T05:42:38.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.814+0000 7faf89ffb640 1 -- 172.21.15.2:0/2508277298 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faf8c00ac30 con 0x7faf94154d50 2024-09-19T05:42:38.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.814+0000 7faf89ffb640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf7806c2b0 0x7faf7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:38.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.814+0000 7faf89ffb640 1 -- 172.21.15.2:0/2508277298 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faf8c08b240 con 0x7faf94154d50 2024-09-19T05:42:38.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.815+0000 7faf99142640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf7806c2b0 0x7faf7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:38.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.815+0000 7faf99142640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf7806c2b0 0x7faf7806e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf900079a0 tx=0x7faf90008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.819+0000 7faf89ffb640 1 -- 172.21.15.2:0/2508277298 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faf8c059360 con 0x7faf94154d50 2024-09-19T05:42:38.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.994+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf940c8d10 con 0x7faf94154d50 2024-09-19T05:42:38.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.995+0000 7faf89ffb640 1 -- 172.21.15.2:0/2508277298 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16621 (secure 0 0 0) 0x7faf8c05d660 con 0x7faf94154d50 2024-09-19T05:42:38.997 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:38.998 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:33.917577+0000","last_up_change":"2024-09-19T05:42:20.842106+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:38.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf7806c2b0 msgr2=0x7faf7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:38.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf7806c2b0 0x7faf7806e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf900079a0 tx=0x7faf90008040 comp rx=0 tx=0).stop 2024-09-19T05:42:38.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf94154d50 msgr2=0x7faf9414d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf94154d50 0x7faf9414d930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c7d0 tx=0x7faf8c00cc90 comp rx=0 tx=0).stop 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 shutdown_connections 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf7806c2b0 0x7faf7806e770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf94154d50 0x7faf9414d930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf940cdb00 0x7faf94067bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 --2- 172.21.15.2:0/2508277298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf940cb6c0 0x7faf941f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 >> 172.21.15.2:0/2508277298 conn(0x7faf9406d560 msgr2=0x7faf94152670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 shutdown_connections 2024-09-19T05:42:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:38.997+0000 7faf9b3a6640 1 -- 172.21.15.2:0/2508277298 wait complete. 2024-09-19T05:42:39.010 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T05:42:39.010 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:39.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.110+0000 7f201623c640 1 Processor -- start 2024-09-19T05:42:39.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.110+0000 7f201623c640 1 -- start start 2024-09-19T05:42:39.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f201623c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f20100d06e0 0x7f20100cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f201623c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f201015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f201623c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2010154af0 0x7f2010154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f201623c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f201015f090 con 0x7f20100d06e0 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f201623c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f20100cb400 con 0x7f2010154af0 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f201623c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f20100cb6e0 con 0x7f20100d0ac0 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f20100d06e0 0x7f20100cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f20100d06e0 0x7f20100cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37518/0 (socket says 172.21.15.2:37518) 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 -- 172.21.15.2:0/3503054020 learned_addr learned my addr 172.21.15.2:0/3503054020 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200effd640 1 --2- 172.21.15.2:0/3503054020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f201015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200ffff640 1 --2- 172.21.15.2:0/3503054020 >> v2:172.21.15.123:3300/0 conn(0x7f2010154af0 0x7f2010154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 -- 172.21.15.2:0/3503054020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 msgr2=0x7f201015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 --2- 172.21.15.2:0/3503054020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f201015eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 -- 172.21.15.2:0/3503054020 >> v2:172.21.15.123:3300/0 conn(0x7f2010154af0 msgr2=0x7f2010154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 --2- 172.21.15.2:0/3503054020 >> v2:172.21.15.123:3300/0 conn(0x7f2010154af0 0x7f2010154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.111+0000 7f200f7fe640 1 -- 172.21.15.2:0/3503054020 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20100cc970 con 0x7f20100d06e0 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.112+0000 7f200f7fe640 1 --2- 172.21.15.2:0/3503054020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20100d06e0 0x7f20100cbb40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20100caac0 tx=0x7f200002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb10023163c35797 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.112+0000 7f200e7fc640 1 -- 172.21.15.2:0/3503054020 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000037070 con 0x7f20100d06e0 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.112+0000 7f200e7fc640 1 -- 172.21.15.2:0/3503054020 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2000032440 con 0x7f20100d06e0 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.112+0000 7f200e7fc640 1 -- 172.21.15.2:0/3503054020 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000032810 con 0x7f20100d06e0 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 -- 172.21.15.2:0/3503054020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20100d06e0 msgr2=0x7f20100cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 --2- 172.21.15.2:0/3503054020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20100d06e0 0x7f20100cbb40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f20100caac0 tx=0x7f200002a1e0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 -- 172.21.15.2:0/3503054020 shutdown_connections 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 --2- 172.21.15.2:0/3503054020 >> v2:172.21.15.123:3300/0 conn(0x7f2010154af0 0x7f2010154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 --2- 172.21.15.2:0/3503054020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f201015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 --2- 172.21.15.2:0/3503054020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20100d06e0 0x7f20100cbb40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 -- 172.21.15.2:0/3503054020 >> 172.21.15.2:0/3503054020 conn(0x7f201006d740 msgr2=0x7f2010073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 -- 172.21.15.2:0/3503054020 shutdown_connections 2024-09-19T05:42:39.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.113+0000 7f201623c640 1 -- 172.21.15.2:0/3503054020 wait complete. 2024-09-19T05:42:39.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.114+0000 7f201623c640 1 Processor -- start 2024-09-19T05:42:39.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.114+0000 7f201623c640 1 -- start start 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f201623c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20100d06e0 0x7f20101f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f201623c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f20101f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f201623c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2010154af0 0x7f20100b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f201623c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f20100c5830 con 0x7f2010154af0 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f201623c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f20100ccf30 con 0x7f20100d06e0 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f201623c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f20100cd450 con 0x7f20100d0ac0 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2010154af0 0x7f20100b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2010154af0 0x7f20100b5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37524/0 (socket says 172.21.15.2:37524) 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 -- 172.21.15.2:0/390606670 learned_addr learned my addr 172.21.15.2:0/390606670 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200f7fe640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20100d06e0 0x7f20101f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200effd640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f20101f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 -- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 msgr2=0x7f20101f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f20101f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 -- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20100d06e0 msgr2=0x7f20101f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20100d06e0 0x7f20101f12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 -- 172.21.15.2:0/390606670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101e6df0 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f200ffff640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2010154af0 0x7f20100b5a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f200800cc60 tx=0x7f200800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.115+0000 7f1ff7fff640 1 -- 172.21.15.2:0/390606670 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008003990 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.116+0000 7f1ff7fff640 1 -- 172.21.15.2:0/390606670 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2008003b30 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.116+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20101e6830 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.116+0000 7f1ff7fff640 1 -- 172.21.15.2:0/390606670 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008004020 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.116+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101f07c0 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.117+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20100cbb40 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.117+0000 7f1ff7fff640 1 -- 172.21.15.2:0/390606670 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f20080041c0 con 0x7f2010154af0 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.117+0000 7f1ff7fff640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ffc06c2e0 0x7f1ffc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.117+0000 7f1ff7fff640 1 -- 172.21.15.2:0/390606670 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f200808b790 con 0x7f2010154af0 2024-09-19T05:42:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.118+0000 7f200f7fe640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ffc06c2e0 0x7f1ffc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.118+0000 7f200f7fe640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ffc06c2e0 0x7f1ffc06e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2000034000 tx=0x7f200002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.122+0000 7f1ff7fff640 1 -- 172.21.15.2:0/390606670 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2008014030 con 0x7f2010154af0 2024-09-19T05:42:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.301+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20100cdb10 con 0x7f2010154af0 2024-09-19T05:42:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.302+0000 7f1ff7fff640 1 -- 172.21.15.2:0/390606670 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16621 (secure 0 0 0) 0x7f20080598b0 con 0x7f2010154af0 2024-09-19T05:42:39.304 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:39.305 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:33.917577+0000","last_up_change":"2024-09-19T05:42:20.842106+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ffc06c2e0 msgr2=0x7f1ffc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ffc06c2e0 0x7f1ffc06e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2000034000 tx=0x7f200002e000 comp rx=0 tx=0).stop 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2010154af0 msgr2=0x7f20100b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2010154af0 0x7f20100b5a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f200800cc60 tx=0x7f200800a5c0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 shutdown_connections 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ffc06c2e0 0x7f1ffc06e7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2010154af0 0x7f20100b5a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f20100d0ac0 0x7f20101f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 --2- 172.21.15.2:0/390606670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f20100d06e0 0x7f20101f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 >> 172.21.15.2:0/390606670 conn(0x7f201006d740 msgr2=0x7f201015e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 shutdown_connections 2024-09-19T05:42:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.304+0000 7f201623c640 1 -- 172.21.15.2:0/390606670 wait complete. 2024-09-19T05:42:39.318 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T05:42:39.318 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T05:42:39.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T05:42:39.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T05:42:39.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-19T05:42:39.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-19T05:42:39.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-19T05:42:39.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-19T05:42:39.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.492+0000 7f605f59e640 1 Processor -- start 2024-09-19T05:42:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.492+0000 7f605f59e640 1 -- start start 2024-09-19T05:42:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.492+0000 7f7d42a82640 1 Processor -- start 2024-09-19T05:42:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.493+0000 7f7d42a82640 1 -- start start 2024-09-19T05:42:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.493+0000 7f18cb075640 1 Processor -- start 2024-09-19T05:42:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.493+0000 7f56a06e4640 1 Processor -- start 2024-09-19T05:42:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.493+0000 7f18cb075640 1 -- start start 2024-09-19T05:42:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f605f59e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6060151650 0x7f60600c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f605f59e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60600c7720 con 0x7f6060157490 2024-09-19T05:42:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f605f59e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60600c7cf0 con 0x7f60600c47f0 2024-09-19T05:42:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f605f59e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f60600c7fb0 con 0x7f6060151650 2024-09-19T05:42:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f7d42a82640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d3c159100 0x7f7d3c06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f7d42a82640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7d3c15ef50 0x7f7d3c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f7d42a82640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3c1594e0 con 0x7f7d3c0719f0 2024-09-19T05:42:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f7d42a82640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c0c8080 con 0x7f7d3c15ef50 2024-09-19T05:42:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f7d42a82640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c15f8e0 con 0x7f7d3c159100 2024-09-19T05:42:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.494+0000 7f56a06e4640 1 -- start start 2024-09-19T05:42:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d41a80640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d3c159100 0x7f7d3c06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d41a80640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d3c159100 0x7f7d3c06ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43940/0 (socket says 172.21.15.2:43940) 2024-09-19T05:42:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d41a80640 1 -- 172.21.15.2:0/3225887122 learned_addr learned my addr 172.21.15.2:0/3225887122 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f605dd9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6060157490 0x7f606006f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51528/0 (socket says 172.21.15.2:51528) 2024-09-19T05:42:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d42281640 1 --2- 172.21.15.2:0/3225887122 >> v2:172.21.15.2:3300/0 conn(0x7f7d3c15ef50 0x7f7d3c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f605dd9b640 1 -- 172.21.15.2:0/2160617491 learned_addr learned my addr 172.21.15.2:0/2160617491 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f605e59c640 1 --2- 172.21.15.2:0/2160617491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6060151650 0x7f60600c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f18cb075640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04aac0 0x7f18bc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f18cb075640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18bc04c360 0x7f18bc006f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f18cb075640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18bc04b410 con 0x7f18bc0dcd30 2024-09-19T05:42:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f18cb075640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0e6bc0 con 0x7f18bc04c360 2024-09-19T05:42:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f18cb075640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0e6ea0 con 0x7f18bc04aac0 2024-09-19T05:42:39.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f56a06e4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5698150d20 0x7f56980c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f56a06e4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f569806ee70 0x7f569806f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f56a06e4640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569815a130 con 0x7f5698072650 2024-09-19T05:42:39.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f56a06e4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f56980c6990 con 0x7f5698150d20 2024-09-19T05:42:39.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f56a06e4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f569815a3a0 con 0x7f569806ee70 2024-09-19T05:42:39.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f18ca874640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18bc04c360 0x7f18bc006f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f18c9872640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04aac0 0x7f18bc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f18ca874640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18bc04c360 0x7f18bc006f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:43956/0 (socket says 172.21.15.2:43956) 2024-09-19T05:42:39.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f18c9872640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04aac0 0x7f18bc006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39102/0 (socket says 172.21.15.2:39102) 2024-09-19T05:42:39.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f18ca874640 1 -- 172.21.15.2:0/1096988832 learned_addr learned my addr 172.21.15.2:0/1096988832 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f569dc7f640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5698072650 0x7f5698071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51534/0 (socket says 172.21.15.2:51534) 2024-09-19T05:42:39.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f569dc7f640 1 -- 172.21.15.2:0/3395658166 learned_addr learned my addr 172.21.15.2:0/3395658166 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f569e480640 1 --2- 172.21.15.2:0/3395658166 >> v2:172.21.15.123:3300/0 conn(0x7f5698150d20 0x7f56980c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f18ca874640 1 -- 172.21.15.2:0/1096988832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04aac0 msgr2=0x7f18bc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f18ca874640 1 --2- 172.21.15.2:0/1096988832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04aac0 0x7f18bc006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f18ca874640 1 -- 172.21.15.2:0/1096988832 >> v1:172.21.15.123:6789/0 conn(0x7f18bc0dcd30 legacy=0x7f18bc04b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f18ca874640 1 -- 172.21.15.2:0/1096988832 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc005f30 con 0x7f18bc04c360 2024-09-19T05:42:39.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d41a80640 1 -- 172.21.15.2:0/3225887122 >> v2:172.21.15.2:3300/0 conn(0x7f7d3c15ef50 msgr2=0x7f7d3c15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d41a80640 1 --2- 172.21.15.2:0/3225887122 >> v2:172.21.15.2:3300/0 conn(0x7f7d3c15ef50 0x7f7d3c15f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d41a80640 1 -- 172.21.15.2:0/3225887122 >> v1:172.21.15.123:6789/0 conn(0x7f7d3c0719f0 legacy=0x7f7d3c072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.495+0000 7f7d41a80640 1 -- 172.21.15.2:0/3225887122 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c0c7470 con 0x7f7d3c159100 2024-09-19T05:42:39.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f569ec81640 1 --2- 172.21.15.2:0/3395658166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f569806ee70 0x7f569806f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f4ae2182640 1 Processor -- start 2024-09-19T05:42:39.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f4a1fa37640 1 Processor -- start 2024-09-19T05:42:39.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f7d41a80640 1 --2- 172.21.15.2:0/3225887122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c159100 0x7f7d3c06ff70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c071660 tx=0x7f7d3402a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6049c7ad96aa264 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f18ca874640 1 --2- 172.21.15.2:0/1096988832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc04c360 0x7f18bc006f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009870 tx=0x7f18c002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f09611cc5695177 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f569ec81640 1 -- 172.21.15.2:0/3395658166 >> v2:172.21.15.123:3300/0 conn(0x7f5698150d20 msgr2=0x7f56980c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f569ec81640 1 --2- 172.21.15.2:0/3395658166 >> v2:172.21.15.123:3300/0 conn(0x7f5698150d20 0x7f56980c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f569d47e640 1 -- 172.21.15.2:0/3395658166 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1811970909 0 0) 0x7f569815a130 con 0x7f5698072650 2024-09-19T05:42:39.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f569ec81640 1 -- 172.21.15.2:0/3395658166 >> v1:172.21.15.2:6789/0 conn(0x7f5698072650 legacy=0x7f5698071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f569ec81640 1 -- 172.21.15.2:0/3395658166 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569815a680 con 0x7f569806ee70 2024-09-19T05:42:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f4a1fa37640 1 -- start start 2024-09-19T05:42:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f4ae2182640 1 -- start start 2024-09-19T05:42:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f569e480640 1 --2- 172.21.15.2:0/3395658166 >> v2:172.21.15.123:3300/0 conn(0x7f5698150d20 0x7f56980c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f18c9071640 1 -- 172.21.15.2:0/1096988832 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0037070 con 0x7f18bc04c360 2024-09-19T05:42:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f605e59c640 1 -- 172.21.15.2:0/2160617491 >> v1:172.21.15.123:6789/0 conn(0x7f60600c47f0 legacy=0x7f60600c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f605e59c640 1 -- 172.21.15.2:0/2160617491 >> v1:172.21.15.2:6789/0 conn(0x7f6060157490 legacy=0x7f606006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f18c9071640 1 -- 172.21.15.2:0/1096988832 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c0032440 con 0x7f18bc04c360 2024-09-19T05:42:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.496+0000 7f605e59c640 1 -- 172.21.15.2:0/2160617491 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60600c8290 con 0x7f6060151650 2024-09-19T05:42:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f605e59c640 1 --2- 172.21.15.2:0/2160617491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6060151650 0x7f60600c71e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f605000d7e0 tx=0x7f605002e900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02b3412a825da74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.497+0000 7f18c9071640 1 -- 172.21.15.2:0/1096988832 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0032810 con 0x7f18bc04c360 2024-09-19T05:42:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f7d40a7e640 1 -- 172.21.15.2:0/3225887122 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34036070 con 0x7f7d3c159100 2024-09-19T05:42:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f7d40a7e640 1 -- 172.21.15.2:0/3225887122 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d34031600 con 0x7f7d3c159100 2024-09-19T05:42:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f7d40a7e640 1 -- 172.21.15.2:0/3225887122 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d340319d0 con 0x7f7d3c159100 2024-09-19T05:42:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f605d59a640 1 -- 172.21.15.2:0/2160617491 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500347c0 con 0x7f6060151650 2024-09-19T05:42:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f605d59a640 1 -- 172.21.15.2:0/2160617491 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605003a070 con 0x7f6060151650 2024-09-19T05:42:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f605d59a640 1 -- 172.21.15.2:0/2160617491 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050034b80 con 0x7f6060151650 2024-09-19T05:42:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f18cb075640 1 -- 172.21.15.2:0/1096988832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc04c360 msgr2=0x7f18bc006f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f18cb075640 1 --2- 172.21.15.2:0/1096988832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc04c360 0x7f18bc006f50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009870 tx=0x7f18c002a210 comp rx=0 tx=0).stop 2024-09-19T05:42:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4a1fa37640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180cd200 0x7f4a180c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4a1fa37640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4a180cd5e0 0x7f4a1815b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4a1fa37640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a180c5c90 con 0x7f4a180b3220 2024-09-19T05:42:39.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4a1fa37640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1815e030 con 0x7f4a180cd5e0 2024-09-19T05:42:39.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4a1fa37640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1815e310 con 0x7f4a180cd200 2024-09-19T05:42:39.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f569ec81640 1 --2- 172.21.15.2:0/3395658166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f569806ee70 0x7f569806f290 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f56900097a0 tx=0x7f569002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc163e5b00916da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4ae2182640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4ad40d8a80 0x7f4ad4048fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1d7d3640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180cd200 0x7f4a180c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1d7d3640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180cd200 0x7f4a180c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39130/0 (socket says 172.21.15.2:39130) 2024-09-19T05:42:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/1213953307 learned_addr learned my addr 172.21.15.2:0/1213953307 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f18cb075640 1 -- 172.21.15.2:0/1096988832 shutdown_connections 2024-09-19T05:42:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f18cb075640 1 --2- 172.21.15.2:0/1096988832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc04c360 0x7f18bc006f50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f18cb075640 1 --2- 172.21.15.2:0/1096988832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04aac0 0x7f18bc006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f18cb075640 1 -- 172.21.15.2:0/1096988832 >> 172.21.15.2:0/1096988832 conn(0x7f18bc050090 msgr2=0x7f18bc0504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f7d42a82640 1 -- 172.21.15.2:0/3225887122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c159100 msgr2=0x7f7d3c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f7d42a82640 1 --2- 172.21.15.2:0/3225887122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c159100 0x7f7d3c06ff70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c071660 tx=0x7f7d3402a640 comp rx=0 tx=0).stop 2024-09-19T05:42:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4ae2182640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4ae2182640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad40502d0 con 0x7f4ad404f160 2024-09-19T05:42:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4ae2182640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad4054f70 con 0x7f4ad40d8a80 2024-09-19T05:42:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.498+0000 7f4ae2182640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad4050540 con 0x7f4ad4007370 2024-09-19T05:42:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f18cb075640 1 -- 172.21.15.2:0/1096988832 shutdown_connections 2024-09-19T05:42:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae1180640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4ad40d8a80 0x7f4ad4048fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae1180640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4ad40d8a80 0x7f4ad4048fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37538/0 (socket says 172.21.15.2:37538) 2024-09-19T05:42:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae1180640 1 -- 172.21.15.2:0/1284255626 learned_addr learned my addr 172.21.15.2:0/1284255626 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae097f640 1 --2- 172.21.15.2:0/1284255626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f18cb075640 1 -- 172.21.15.2:0/1096988832 wait complete. 2024-09-19T05:42:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1dfd4640 1 --2- 172.21.15.2:0/1213953307 >> v2:172.21.15.2:3300/0 conn(0x7f4a180cd5e0 0x7f4a1815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/1213953307 >> v2:172.21.15.2:3300/0 conn(0x7f4a180cd5e0 msgr2=0x7f4a1815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1d7d3640 1 --2- 172.21.15.2:0/1213953307 >> v2:172.21.15.2:3300/0 conn(0x7f4a180cd5e0 0x7f4a1815b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/1213953307 >> v1:172.21.15.123:6789/0 conn(0x7f4a180b3220 legacy=0x7f4a18154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f605f59e640 1 -- 172.21.15.2:0/2160617491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6060151650 msgr2=0x7f60600c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/1213953307 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1815e5f0 con 0x7f4a180cd200 2024-09-19T05:42:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f7d42a82640 1 -- 172.21.15.2:0/3225887122 shutdown_connections 2024-09-19T05:42:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f7d42a82640 1 --2- 172.21.15.2:0/3225887122 >> v2:172.21.15.2:3300/0 conn(0x7f7d3c15ef50 0x7f7d3c15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f7d42a82640 1 --2- 172.21.15.2:0/3225887122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c159100 0x7f7d3c06ff70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f7d42a82640 1 -- 172.21.15.2:0/3225887122 >> 172.21.15.2:0/3225887122 conn(0x7f7d3c06bc10 msgr2=0x7f7d3c06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f605f59e640 1 --2- 172.21.15.2:0/2160617491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6060151650 0x7f60600c71e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f605000d7e0 tx=0x7f605002e900 comp rx=0 tx=0).stop 2024-09-19T05:42:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f569d47e640 1 -- 172.21.15.2:0/3395658166 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690037070 con 0x7f569806ee70 2024-09-19T05:42:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f569d47e640 1 -- 172.21.15.2:0/3395658166 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569002ae40 con 0x7f569806ee70 2024-09-19T05:42:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f569d47e640 1 -- 172.21.15.2:0/3395658166 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690032660 con 0x7f569806ee70 2024-09-19T05:42:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae1180640 1 -- 172.21.15.2:0/1284255626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 msgr2=0x7f4ad40e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae1180640 1 --2- 172.21.15.2:0/1284255626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae1180640 1 -- 172.21.15.2:0/1284255626 >> v1:172.21.15.2:6789/0 conn(0x7f4ad404f160 legacy=0x7f4ad40e20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.499+0000 7f4ae1180640 1 -- 172.21.15.2:0/1284255626 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad4006890 con 0x7f4ad40d8a80 2024-09-19T05:42:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f605f59e640 1 -- 172.21.15.2:0/2160617491 shutdown_connections 2024-09-19T05:42:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f605f59e640 1 --2- 172.21.15.2:0/2160617491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6060151650 0x7f60600c71e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f605f59e640 1 -- 172.21.15.2:0/2160617491 >> 172.21.15.2:0/2160617491 conn(0x7f6060070520 msgr2=0x7f6060070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7efcb7dc4640 1 Processor -- start 2024-09-19T05:42:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f605f59e640 1 -- 172.21.15.2:0/2160617491 shutdown_connections 2024-09-19T05:42:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f605f59e640 1 -- 172.21.15.2:0/2160617491 wait complete. 2024-09-19T05:42:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f4ae097f640 1 --2- 172.21.15.2:0/1284255626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f2fbe21e640 1 Processor -- start 2024-09-19T05:42:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f7d42a82640 1 -- 172.21.15.2:0/3225887122 shutdown_connections 2024-09-19T05:42:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f56a06e4640 1 -- 172.21.15.2:0/3395658166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f569806ee70 msgr2=0x7f569806f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f56a06e4640 1 --2- 172.21.15.2:0/3395658166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f569806ee70 0x7f569806f290 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f56900097a0 tx=0x7f569002a0f0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f18cb075640 1 Processor -- start 2024-09-19T05:42:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f18cb075640 1 -- start start 2024-09-19T05:42:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7f2fbe21e640 1 -- start start 2024-09-19T05:42:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f4ae1180640 1 --2- 172.21.15.2:0/1284255626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad40d8a80 0x7f4ad4048fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000b60 tx=0x7f4adc0a6b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd699f364d279f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f4a1d7d3640 1 --2- 172.21.15.2:0/1213953307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180cd200 0x7f4a180c5670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009870 tx=0x7f4a0802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=795a7359dd2e2abd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.500+0000 7efcb7dc4640 1 -- start start 2024-09-19T05:42:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f4ad3fff640 1 -- 172.21.15.2:0/1284255626 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0a8960 con 0x7f4ad40d8a80 2024-09-19T05:42:39.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f4a0ffff640 1 -- 172.21.15.2:0/1213953307 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08037070 con 0x7f4a180cd200 2024-09-19T05:42:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f4ad3fff640 1 -- 172.21.15.2:0/1284255626 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc0a8b00 con 0x7f4ad40d8a80 2024-09-19T05:42:39.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f4a0ffff640 1 -- 172.21.15.2:0/1213953307 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08032440 con 0x7f4a180cd200 2024-09-19T05:42:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18cb075640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc04aac0 0x7f18bc1786b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18cb075640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04c360 0x7f18bc17c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18cb075640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc0dcd30 0x7f18bc04cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18cb075640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0330c0 con 0x7f18bc04aac0 2024-09-19T05:42:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18cb075640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc04b100 con 0x7f18bc0dcd30 2024-09-19T05:42:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18cb075640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0dd110 con 0x7f18bc04c360 2024-09-19T05:42:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04c360 0x7f18bc17c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7efcb7dc4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efcb0158de0 0x7efcb0159230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7efcb7dc4640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb00c98b0 con 0x7efcb0162790 2024-09-19T05:42:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7efcb7dc4640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb00c9b90 con 0x7efcb0072ae0 2024-09-19T05:42:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7efcb7dc4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efcb00c47f0 con 0x7efcb0158de0 2024-09-19T05:42:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f2fbe21e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2fb80c9130 0x7f2fb8071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f605f59e640 1 Processor -- start 2024-09-19T05:42:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04c360 0x7f18bc17c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39144/0 (socket says 172.21.15.2:39144) 2024-09-19T05:42:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 -- 172.21.15.2:0/722633689 learned_addr learned my addr 172.21.15.2:0/722633689 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18ca874640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc0dcd30 0x7f18bc04cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7efcb6361640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efcb0072ae0 0x7efcb0072f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:42964/0 (socket says 172.21.15.2:42964) 2024-09-19T05:42:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7efcb6361640 1 -- 172.21.15.2:0/3703815028 learned_addr learned my addr 172.21.15.2:0/3703815028 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18ca073640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc04aac0 0x7f18bc1786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f2fbe21e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2fb80c87a0 0x7f2fb806fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f605f59e640 1 -- start start 2024-09-19T05:42:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 -- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc0dcd30 msgr2=0x7f18bc04cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.501+0000 7f7d42a82640 1 -- 172.21.15.2:0/3225887122 wait complete. 2024-09-19T05:42:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f2fbe21e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb8157650 con 0x7f2fb80c9510 2024-09-19T05:42:39.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f7d42a82640 1 Processor -- start 2024-09-19T05:42:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18bc0dcd30 0x7f18bc04cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 -- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc04aac0 msgr2=0x7f18bc1786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc04aac0 0x7f18bc1786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f18c9872640 1 -- 172.21.15.2:0/722633689 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc178d60 con 0x7f18bc04c360 2024-09-19T05:42:39.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fb7fff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2fb80c87a0 0x7f2fb806fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fb7fff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2fb80c87a0 0x7f2fb806fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37570/0 (socket says 172.21.15.2:37570) 2024-09-19T05:42:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fb7fff640 1 -- 172.21.15.2:0/2295170778 learned_addr learned my addr 172.21.15.2:0/2295170778 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fbe21e640 1 -- 172.21.15.2:0/2295170778 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb806ee70 con 0x7f2fb80c87a0 2024-09-19T05:42:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f18ca073640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18bc04aac0 0x7f18bc1786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:39.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fbe21e640 1 -- 172.21.15.2:0/2295170778 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb8157930 con 0x7f2fb80c9130 2024-09-19T05:42:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4ae2182640 1 -- 172.21.15.2:0/1284255626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad40d8a80 msgr2=0x7f4ad4048fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4ae2182640 1 --2- 172.21.15.2:0/1284255626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad40d8a80 0x7f4ad4048fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000b60 tx=0x7f4adc0a6b40 comp rx=0 tx=0).stop 2024-09-19T05:42:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7efcb535f640 1 --2- 172.21.15.2:0/3703815028 >> v2:172.21.15.123:3300/0 conn(0x7efcb0158de0 0x7efcb0159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f4a1fa37640 1 -- 172.21.15.2:0/1213953307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180cd200 msgr2=0x7f4a180c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.502+0000 7f4a1fa37640 1 --2- 172.21.15.2:0/1213953307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180cd200 0x7f4a180c5670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009870 tx=0x7f4a0802a210 comp rx=0 tx=0).stop 2024-09-19T05:42:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4a1fa37640 1 -- 172.21.15.2:0/1213953307 shutdown_connections 2024-09-19T05:42:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4a1fa37640 1 --2- 172.21.15.2:0/1213953307 >> v2:172.21.15.2:3300/0 conn(0x7f4a180cd5e0 0x7f4a1815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4a1fa37640 1 --2- 172.21.15.2:0/1213953307 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180cd200 0x7f4a180c5670 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009870 tx=0x7f4a0802a210 comp rx=0 tx=0).stop 2024-09-19T05:42:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4a1fa37640 1 -- 172.21.15.2:0/1213953307 >> 172.21.15.2:0/1213953307 conn(0x7f4a18067950 msgr2=0x7f4a18067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f56a06e4640 1 -- 172.21.15.2:0/3395658166 shutdown_connections 2024-09-19T05:42:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f56a06e4640 1 --2- 172.21.15.2:0/3395658166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f569806ee70 0x7f569806f290 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f56a06e4640 1 --2- 172.21.15.2:0/3395658166 >> v2:172.21.15.123:3300/0 conn(0x7f5698150d20 0x7f56980c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4a1fa37640 1 -- 172.21.15.2:0/1213953307 shutdown_connections 2024-09-19T05:42:39.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f56a06e4640 1 -- 172.21.15.2:0/3395658166 >> 172.21.15.2:0/3395658166 conn(0x7f569806bc10 msgr2=0x7f569806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f56a06e4640 1 -- 172.21.15.2:0/3395658166 shutdown_connections 2024-09-19T05:42:39.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f56a06e4640 1 -- 172.21.15.2:0/3395658166 wait complete. 2024-09-19T05:42:39.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f18c9872640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04c360 0x7f18bc17c550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e8a0 tx=0x7f18b800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fb7fff640 1 -- 172.21.15.2:0/2295170778 >> v2:172.21.15.123:3300/0 conn(0x7f2fb80c9130 msgr2=0x7f2fb8071fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:42:39.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fb7fff640 1 --2- 172.21.15.2:0/2295170778 >> v2:172.21.15.123:3300/0 conn(0x7f2fb80c9130 0x7f2fb8071fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fb7fff640 1 -- 172.21.15.2:0/2295170778 >> v1:172.21.15.123:6789/0 conn(0x7f2fb80c9510 legacy=0x7f2fb8156b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:42:39.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f2fb7fff640 1 -- 172.21.15.2:0/2295170778 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb806f0c0 con 0x7f2fb80c87a0 2024-09-19T05:42:39.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f7d42a82640 1 -- start start 2024-09-19T05:42:39.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f605f59e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60600c47f0 0x7f6060148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605f59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6060151650 0x7f60601459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f4ae2182640 1 -- 172.21.15.2:0/1284255626 shutdown_connections 2024-09-19T05:42:39.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f4ae2182640 1 --2- 172.21.15.2:0/1284255626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f4ae2182640 1 --2- 172.21.15.2:0/1284255626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad40d8a80 0x7f4ad4048fd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f4ae2182640 1 -- 172.21.15.2:0/1284255626 >> 172.21.15.2:0/1284255626 conn(0x7f4ad4054050 msgr2=0x7f4ad4054460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605f59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6060157490 0x7f6060145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605f59e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f606006efd0 con 0x7f6060151650 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605f59e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f60600c8850 con 0x7f6060157490 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605f59e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f60600c6990 con 0x7f60600c47f0 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605e59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60600c47f0 0x7f6060148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8017070 con 0x7f18bc04c360 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b800c940 con 0x7f18bc04c360 2024-09-19T05:42:39.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b800cd30 con 0x7f18bc04c360 2024-09-19T05:42:39.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6060157490 0x7f6060145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f7d42a82640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c0719f0 0x7f7d3c1fdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f7d42a82640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d3c159100 0x7f7d3c0c0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6060157490 0x7f6060145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44012/0 (socket says 172.21.15.2:44012) 2024-09-19T05:42:39.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f7d42a82640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d3c15ef50 0x7f7d3c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f7d42a82640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0c7d10 con 0x7f7d3c159100 2024-09-19T05:42:39.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f7d42a82640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0c7a30 con 0x7f7d3c0719f0 2024-09-19T05:42:39.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f7d42a82640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c073490 con 0x7f7d3c15ef50 2024-09-19T05:42:39.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f605ed9d640 1 -- 172.21.15.2:0/344592407 learned_addr learned my addr 172.21.15.2:0/344592407 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc179040 con 0x7f18bc04c360 2024-09-19T05:42:39.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc17bd50 con 0x7f18bc04c360 2024-09-19T05:42:39.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f4ae2182640 1 -- 172.21.15.2:0/1284255626 shutdown_connections 2024-09-19T05:42:39.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d42281640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d3c15ef50 0x7f7d3c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.504+0000 7f2fb7fff640 1 --2- 172.21.15.2:0/2295170778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c87a0 0x7f2fb806fba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80cd510 tx=0x7f2fb002bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9961bf155b0ba597 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d41a80640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c0719f0 0x7f7d3c1fdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d3c159100 0x7f7d3c0c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d3c159100 0x7f7d3c0c0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37588/0 (socket says 172.21.15.2:37588) 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 -- 172.21.15.2:0/2576818753 learned_addr learned my addr 172.21.15.2:0/2576818753 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f605dd9b640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6060151650 0x7f60601459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f605ed9d640 1 -- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60600c47f0 msgr2=0x7f6060148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f605ed9d640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f60600c47f0 0x7f6060148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f605ed9d640 1 -- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6060151650 msgr2=0x7f60601459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f56a06e4640 1 Processor -- start 2024-09-19T05:42:39.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18bc04b800 con 0x7f18bc04c360 2024-09-19T05:42:39.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f4ae2182640 1 -- 172.21.15.2:0/1284255626 wait complete. 2024-09-19T05:42:39.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7f4a1fa37640 1 -- 172.21.15.2:0/1213953307 wait complete. 2024-09-19T05:42:39.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f4a1fa37640 1 Processor -- start 2024-09-19T05:42:39.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f4a1fa37640 1 -- start start 2024-09-19T05:42:39.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7efcb535f640 1 -- 172.21.15.2:0/3703815028 >> v1:172.21.15.123:6789/0 conn(0x7efcb0072ae0 legacy=0x7efcb0072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7efcb535f640 1 -- 172.21.15.2:0/3703815028 >> v1:172.21.15.2:6789/0 conn(0x7efcb0162790 legacy=0x7efcb0162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.503+0000 7efcb535f640 1 -- 172.21.15.2:0/3703815028 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb00c9e70 con 0x7efcb0158de0 2024-09-19T05:42:39.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f605ed9d640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6060151650 0x7f60601459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f605ed9d640 1 -- 172.21.15.2:0/344592407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606014a0c0 con 0x7f6060157490 2024-09-19T05:42:39.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f605dd9b640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6060151650 0x7f60601459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:39.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7efcb535f640 1 --2- 172.21.15.2:0/3703815028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0158de0 0x7efcb0159230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efca00097a0 tx=0x7efca002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=662260bf1d736538 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 -- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d3c15ef50 msgr2=0x7f7d3c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d3c15ef50 0x7f7d3c0cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 -- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c0719f0 msgr2=0x7f7d3c1fdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d3c0719f0 0x7f7d3c1fdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f7d4127f640 1 -- 172.21.15.2:0/2576818753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c1e7910 con 0x7f7d3c159100 2024-09-19T05:42:39.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f56a06e4640 1 -- start start 2024-09-19T05:42:39.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f2fb67fc640 1 -- 172.21.15.2:0/2295170778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb002c720 con 0x7f2fb80c87a0 2024-09-19T05:42:39.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f2fb67fc640 1 -- 172.21.15.2:0/2295170778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb002c8c0 con 0x7f2fb80c87a0 2024-09-19T05:42:39.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.505+0000 7f2fb67fc640 1 -- 172.21.15.2:0/2295170778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb002cc90 con 0x7f2fb80c87a0 2024-09-19T05:42:39.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7efcb4b5e640 1 -- 172.21.15.2:0/3703815028 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0037070 con 0x7efcb0158de0 2024-09-19T05:42:39.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7efcb4b5e640 1 -- 172.21.15.2:0/3703815028 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca002acd0 con 0x7efcb0158de0 2024-09-19T05:42:39.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f605ed9d640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6060157490 0x7f6060145ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f605800c730 tx=0x7f605800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f7d4127f640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d3c159100 0x7f7d3c0c0990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c780 tx=0x7f7d2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7efcb4b5e640 1 -- 172.21.15.2:0/3703815028 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0032660 con 0x7efcb0158de0 2024-09-19T05:42:39.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f56a06e4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f569806ee70 0x7f569815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f56a06e4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5698072650 0x7f56981ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f4a1fa37640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180b3220 0x7f4a181502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f56a06e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5698150d20 0x7f569814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058016020 con 0x7f6060157490 2024-09-19T05:42:39.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6058006d50 con 0x7f6060157490 2024-09-19T05:42:39.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058003bb0 con 0x7f6060157490 2024-09-19T05:42:39.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f56a06e4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f569806f7d0 con 0x7f5698150d20 2024-09-19T05:42:39.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f56a06e4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f569815ac40 con 0x7f569806ee70 2024-09-19T05:42:39.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f56a06e4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f56980c8cb0 con 0x7f5698072650 2024-09-19T05:42:39.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1fa37640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a180cd5e0 0x7f4a181585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c017070 con 0x7f7d3c159100 2024-09-19T05:42:39.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5698150d20 0x7f569814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5698150d20 0x7f569814a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37600/0 (socket says 172.21.15.2:37600) 2024-09-19T05:42:39.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 -- 172.21.15.2:0/213223002 learned_addr learned my addr 172.21.15.2:0/213223002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1fa37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181e5a80 0x7f4a1815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180b3220 0x7f4a181502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180b3220 0x7f4a181502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39184/0 (socket says 172.21.15.2:39184) 2024-09-19T05:42:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/856083875 learned_addr learned my addr 172.21.15.2:0/856083875 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f7d42a82640 1 -- 172.21.15.2:0/2576818753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c1e5f70 con 0x7f7d3c159100 2024-09-19T05:42:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f7d42a82640 1 -- 172.21.15.2:0/2576818753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c1e6530 con 0x7f7d3c159100 2024-09-19T05:42:39.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4a181550c0 con 0x7f4a180cd5e0 2024-09-19T05:42:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a18154f40 con 0x7f4a181e5a80 2024-09-19T05:42:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4a180c5f50 con 0x7f4a180b3220 2024-09-19T05:42:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3703815028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0158de0 msgr2=0x7efcb0159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7efcb7dc4640 1 --2- 172.21.15.2:0/3703815028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0158de0 0x7efcb0159230 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efca00097a0 tx=0x7efca002a040 comp rx=0 tx=0).stop 2024-09-19T05:42:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3703815028 shutdown_connections 2024-09-19T05:42:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7efcb7dc4640 1 --2- 172.21.15.2:0/3703815028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0158de0 0x7efcb0159230 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3703815028 >> 172.21.15.2:0/3703815028 conn(0x7efcb006bc10 msgr2=0x7efcb006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4ae2182640 1 Processor -- start 2024-09-19T05:42:39.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4ae2182640 1 -- start start 2024-09-19T05:42:39.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1dfd4640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181e5a80 0x7f4a1815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1cfd2640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a180cd5e0 0x7f4a181585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 -- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5698072650 msgr2=0x7f56981ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5698072650 0x7f56981ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 -- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f569806ee70 msgr2=0x7f569815b060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:42:39.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f569806ee70 0x7f569815b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f569ec81640 1 -- 172.21.15.2:0/213223002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569814b210 con 0x7f5698150d20 2024-09-19T05:42:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f2fbe21e640 1 -- 172.21.15.2:0/2295170778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c87a0 msgr2=0x7f2fb806fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.506+0000 7f2fbe21e640 1 --2- 172.21.15.2:0/2295170778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c87a0 0x7f2fb806fba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80cd510 tx=0x7f2fb002bc20 comp rx=0 tx=0).stop 2024-09-19T05:42:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60601ecd70 con 0x7f6060157490 2024-09-19T05:42:39.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60601451a0 con 0x7f6060157490 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f569ec81640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5698150d20 0x7f569814a170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f569002a6e0 tx=0x7f56900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181e5a80 msgr2=0x7f4a1815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181e5a80 0x7f4a1815d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/856083875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a180cd5e0 msgr2=0x7f4a181585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4a180cd5e0 0x7f4a181585e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1d7d3640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a181fd810 con 0x7f4a180b3220 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f4a1dfd4640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a181e5a80 0x7f4a1815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:39.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3703815028 shutdown_connections 2024-09-19T05:42:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3703815028 wait complete. 2024-09-19T05:42:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f2fbe21e640 1 -- 172.21.15.2:0/2295170778 shutdown_connections 2024-09-19T05:42:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f2fbe21e640 1 --2- 172.21.15.2:0/2295170778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c87a0 0x7f2fb806fba0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f2fbe21e640 1 --2- 172.21.15.2:0/2295170778 >> v2:172.21.15.123:3300/0 conn(0x7f2fb80c9130 0x7f2fb8071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f2fbe21e640 1 -- 172.21.15.2:0/2295170778 >> 172.21.15.2:0/2295170778 conn(0x7f2fb806bc10 msgr2=0x7f2fb806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f4ae2182640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.507+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0052d0 con 0x7f7d3c159100 2024-09-19T05:42:39.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00a860 con 0x7f7d3c159100 2024-09-19T05:42:39.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f7d42a82640 1 -- 172.21.15.2:0/2576818753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d24001030 con 0x7f7d3c159100 2024-09-19T05:42:39.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690041020 con 0x7f5698150d20 2024-09-19T05:42:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f4a1d7d3640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180b3220 0x7f4a181502b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4a0802a780 tx=0x7f4a080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae2182640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad404f160 0x7f4ad41777f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f18b801a070 con 0x7f18bc04c360 2024-09-19T05:42:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569003d670 con 0x7f5698150d20 2024-09-19T05:42:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae2182640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad40d8a80 0x7f4ad416b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569814b4f0 con 0x7f5698150d20 2024-09-19T05:42:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56981f1e50 con 0x7f5698150d20 2024-09-19T05:42:39.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690037040 con 0x7f5698150d20 2024-09-19T05:42:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae1180640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae1180640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39192/0 (socket says 172.21.15.2:39192) 2024-09-19T05:42:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08041020 con 0x7f4a180b3220 2024-09-19T05:42:39.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae1180640 1 -- 172.21.15.2:0/1713043685 learned_addr learned my addr 172.21.15.2:0/1713043685 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f18b6ffd640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f18a806c3b0 0x7f18a806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae2182640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4049510 con 0x7f4ad40d8a80 2024-09-19T05:42:39.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08030c70 con 0x7f4a180b3220 2024-09-19T05:42:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a18201820 con 0x7f4a180b3220 2024-09-19T05:42:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08037040 con 0x7f4a180b3220 2024-09-19T05:42:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae2182640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4006e50 con 0x7f4ad404f160 2024-09-19T05:42:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f18ca073640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f18a806c3b0 0x7f18a806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f18b808b780 con 0x7f18bc04c360 2024-09-19T05:42:39.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.508+0000 7f2fbe21e640 1 -- 172.21.15.2:0/2295170778 shutdown_connections 2024-09-19T05:42:39.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4ae2182640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad404f720 con 0x7f4ad4007370 2024-09-19T05:42:39.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6060145580 con 0x7f6060157490 2024-09-19T05:42:39.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a181fdd90 con 0x7f4a180b3220 2024-09-19T05:42:39.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f18b6ffd640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f18a8074630 0x7f18a8076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18a8077100 con 0x7f18a8074630 2024-09-19T05:42:39.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f18b808bb70 con 0x7f18bc04c360 2024-09-19T05:42:39.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f18ca874640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f18a8074630 0x7f18a8076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f18ca073640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f18a806c3b0 0x7f18a806e870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18c406a4c0 tx=0x7f18c406e040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7d2c01a070 con 0x7f7d3c159100 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f4ae097f640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad404f160 0x7f4ad41777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f4ae097f640 1 -- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 msgr2=0x7f4ad40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f4ae097f640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ad4007370 0x7f4ad40ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f4ae097f640 1 -- 172.21.15.2:0/1713043685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad40d8a80 msgr2=0x7f4ad416b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f4ae097f640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad40d8a80 0x7f4ad416b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7f4ae097f640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad4169b70 con 0x7f4ad404f160 2024-09-19T05:42:39.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f7d327fc640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d2006c380 0x7f7d2006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f4ae097f640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad404f160 0x7f4ad41777f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c5e0 tx=0x7f4ad800caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7d2c08a7e0 con 0x7f7d3c159100 2024-09-19T05:42:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f18ca874640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f18a8074630 0x7f18a8076a50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f2fbe21e640 1 -- 172.21.15.2:0/2295170778 wait complete. 2024-09-19T05:42:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5680001030 con 0x7f5698150d20 2024-09-19T05:42:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.509+0000 7efcb7dc4640 1 Processor -- start 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.510+0000 7efcb7dc4640 1 -- start start 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb7dc4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcb0072ae0 0x7efcb014a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb7dc4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcb0158de0 0x7efcb01611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb7dc4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0162790 0x7efcb0161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb7dc4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efcb00722e0 con 0x7efcb0072ae0 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb7dc4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0159770 con 0x7efcb0162790 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb7dc4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efcb00725c0 con 0x7efcb0158de0 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb6361640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0162790 0x7efcb0161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb6361640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0162790 0x7efcb0161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44056/0 (socket says 172.21.15.2:44056) 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7efcb6361640 1 -- 172.21.15.2:0/3350771846 learned_addr learned my addr 172.21.15.2:0/3350771846 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efcb6361640 1 -- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcb0158de0 msgr2=0x7efcb01611e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efcb6361640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efcb0158de0 0x7efcb01611e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efcb6361640 1 -- 172.21.15.2:0/3350771846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcb0072ae0 msgr2=0x7efcb014a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efcb6361640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcb0072ae0 0x7efcb014a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efcb6361640 1 -- 172.21.15.2:0/3350771846 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb0162290 con 0x7efcb0162790 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f60580050d0 con 0x7f6060157490 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f604effd640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f604006c3b0 0x7f604006e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f605808b120 con 0x7f6060157490 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f604effd640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f60400745b0 0x7f60400769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f605801eec0 con 0x7f60400745b0 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f605dd9b640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f60400745b0 0x7f60400769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6058092050 con 0x7f6060157490 2024-09-19T05:42:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f605dd9b640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f60400745b0 0x7f60400769d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5690044020 con 0x7f5698150d20 2024-09-19T05:42:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18a8077100 con 0x7f18a8074630 2024-09-19T05:42:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8015070 con 0x7f4ad404f160 2024-09-19T05:42:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad8011800 con 0x7f4ad404f160 2024-09-19T05:42:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8011bb0 con 0x7f4ad404f160 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f4ae2182640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad4052630 con 0x7f4ad404f160 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a180cac10 con 0x7f4a180b3220 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4a08044020 con 0x7f4a180b3220 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f4a0dffb640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f49fc06c300 0x7f49fc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4a080abd00 con 0x7f4a180b3220 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efcb6361640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0162790 0x7efcb0161720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efca800c940 tx=0x7efca800ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8018070 con 0x7efcb0162790 2024-09-19T05:42:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca800a700 con 0x7efcb0162790 2024-09-19T05:42:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca800aad0 con 0x7efcb0162790 2024-09-19T05:42:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f4ae2182640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad40cf270 con 0x7f4ad404f160 2024-09-19T05:42:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f4a0dffb640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f49fc074520 0x7f49fc076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a0802be50 con 0x7f49fc074520 2024-09-19T05:42:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f2fbe21e640 1 Processor -- start 2024-09-19T05:42:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4a080ae320 con 0x7f4a180b3220 2024-09-19T05:42:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f605e59c640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f604006c3b0 0x7f604006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.513+0000 7f568effd640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f567c06c380 0x7f567c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f2fbe21e640 1 -- start start 2024-09-19T05:42:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb0161fd0 con 0x7efcb0162790 2024-09-19T05:42:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f7d41a80640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d2006c380 0x7f7d2006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.511+0000 7f7d327fc640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7d20074520 0x7f7d20076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f7d42281640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7d20074520 0x7f7d20076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f7d41a80640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d2006c380 0x7f7d2006e840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d34033000 tx=0x7f7d34009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f7d42281640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7d20074520 0x7f7d20076940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.512+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d2c0052d0 con 0x7f7d20074520 2024-09-19T05:42:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c091050 con 0x7f7d3c159100 2024-09-19T05:42:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d2c0052d0 con 0x7f7d20074520 2024-09-19T05:42:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb01f5930 con 0x7efcb0162790 2024-09-19T05:42:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f569e480640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f567c06c380 0x7f567c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f56900ac4d0 con 0x7f5698150d20 2024-09-19T05:42:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f605e59c640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f604006c3b0 0x7f604006e870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6050039000 tx=0x7f6050040040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f605801eec0 con 0x7f60400745b0 2024-09-19T05:42:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ad40d92f0 con 0x7f4ad404f160 2024-09-19T05:42:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.514+0000 7f4a1dfd4640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f49fc074520 0x7f49fc076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f569e480640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f567c06c380 0x7f567c06e840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5694009fc0 tx=0x7f569400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f568effd640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f567c0745a0 0x7f567c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efcb006f640 con 0x7efcb0162790 2024-09-19T05:42:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f2fbe21e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fb80c87a0 0x7f2fb80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f2fbe21e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c9130 0x7f2fb8153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5690008eb0 con 0x7f567c0745a0 2024-09-19T05:42:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fb80c87a0 0x7f2fb80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f2fb77fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fb80c87a0 0x7f2fb80cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39216/0 (socket says 172.21.15.2:39216) 2024-09-19T05:42:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f2fb77fe640 1 -- 172.21.15.2:0/1261852193 learned_addr learned my addr 172.21.15.2:0/1261852193 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f4a1cfd2640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f49fc06c300 0x7f49fc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f56900ac8c0 con 0x7f5698150d20 2024-09-19T05:42:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.515+0000 7f2fb6ffd640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c9130 0x7f2fb8153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f2fbe21e640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb80c9510 0x7f2fb8153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f569dc7f640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f567c0745a0 0x7f567c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb81573c0 con 0x7f2fb80c9130 2024-09-19T05:42:39.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8157240 con 0x7f2fb80c9510 2024-09-19T05:42:39.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb8071b50 con 0x7f2fb80c87a0 2024-09-19T05:42:39.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f2fb7fff640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb80c9510 0x7f2fb8153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f4a1dfd4640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f49fc074520 0x7f49fc076940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f4a1cfd2640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f49fc06c300 0x7f49fc06e7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007fd0 tx=0x7f4a14009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efca801b070 con 0x7efcb0162790 2024-09-19T05:42:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f569dc7f640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f567c0745a0 0x7f567c0769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.516+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4ad8018070 con 0x7f4ad404f160 2024-09-19T05:42:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4ad1ffb640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4ac006c3b0 0x7f4ac006e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7efca67fc640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efc9406c380 0x7efc9406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f2fb6ffd640 1 -- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fb80c87a0 msgr2=0x7f2fb80cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f2fb6ffd640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2fb80c87a0 0x7f2fb80cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f2fb6ffd640 1 -- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb80c9510 msgr2=0x7f2fb8153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f2fb6ffd640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fb80c9510 0x7f2fb8153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f2fb6ffd640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb81515d0 con 0x7f2fb80c9130 2024-09-19T05:42:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efca808ad70 con 0x7efcb0162790 2024-09-19T05:42:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad808b840 con 0x7f4ad404f160 2024-09-19T05:42:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7efca67fc640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efc94074600 0x7efc94076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efca800ae50 con 0x7efc94074600 2024-09-19T05:42:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7efca808d3d0 con 0x7efcb0162790 2024-09-19T05:42:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4ae1180640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4ac006c3b0 0x7f4ac006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a0802be50 con 0x7f49fc074520 2024-09-19T05:42:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4ad1ffb640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac0074630 0x7f4ac0076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7efcb535f640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efc94074600 0x7efc94076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7efcb5b60640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efc9406c380 0x7efc9406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f2fb6ffd640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c9130 0x7f2fb8153a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80c9920 tx=0x7f2fac00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5690008eb0 con 0x7f567c0745a0 2024-09-19T05:42:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac012b80 con 0x7f2fb80c9130 2024-09-19T05:42:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac012d40 con 0x7f2fb80c9130 2024-09-19T05:42:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ad8009f20 con 0x7f4ac0074630 2024-09-19T05:42:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4ad808bc30 con 0x7f4ad404f160 2024-09-19T05:42:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.517+0000 7f4ae1981640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac0074630 0x7f4ac0076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac01b640 con 0x7f2fb80c9130 2024-09-19T05:42:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7f4ae1180640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4ac006c3b0 0x7f4ac006e870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4adc06ab10 tx=0x7f4adc06e030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb81518b0 con 0x7f2fb80c9130 2024-09-19T05:42:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7efcb5b60640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efc9406c380 0x7efc9406e840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcac0015e0 tx=0x7efcac0079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.518+0000 7efcb535f640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efc94074600 0x7efc94076a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.519+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb8151e70 con 0x7f2fb80c9130 2024-09-19T05:42:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.519+0000 7f4ae1981640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac0074630 0x7f4ac0076a50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.519+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efca800ae50 con 0x7efc94074600 2024-09-19T05:42:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.519+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ad8009f20 con 0x7f4ac0074630 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.521+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fb8073350 con 0x7f2fb80c9130 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.522+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2fac01b7e0 con 0x7f2fb80c9130 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.522+0000 7f2f9ffff640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2fa406c380 0x7f2fa406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.523+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2fac08ef00 con 0x7f2fb80c9130 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.523+0000 7f2f9ffff640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2fa4074600 0x7f2fa4076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.523+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fac01be50 con 0x7f2fa4074600 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.523+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f2fac091560 con 0x7f2fb80c9130 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.523+0000 7f2fb77fe640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2fa406c380 0x7f2fa406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.523+0000 7f2fb7fff640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2fa4074600 0x7f2fa4076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.524+0000 7f2fb7fff640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2fa4074600 0x7f2fa4076a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.524+0000 7f2fb77fe640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2fa406c380 0x7f2fa406e840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009a30 tx=0x7f2fa8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.525+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fac01be50 con 0x7f2fa4074600 2024-09-19T05:42:39.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.544+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18bc004a90 con 0x7f18a8074630 2024-09-19T05:42:39.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.544+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60600c4f20 con 0x7f60400745b0 2024-09-19T05:42:39.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.544+0000 7f7d42a82640 1 -- 172.21.15.2:0/2576818753 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d24002dd0 con 0x7f7d20074520 2024-09-19T05:42:39.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.544+0000 7f18b6ffd640 1 -- 172.21.15.2:0/722633689 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f18bc004a90 con 0x7f18a8074630 2024-09-19T05:42:39.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f7d327fc640 1 -- 172.21.15.2:0/2576818753 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7d24002dd0 con 0x7f7d20074520 2024-09-19T05:42:39.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.544+0000 7f604effd640 1 -- 172.21.15.2:0/344592407 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f60600c4f20 con 0x7f60400745b0 2024-09-19T05:42:39.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f60400745b0 msgr2=0x7f60400769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f60400745b0 0x7f60400769d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f604006c3b0 msgr2=0x7f604006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f604006c3b0 0x7f604006e870 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6050039000 tx=0x7f6050040040 comp rx=0 tx=0).stop 2024-09-19T05:42:39.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6060157490 msgr2=0x7f6060145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 --2- 172.21.15.2:0/344592407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6060157490 0x7f6060145ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f605800c730 tx=0x7f605800cbf0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605ed9d640 1 -- 172.21.15.2:0/344592407 reap_dead start 2024-09-19T05:42:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 shutdown_connections 2024-09-19T05:42:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 >> 172.21.15.2:0/344592407 conn(0x7f6060070520 msgr2=0x7f60600c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f7d1ffff640 1 -- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7d20074520 msgr2=0x7f7d20076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f7d1ffff640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7d20074520 0x7f7d20076940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.545+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 shutdown_connections 2024-09-19T05:42:39.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f605f59e640 1 -- 172.21.15.2:0/344592407 wait complete. 2024-09-19T05:42:39.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 -- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d2006c380 msgr2=0x7f7d2006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d2006c380 0x7f7d2006e840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d34033000 tx=0x7f7d34009000 comp rx=0 tx=0).stop 2024-09-19T05:42:39.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 -- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d3c159100 msgr2=0x7f7d3c0c0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 --2- 172.21.15.2:0/2576818753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d3c159100 0x7f7d3c0c0990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c780 tx=0x7f7d2c00cc40 comp rx=0 tx=0).stop 2024-09-19T05:42:39.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d42281640 1 -- 172.21.15.2:0/2576818753 reap_dead start 2024-09-19T05:42:39.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f18a8074630 msgr2=0x7f18a8076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f18cb075640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f18a8074630 0x7f18a8076a50 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 -- 172.21.15.2:0/2576818753 shutdown_connections 2024-09-19T05:42:39.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 -- 172.21.15.2:0/2576818753 >> 172.21.15.2:0/2576818753 conn(0x7f7d3c06bc10 msgr2=0x7f7d3c072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 -- 172.21.15.2:0/2576818753 shutdown_connections 2024-09-19T05:42:39.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.546+0000 7f7d1ffff640 1 -- 172.21.15.2:0/2576818753 wait complete. 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.547+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f18a806c3b0 msgr2=0x7f18a806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.547+0000 7f18cb075640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f18a806c3b0 0x7f18a806e870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18c406a4c0 tx=0x7f18c406e040 comp rx=0 tx=0).stop 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.547+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04c360 msgr2=0x7f18bc17c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.547+0000 7f18cb075640 1 --2- 172.21.15.2:0/722633689 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18bc04c360 0x7f18bc17c550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e8a0 tx=0x7f18b800ed60 comp rx=0 tx=0).stop 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.547+0000 7f18ca874640 1 -- 172.21.15.2:0/722633689 reap_dead start 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.547+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 shutdown_connections 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.547+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 >> 172.21.15.2:0/722633689 conn(0x7f18bc050090 msgr2=0x7f18bc005810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.548+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a180c5670 con 0x7f49fc074520 2024-09-19T05:42:39.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.548+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 shutdown_connections 2024-09-19T05:42:39.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.548+0000 7f18cb075640 1 -- 172.21.15.2:0/722633689 wait complete. 2024-09-19T05:42:39.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.549+0000 7f4a0dffb640 1 -- 172.21.15.2:0/856083875 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a180c5670 con 0x7f49fc074520 2024-09-19T05:42:39.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.549+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f49fc074520 msgr2=0x7f49fc076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.549+0000 7f4a1fa37640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f49fc074520 0x7f49fc076940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.549+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f49fc06c300 msgr2=0x7f49fc06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4a1fa37640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f49fc06c300 0x7f49fc06e7c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007fd0 tx=0x7f4a14009040 comp rx=0 tx=0).stop 2024-09-19T05:42:39.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180b3220 msgr2=0x7f4a181502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4a1fa37640 1 --2- 172.21.15.2:0/856083875 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a180b3220 0x7f4a181502b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4a0802a780 tx=0x7f4a080046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4a1dfd4640 1 -- 172.21.15.2:0/856083875 reap_dead start 2024-09-19T05:42:39.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5680002dd0 con 0x7f567c0745a0 2024-09-19T05:42:39.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ad40d92f0 con 0x7f4ac0074630 2024-09-19T05:42:39.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f568effd640 1 -- 172.21.15.2:0/213223002 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5680002dd0 con 0x7f567c0745a0 2024-09-19T05:42:39.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4ad1ffb640 1 -- 172.21.15.2:0/1713043685 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ad40d92f0 con 0x7f4ac0074630 2024-09-19T05:42:39.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 shutdown_connections 2024-09-19T05:42:39.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.550+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 >> 172.21.15.2:0/856083875 conn(0x7f4a18067950 msgr2=0x7f4a181e6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 shutdown_connections 2024-09-19T05:42:39.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4a1fa37640 1 -- 172.21.15.2:0/856083875 wait complete. 2024-09-19T05:42:39.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efcb0152810 con 0x7efc94074600 2024-09-19T05:42:39.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f567c0745a0 msgr2=0x7f567c0769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f56a06e4640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f567c0745a0 0x7f567c0769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac0074630 msgr2=0x7f4ac0076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4abf7fe640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac0074630 0x7f4ac0076a50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7efca67fc640 1 -- 172.21.15.2:0/3350771846 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efcb0152810 con 0x7efc94074600 2024-09-19T05:42:39.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f567c06c380 msgr2=0x7f567c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f56a06e4640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f567c06c380 0x7f567c06e840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5694009fc0 tx=0x7f569400b040 comp rx=0 tx=0).stop 2024-09-19T05:42:39.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5698150d20 msgr2=0x7f569814a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f56a06e4640 1 --2- 172.21.15.2:0/213223002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5698150d20 0x7f569814a170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f569002a6e0 tx=0x7f56900046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4ac006c3b0 msgr2=0x7f4ac006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4abf7fe640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4ac006c3b0 0x7f4ac006e870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4adc06ab10 tx=0x7f4adc06e030 comp rx=0 tx=0).stop 2024-09-19T05:42:39.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad404f160 msgr2=0x7f4ad41777f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4abf7fe640 1 --2- 172.21.15.2:0/1713043685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ad404f160 0x7f4ad41777f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c5e0 tx=0x7f4ad800caa0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f4ae1981640 1 -- 172.21.15.2:0/1713043685 reap_dead start 2024-09-19T05:42:39.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.551+0000 7f569ec81640 1 -- 172.21.15.2:0/213223002 reap_dead start 2024-09-19T05:42:39.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 shutdown_connections 2024-09-19T05:42:39.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 >> 172.21.15.2:0/213223002 conn(0x7f569806bc10 msgr2=0x7f56980c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 shutdown_connections 2024-09-19T05:42:39.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 >> 172.21.15.2:0/1713043685 conn(0x7f4ad4054050 msgr2=0x7f4ad4004e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 shutdown_connections 2024-09-19T05:42:39.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f56a06e4640 1 -- 172.21.15.2:0/213223002 wait complete. 2024-09-19T05:42:39.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efc94074600 msgr2=0x7efc94076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7efcb7dc4640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efc94074600 0x7efc94076a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 shutdown_connections 2024-09-19T05:42:39.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7f4abf7fe640 1 -- 172.21.15.2:0/1713043685 wait complete. 2024-09-19T05:42:39.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efc9406c380 msgr2=0x7efc9406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7efcb7dc4640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efc9406c380 0x7efc9406e840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcac0015e0 tx=0x7efcac0079b0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0162790 msgr2=0x7efcb0161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7efcb7dc4640 1 --2- 172.21.15.2:0/3350771846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efcb0162790 0x7efcb0161720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efca800c940 tx=0x7efca800ce00 comp rx=0 tx=0).stop 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.552+0000 7efcb6361640 1 -- 172.21.15.2:0/3350771846 reap_dead start 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.553+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 shutdown_connections 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.553+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 >> 172.21.15.2:0/3350771846 conn(0x7efcb006bc10 msgr2=0x7efcb00c70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.553+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 shutdown_connections 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.553+0000 7efcb7dc4640 1 -- 172.21.15.2:0/3350771846 wait complete. 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.555+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fb8073350 con 0x7f2fa4074600 2024-09-19T05:42:39.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.555+0000 7f2f9ffff640 1 -- 172.21.15.2:0/1261852193 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2fb8073350 con 0x7f2fa4074600 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.557+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2fa4074600 msgr2=0x7f2fa4076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.557+0000 7f2fbe21e640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2fa4074600 0x7f2fa4076a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.557+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2fa406c380 msgr2=0x7f2fa406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.557+0000 7f2fbe21e640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2fa406c380 0x7f2fa406e840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009a30 tx=0x7f2fa8009190 comp rx=0 tx=0).stop 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.557+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c9130 msgr2=0x7f2fb8153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.557+0000 7f2fbe21e640 1 --2- 172.21.15.2:0/1261852193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80c9130 0x7f2fb8153a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80c9920 tx=0x7f2fac00b7f0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.557+0000 7f2fb7fff640 1 -- 172.21.15.2:0/1261852193 reap_dead start 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.558+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 shutdown_connections 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.558+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 >> 172.21.15.2:0/1261852193 conn(0x7f2fb806bc10 msgr2=0x7f2fb814e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.558+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 shutdown_connections 2024-09-19T05:42:39.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.558+0000 7f2fbe21e640 1 -- 172.21.15.2:0/1261852193 wait complete. 2024-09-19T05:42:39.612 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:39.612 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T05:42:39.613 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-19T05:42:39.614 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T05:42:39.614 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:39.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T05:42:39.615 INFO:teuthology.orchestra.run.smithi002.stdout:55834574854 2024-09-19T05:42:39.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T05:42:39.616 INFO:teuthology.orchestra.run.smithi002.stdout:64424509446 2024-09-19T05:42:39.617 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T05:42:39.617 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-19T05:42:39.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T05:42:39.621 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:39.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T05:42:39.622 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:39.622 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T05:42:39.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.828+0000 7f6aed9df640 1 Processor -- start 2024-09-19T05:42:39.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.828+0000 7f6aed9df640 1 -- start start 2024-09-19T05:42:39.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.829+0000 7f8768991640 1 Processor -- start 2024-09-19T05:42:39.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.829+0000 7f2afb242640 1 Processor -- start 2024-09-19T05:42:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.829+0000 7f96055f8640 1 Processor -- start 2024-09-19T05:42:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.829+0000 7f96055f8640 1 -- start start 2024-09-19T05:42:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.829+0000 7f6aed9df640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ae8150d20 0x7f6ae80c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6aed9df640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6ae8072650 0x7f6ae8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6aed9df640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae815a130 con 0x7f6ae8072090 2024-09-19T05:42:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6aed9df640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae80c6990 con 0x7f6ae8072650 2024-09-19T05:42:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6aed9df640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae815a3a0 con 0x7f6ae8150d20 2024-09-19T05:42:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6ae77fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6ae8072090 0x7f6ae806ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51572/0 (socket says 172.21.15.2:51572) 2024-09-19T05:42:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6ae77fe640 1 -- 172.21.15.2:0/2765046806 learned_addr learned my addr 172.21.15.2:0/2765046806 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f0513fff640 1 Processor -- start 2024-09-19T05:42:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6ae67fc640 1 --2- 172.21.15.2:0/2765046806 >> v2:172.21.15.2:3300/0 conn(0x7f6ae8072650 0x7f6ae8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f6ae6ffd640 1 --2- 172.21.15.2:0/2765046806 >> v2:172.21.15.123:3300/0 conn(0x7f6ae8150d20 0x7f6ae80c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.829+0000 7f8768991640 1 -- start start 2024-09-19T05:42:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f96055f8640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96000719a0 0x7f9600072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f96055f8640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96000c8e80 con 0x7f96000c8970 2024-09-19T05:42:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f6ae5ffb640 1 -- 172.21.15.2:0/2765046806 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2690423813 0 0) 0x7f6ae815a130 con 0x7f6ae8072090 2024-09-19T05:42:39.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f6ae5ffb640 1 -- 172.21.15.2:0/2765046806 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ad4003520 con 0x7f6ae8072090 2024-09-19T05:42:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f95ff7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f96000c8970 0x7f96000cc6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51576/0 (socket says 172.21.15.2:51576) 2024-09-19T05:42:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f95ff7fe640 1 -- 172.21.15.2:0/3225364721 learned_addr learned my addr 172.21.15.2:0/3225364721 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f95fe7fc640 1 --2- 172.21.15.2:0/3225364721 >> v2:172.21.15.123:3300/0 conn(0x7f96000719a0 0x7f9600072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f96055f8640 1 -- 172.21.15.2:0/3225364721 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96000c9160 con 0x7f960015aae0 2024-09-19T05:42:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f27ed423640 1 Processor -- start 2024-09-19T05:42:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f96055f8640 1 -- 172.21.15.2:0/3225364721 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f96000c6170 con 0x7f96000719a0 2024-09-19T05:42:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f6ae67fc640 1 -- 172.21.15.2:0/2765046806 >> v2:172.21.15.123:3300/0 conn(0x7f6ae8150d20 msgr2=0x7f6ae80c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f6ae67fc640 1 --2- 172.21.15.2:0/2765046806 >> v2:172.21.15.123:3300/0 conn(0x7f6ae8150d20 0x7f6ae80c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f6ae67fc640 1 -- 172.21.15.2:0/2765046806 >> v1:172.21.15.2:6789/0 conn(0x7f6ae8072090 legacy=0x7f6ae806ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f6ae67fc640 1 -- 172.21.15.2:0/2765046806 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae815a680 con 0x7f6ae8072650 2024-09-19T05:42:39.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1568203556 0 0) 0x7f96000c8e80 con 0x7f96000c8970 2024-09-19T05:42:39.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.829+0000 7f2afb242640 1 -- start start 2024-09-19T05:42:39.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f8768991640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8764162790 0x7f8764162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f8768991640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f87641594f0 0x7f8764072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f2afb242640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af4162d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f8768991640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87640c98b0 con 0x7f87641630b0 2024-09-19T05:42:39.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f8768991640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f87640c47f0 con 0x7f87641594f0 2024-09-19T05:42:39.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f8768991640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f87640c9b90 con 0x7f8764162790 2024-09-19T05:42:39.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95e8005730 con 0x7f96000c8970 2024-09-19T05:42:39.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726565027 0 0) 0x7f96000c9160 con 0x7f960015aae0 2024-09-19T05:42:39.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96000c8e80 con 0x7f960015aae0 2024-09-19T05:42:39.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f2afb242640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af415fd30 con 0x7f2af40c7020 2024-09-19T05:42:39.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f2afb242640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af4163280 con 0x7f2af415ed40 2024-09-19T05:42:39.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f8762d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f87641594f0 0x7f8764072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f2afb242640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2af4072410 con 0x7f2af40c6c40 2024-09-19T05:42:39.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f0513fff640 1 -- start start 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f0513fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0dcb60 0x7f050c0e3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f0513fff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f050c04f050 0x7f050c04f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f0513fff640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f050c0e38c0 con 0x7f050c007010 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f0513fff640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f050c005f70 con 0x7f050c04f050 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f0513fff640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f050c006250 con 0x7f050c0dcb60 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f05127fc640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f050c007010 0x7f050c050000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51592/0 (socket says 172.21.15.2:51592) 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f05127fc640 1 -- 172.21.15.2:0/1526218773 learned_addr learned my addr 172.21.15.2:0/1526218773 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f876259c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8764162790 0x7f8764162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f8761d9b640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f87641630b0 0x7f8764158de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:42990/0 (socket says 172.21.15.2:42990) 2024-09-19T05:42:39.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f8761d9b640 1 -- 172.21.15.2:0/1473707706 learned_addr learned my addr 172.21.15.2:0/1473707706 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.830+0000 7f27ed423640 1 -- start start 2024-09-19T05:42:39.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f27ed423640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e8155700 0x7f27e80721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f27ed423640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f27e80c6ed0 0x7f27e8162d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.831+0000 7f27ed423640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f27e80c7660 0x7f27e80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f27ed423640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f27e80c4a00 con 0x7f27e80c7660 2024-09-19T05:42:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f27ed423640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f27e80c8320 con 0x7f27e80c6ed0 2024-09-19T05:42:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f27ed423640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f27e80c85c0 con 0x7f27e8155700 2024-09-19T05:42:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f27e67fc640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f27e80c6ed0 0x7f27e8162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f2afa240640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af4162d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f2afa240640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af4162d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39234/0 (socket says 172.21.15.2:39234) 2024-09-19T05:42:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f2afa240640 1 -- 172.21.15.2:0/2037322877 learned_addr learned my addr 172.21.15.2:0/2037322877 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790590415 0 0) 0x7f050c0e38c0 con 0x7f050c007010 2024-09-19T05:42:39.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04fc003520 con 0x7f050c007010 2024-09-19T05:42:39.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f6ae67fc640 1 --2- 172.21.15.2:0/2765046806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8072650 0x7f6ae8071b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8070700 tx=0x7f6ad802c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabdf794509dad3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0512ffd640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0dcb60 0x7f050c0e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f05137fe640 1 --2- 172.21.15.2:0/1526218773 >> v2:172.21.15.123:3300/0 conn(0x7f050c04f050 0x7f050c04f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 522480808 0 0) 0x7f04fc003520 con 0x7f050c007010 2024-09-19T05:42:39.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0dcb60 msgr2=0x7f050c0e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0dcb60 0x7f050c0e3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 >> v2:172.21.15.123:3300/0 conn(0x7f050c04f050 msgr2=0x7f050c04f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 --2- 172.21.15.2:0/1526218773 >> v2:172.21.15.123:3300/0 conn(0x7f050c04f050 0x7f050c04f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c006530 con 0x7f050c007010 2024-09-19T05:42:39.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7fe1b5e4c640 1 Processor -- start 2024-09-19T05:42:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f27e77fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f27e80c7660 0x7f27e80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f27e77fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f27e80c7660 0x7f27e80c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37662/0 (socket says 172.21.15.2:37662) 2024-09-19T05:42:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f27e77fe640 1 -- 172.21.15.2:0/2026321369 learned_addr learned my addr 172.21.15.2:0/2026321369 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f0514063850 con 0x7f050c007010 2024-09-19T05:42:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 >> v1:172.21.15.2:6789/0 conn(0x7f050c007010 legacy=0x7f050c050000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04fc0039b0 0x7f04fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc004800 0x7f04fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f876159a640 1 -- 172.21.15.2:0/1473707706 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3384614804 0 0) 0x7f87640c98b0 con 0x7f87641630b0 2024-09-19T05:42:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f876159a640 1 -- 172.21.15.2:0/1473707706 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f874c003520 con 0x7f87641630b0 2024-09-19T05:42:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04fc005210 0x7f04fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f050c005f70 con 0x7f04fc004800 2024-09-19T05:42:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f050c006250 con 0x7f04fc0039b0 2024-09-19T05:42:39.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc00c120 con 0x7f04fc005210 2024-09-19T05:42:39.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7fc4f7fff640 1 Processor -- start 2024-09-19T05:42:39.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f27e77fe640 1 -- 172.21.15.2:0/2026321369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e8155700 msgr2=0x7f27e80721a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:42:39.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f27e77fe640 1 --2- 172.21.15.2:0/2026321369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e8155700 0x7f27e80721a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f27e77fe640 1 -- 172.21.15.2:0/2026321369 >> v2:172.21.15.123:3300/0 conn(0x7f27e80c6ed0 msgr2=0x7f27e8162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f27e77fe640 1 --2- 172.21.15.2:0/2026321369 >> v2:172.21.15.123:3300/0 conn(0x7f27e80c6ed0 0x7f27e8162d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f27e77fe640 1 -- 172.21.15.2:0/2026321369 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e80c88a0 con 0x7f27e80c7660 2024-09-19T05:42:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f6ae5ffb640 1 -- 172.21.15.2:0/2765046806 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8039070 con 0x7f6ae8072650 2024-09-19T05:42:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.832+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1212824887 0 0) 0x7f95e8005730 con 0x7f96000c8970 2024-09-19T05:42:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 >> v2:172.21.15.123:3300/0 conn(0x7f96000719a0 msgr2=0x7f9600072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 --2- 172.21.15.2:0/3225364721 >> v2:172.21.15.123:3300/0 conn(0x7f96000719a0 0x7f9600072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 >> v1:172.21.15.123:6789/0 conn(0x7f960015aae0 legacy=0x7f9600150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96000c9440 con 0x7f96000c8970 2024-09-19T05:42:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f95f00019f0 con 0x7f96000c8970 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 >> v1:172.21.15.2:6789/0 conn(0x7f96000c8970 legacy=0x7f96000cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e8006e50 0x7f95e8007250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80059b0 0x7f95e8005e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f95e800dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96000c6170 con 0x7f95e8006e50 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95e800e420 con 0x7f95e80059b0 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f95e800e620 con 0x7f95e80063e0 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95ff7fe640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f95e800dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f95ff7fe640 1 -- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80059b0 msgr2=0x7f95e8005e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f95ff7fe640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80059b0 0x7f95e8005e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f95ff7fe640 1 -- 172.21.15.2:0/3225364721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e8006e50 msgr2=0x7f95e8007250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f95ff7fe640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e8006e50 0x7f95e8007250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f95ff7fe640 1 -- 172.21.15.2:0/3225364721 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e8005730 con 0x7f95e80063e0 2024-09-19T05:42:39.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7fc4f7fff640 1 -- start start 2024-09-19T05:42:39.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f6ae5ffb640 1 -- 172.21.15.2:0/2765046806 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad8008640 con 0x7f6ae8072650 2024-09-19T05:42:39.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f2af923e640 1 -- 172.21.15.2:0/2037322877 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344635040 0 0) 0x7f2af4163280 con 0x7f2af415ed40 2024-09-19T05:42:39.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f2af923e640 1 -- 172.21.15.2:0/2037322877 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ae0003520 con 0x7f2af415ed40 2024-09-19T05:42:39.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f6ae5ffb640 1 -- 172.21.15.2:0/2765046806 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8008a10 con 0x7f6ae8072650 2024-09-19T05:42:39.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f2afa240640 1 -- 172.21.15.2:0/2037322877 >> v1:172.21.15.123:6789/0 conn(0x7f2af415ed40 legacy=0x7f2af40728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7fe1b5e4c640 1 -- start start 2024-09-19T05:42:39.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f2afa240640 1 -- 172.21.15.2:0/2037322877 >> v1:172.21.15.2:6789/0 conn(0x7f2af40c7020 legacy=0x7f2af41589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f6aed9df640 1 -- 172.21.15.2:0/2765046806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8072650 msgr2=0x7f6ae8071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f6aed9df640 1 --2- 172.21.15.2:0/2765046806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8072650 0x7f6ae8071b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8070700 tx=0x7f6ad802c800 comp rx=0 tx=0).stop 2024-09-19T05:42:39.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f876259c640 1 -- 172.21.15.2:0/1473707706 >> v2:172.21.15.2:3300/0 conn(0x7f87641594f0 msgr2=0x7f8764072ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f876259c640 1 --2- 172.21.15.2:0/1473707706 >> v2:172.21.15.2:3300/0 conn(0x7f87641594f0 0x7f8764072ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f876259c640 1 -- 172.21.15.2:0/1473707706 >> v1:172.21.15.123:6789/0 conn(0x7f87641630b0 legacy=0x7f8764158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.833+0000 7f876259c640 1 -- 172.21.15.2:0/1473707706 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87640c9e70 con 0x7f8764162790 2024-09-19T05:42:39.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f95ff7fe640 1 -- 172.21.15.2:0/3225364721 reap_dead start 2024-09-19T05:42:39.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f95ff7fe640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f95e800dc50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95f000a730 tx=0x7f95e8007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89288818635e38d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f2afa240640 1 -- 172.21.15.2:0/2037322877 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af415f1d0 con 0x7f2af40c6c40 2024-09-19T05:42:39.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f05127fc640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc004800 0x7f04fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.834+0000 7f0512ffd640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04fc0039b0 0x7f04fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f0512ffd640 1 -- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04fc005210 msgr2=0x7f04fc00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f0512ffd640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04fc005210 0x7f04fc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f0512ffd640 1 -- 172.21.15.2:0/1526218773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc004800 msgr2=0x7f04fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f0512ffd640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc004800 0x7f04fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f0512ffd640 1 -- 172.21.15.2:0/1526218773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc003520 con 0x7f04fc0039b0 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fc4f7fff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc4f81577f0 0x7fc4f80719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fc4f7fff640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f80c8740 con 0x7fc4f80c6fb0 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fc4f7fff640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f8156a50 con 0x7fc4f806c700 2024-09-19T05:42:39.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fc4f7fff640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f80c77e0 con 0x7fc4f81577f0 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1b5e4c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe1a80e6270 0x7fe1a804bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1b5e4c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe1a8045c30 0x7fe1a8046010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1b5e4c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1a804a960 con 0x7fe1a80e6650 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1b5e4c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a8005a50 con 0x7fe1a8045c30 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1b5e4c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe1a80e34b0 con 0x7fe1a80e6270 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1affff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe1a8045c30 0x7fe1a8046010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1affff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe1a8045c30 0x7fe1a8046010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37684/0 (socket says 172.21.15.2:37684) 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fc4f67fc640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc4f81577f0 0x7fc4f80719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fc4f6ffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fc4f80c6fb0 0x7fc4f815aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51600/0 (socket says 172.21.15.2:51600) 2024-09-19T05:42:39.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fc4f6ffd640 1 -- 172.21.15.2:0/560023989 learned_addr learned my addr 172.21.15.2:0/560023989 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:39.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7fe1affff640 1 -- 172.21.15.2:0/2204911343 learned_addr learned my addr 172.21.15.2:0/2204911343 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f6aed9df640 1 -- 172.21.15.2:0/2765046806 shutdown_connections 2024-09-19T05:42:39.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f6aed9df640 1 --2- 172.21.15.2:0/2765046806 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8072650 0x7f6ae8071b50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f6aed9df640 1 --2- 172.21.15.2:0/2765046806 >> v2:172.21.15.123:3300/0 conn(0x7f6ae8150d20 0x7f6ae80c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f6aed9df640 1 -- 172.21.15.2:0/2765046806 >> 172.21.15.2:0/2765046806 conn(0x7f6ae806bc10 msgr2=0x7f6ae806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f27e77fe640 1 --2- 172.21.15.2:0/2026321369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e80c7660 0x7f27e80c7ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27e80cd7e0 tx=0x7f27d802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f3a0f431ca07a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f6aed9df640 1 -- 172.21.15.2:0/2765046806 shutdown_connections 2024-09-19T05:42:39.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fc4f67fc640 1 -- 172.21.15.2:0/560023989 >> v1:172.21.15.123:6789/0 conn(0x7fc4f806c700 legacy=0x7fc4f80c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fc4f67fc640 1 -- 172.21.15.2:0/560023989 >> v1:172.21.15.2:6789/0 conn(0x7fc4f80c6fb0 legacy=0x7fc4f815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fc4f67fc640 1 -- 172.21.15.2:0/560023989 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f80c7ac0 con 0x7fc4f81577f0 2024-09-19T05:42:39.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f27e5ffb640 1 -- 172.21.15.2:0/2026321369 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8037070 con 0x7f27e80c7660 2024-09-19T05:42:39.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f27e5ffb640 1 -- 172.21.15.2:0/2026321369 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d8032440 con 0x7f27e80c7660 2024-09-19T05:42:39.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f27e5ffb640 1 -- 172.21.15.2:0/2026321369 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d80327f0 con 0x7f27e80c7660 2024-09-19T05:42:39.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f000a350 con 0x7f95e80063e0 2024-09-19T05:42:39.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f002f070 con 0x7f95e80063e0 2024-09-19T05:42:39.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.835+0000 7f95fdffb640 1 -- 172.21.15.2:0/3225364721 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f002bb80 con 0x7f95e80063e0 2024-09-19T05:42:39.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fe1b4e4a640 1 --2- 172.21.15.2:0/2204911343 >> v2:172.21.15.123:3300/0 conn(0x7fe1a80e6270 0x7fe1a804bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f2afa240640 1 --2- 172.21.15.2:0/2037322877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af4162d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098a0 tx=0x7f2af002a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e15aab948a0a6a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fe1affff640 1 -- 172.21.15.2:0/2204911343 >> v2:172.21.15.123:3300/0 conn(0x7fe1a80e6270 msgr2=0x7fe1a804bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f6aed9df640 1 -- 172.21.15.2:0/2765046806 wait complete. 2024-09-19T05:42:39.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f2af923e640 1 -- 172.21.15.2:0/2037322877 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0037070 con 0x7f2af40c6c40 2024-09-19T05:42:39.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f96055f8640 1 -- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 msgr2=0x7f95e800dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f96055f8640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f95e800dc50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95f000a730 tx=0x7f95e8007a40 comp rx=0 tx=0).stop 2024-09-19T05:42:39.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f2af923e640 1 -- 172.21.15.2:0/2037322877 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af0032640 con 0x7f2af40c6c40 2024-09-19T05:42:39.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fe1affff640 1 --2- 172.21.15.2:0/2204911343 >> v2:172.21.15.123:3300/0 conn(0x7fe1a80e6270 0x7fe1a804bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fe1affff640 1 -- 172.21.15.2:0/2204911343 >> v1:172.21.15.2:6789/0 conn(0x7fe1a80e6650 legacy=0x7fe1a80dc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:39.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f876259c640 1 --2- 172.21.15.2:0/1473707706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8764162790 0x7f8764162b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8764071bf0 tx=0x7f875402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642a5aeca0c6fcda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f876159a640 1 -- 172.21.15.2:0/1473707706 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754037070 con 0x7f8764162790 2024-09-19T05:42:39.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f2af923e640 1 -- 172.21.15.2:0/2037322877 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0032a10 con 0x7f2af40c6c40 2024-09-19T05:42:39.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fe1affff640 1 -- 172.21.15.2:0/2204911343 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a80e36e0 con 0x7fe1a8045c30 2024-09-19T05:42:39.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7fe1b4e4a640 1 --2- 172.21.15.2:0/2204911343 >> v2:172.21.15.123:3300/0 conn(0x7fe1a80e6270 0x7fe1a804bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:39.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 -- 172.21.15.2:0/2026321369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e80c7660 msgr2=0x7f27e80c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f0512ffd640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04fc0039b0 0x7f04fc003dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f050c011e30 tx=0x7f04fc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5659269e94c5e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f876159a640 1 -- 172.21.15.2:0/1473707706 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754032440 con 0x7f8764162790 2024-09-19T05:42:39.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7fc4f67fc640 1 --2- 172.21.15.2:0/560023989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f81577f0 0x7fc4f80719f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009d90 tx=0x7fc4f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848cf8d374167b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.836+0000 7f876159a640 1 -- 172.21.15.2:0/1473707706 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754032850 con 0x7f8764162790 2024-09-19T05:42:39.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 --2- 172.21.15.2:0/2026321369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e80c7660 0x7f27e80c7ad0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27e80cd7e0 tx=0x7f27d802a2a0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f96055f8640 1 -- 172.21.15.2:0/3225364721 shutdown_connections 2024-09-19T05:42:39.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f96055f8640 1 --2- 172.21.15.2:0/3225364721 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f95e800dc50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f96055f8640 1 -- 172.21.15.2:0/3225364721 >> 172.21.15.2:0/3225364721 conn(0x7f960006f980 msgr2=0x7f960006fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7fc4f5ffb640 1 -- 172.21.15.2:0/560023989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0007740 con 0x7fc4f81577f0 2024-09-19T05:42:39.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7fc4f5ffb640 1 -- 172.21.15.2:0/560023989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f00078e0 con 0x7fc4f81577f0 2024-09-19T05:42:39.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7fc4f5ffb640 1 -- 172.21.15.2:0/560023989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0007cb0 con 0x7fc4f81577f0 2024-09-19T05:42:39.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f2afb242640 1 -- 172.21.15.2:0/2037322877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 msgr2=0x7f2af4162d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f8768991640 1 -- 172.21.15.2:0/1473707706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8764162790 msgr2=0x7f8764162b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f8768991640 1 --2- 172.21.15.2:0/1473707706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8764162790 0x7f8764162b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8764071bf0 tx=0x7f875402a1c0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504037070 con 0x7f04fc0039b0 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050402aa20 con 0x7f04fc0039b0 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 -- 172.21.15.2:0/2026321369 shutdown_connections 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 --2- 172.21.15.2:0/2026321369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e80c7660 0x7f27e80c7ad0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 --2- 172.21.15.2:0/2026321369 >> v2:172.21.15.123:3300/0 conn(0x7f27e80c6ed0 0x7f27e8162d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 --2- 172.21.15.2:0/2026321369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e8155700 0x7f27e80721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 -- 172.21.15.2:0/2026321369 >> 172.21.15.2:0/2026321369 conn(0x7f27e80ccc60 msgr2=0x7f27e80cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f0511ffb640 1 -- 172.21.15.2:0/1526218773 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400a4b0 con 0x7f04fc0039b0 2024-09-19T05:42:39.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f6aed9df640 1 Processor -- start 2024-09-19T05:42:39.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f96055f8640 1 -- 172.21.15.2:0/3225364721 shutdown_connections 2024-09-19T05:42:39.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f2afb242640 1 --2- 172.21.15.2:0/2037322877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af4162d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098a0 tx=0x7f2af002a320 comp rx=0 tx=0).stop 2024-09-19T05:42:39.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f96055f8640 1 -- 172.21.15.2:0/3225364721 wait complete. 2024-09-19T05:42:39.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f2afb242640 1 -- 172.21.15.2:0/2037322877 shutdown_connections 2024-09-19T05:42:39.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f2afb242640 1 --2- 172.21.15.2:0/2037322877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af4162d40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f2afb242640 1 -- 172.21.15.2:0/2037322877 >> 172.21.15.2:0/2037322877 conn(0x7f2af406bc10 msgr2=0x7f2af406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.837+0000 7f27ed423640 1 -- 172.21.15.2:0/2026321369 shutdown_connections 2024-09-19T05:42:39.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f2afb242640 1 -- 172.21.15.2:0/2037322877 shutdown_connections 2024-09-19T05:42:39.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f8768991640 1 -- 172.21.15.2:0/1473707706 shutdown_connections 2024-09-19T05:42:39.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f8768991640 1 --2- 172.21.15.2:0/1473707706 >> v2:172.21.15.2:3300/0 conn(0x7f87641594f0 0x7f8764072ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f8768991640 1 --2- 172.21.15.2:0/1473707706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8764162790 0x7f8764162b70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f8768991640 1 -- 172.21.15.2:0/1473707706 >> 172.21.15.2:0/1473707706 conn(0x7f876406bc10 msgr2=0x7f876406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fe1affff640 1 --2- 172.21.15.2:0/2204911343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8046010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d300 tx=0x7fe1a402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fb2ec0258a77de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f6aed9df640 1 -- start start 2024-09-19T05:42:39.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fe1af7fe640 1 -- 172.21.15.2:0/2204911343 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4037070 con 0x7fe1a8045c30 2024-09-19T05:42:39.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fc4f7fff640 1 -- 172.21.15.2:0/560023989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f81577f0 msgr2=0x7fc4f80719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fc4f7fff640 1 --2- 172.21.15.2:0/560023989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f81577f0 0x7fc4f80719f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009d90 tx=0x7fc4f002a6a0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fe1af7fe640 1 -- 172.21.15.2:0/2204911343 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a402ae50 con 0x7fe1a8045c30 2024-09-19T05:42:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f8768991640 1 -- 172.21.15.2:0/1473707706 shutdown_connections 2024-09-19T05:42:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f0513fff640 1 -- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04fc0039b0 msgr2=0x7f04fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f0513fff640 1 --2- 172.21.15.2:0/1526218773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04fc0039b0 0x7f04fc003dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f050c011e30 tx=0x7f04fc005830 comp rx=0 tx=0).stop 2024-09-19T05:42:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f05137fe640 1 -- 172.21.15.2:0/1526218773 reap_dead start 2024-09-19T05:42:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f27ed423640 1 -- 172.21.15.2:0/2026321369 wait complete. 2024-09-19T05:42:39.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f0513fff640 1 -- 172.21.15.2:0/1526218773 shutdown_connections 2024-09-19T05:42:39.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f0513fff640 1 -- 172.21.15.2:0/1526218773 >> 172.21.15.2:0/1526218773 conn(0x7f050c054060 msgr2=0x7f050c054470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6aed9df640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae8072090 0x7f6ae81f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6aed9df640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8072650 0x7f6ae814bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6aed9df640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8150d20 0x7f6ae814f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6aed9df640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae80c8cb0 con 0x7f6ae8150d20 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6aed9df640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae815ac40 con 0x7f6ae8072650 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6aed9df640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae80c80a0 con 0x7f6ae8072090 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8150d20 0x7f6ae814f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8072650 0x7f6ae814bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6ae77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8150d20 0x7f6ae814f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37694/0 (socket says 172.21.15.2:37694) 2024-09-19T05:42:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f6ae77fe640 1 -- 172.21.15.2:0/1523760448 learned_addr learned my addr 172.21.15.2:0/1523760448 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f96055f8640 1 Processor -- start 2024-09-19T05:42:39.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f2afb242640 1 -- 172.21.15.2:0/2037322877 wait complete. 2024-09-19T05:42:39.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7f2afb242640 1 Processor -- start 2024-09-19T05:42:39.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f96055f8640 1 -- start start 2024-09-19T05:42:39.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fe1af7fe640 1 -- 172.21.15.2:0/2204911343 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4032860 con 0x7fe1a8045c30 2024-09-19T05:42:39.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f2afb242640 1 -- start start 2024-09-19T05:42:39.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/2204911343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 msgr2=0x7fe1a8046010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/2204911343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8046010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d300 tx=0x7fe1a402a300 comp rx=0 tx=0).stop 2024-09-19T05:42:39.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f6ae77fe640 1 -- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae8072090 msgr2=0x7f6ae81f87f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:42:39.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f6ae77fe640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae8072090 0x7f6ae81f87f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f6ae77fe640 1 -- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8072650 msgr2=0x7f6ae814bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f6ae77fe640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8072650 0x7f6ae814bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f6ae77fe640 1 -- 172.21.15.2:0/1523760448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae8145a70 con 0x7f6ae8150d20 2024-09-19T05:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7f8768991640 1 -- 172.21.15.2:0/1473707706 wait complete. 2024-09-19T05:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f27ed423640 1 Processor -- start 2024-09-19T05:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f27ed423640 1 -- start start 2024-09-19T05:42:39.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fc4f7fff640 1 -- 172.21.15.2:0/560023989 shutdown_connections 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fc4f7fff640 1 --2- 172.21.15.2:0/560023989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f81577f0 0x7fc4f80719f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fc4f7fff640 1 -- 172.21.15.2:0/560023989 >> 172.21.15.2:0/560023989 conn(0x7fc4f806bc10 msgr2=0x7fc4f806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.838+0000 7fc4f7fff640 1 -- 172.21.15.2:0/560023989 shutdown_connections 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.839+0000 7fc4f7fff640 1 -- 172.21.15.2:0/560023989 wait complete. 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f0513fff640 1 -- 172.21.15.2:0/1526218773 shutdown_connections 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/2204911343 shutdown_connections 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/2204911343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8046010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/2204911343 >> v2:172.21.15.123:3300/0 conn(0x7fe1a80e6270 0x7fe1a804bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7f0513fff640 1 -- 172.21.15.2:0/1526218773 wait complete. 2024-09-19T05:42:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/2204911343 >> 172.21.15.2:0/2204911343 conn(0x7fe1a804fc50 msgr2=0x7fe1a8050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:39.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f6ae77fe640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8150d20 0x7f6ae814f5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c880 tx=0x7f6ae000cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f8768991640 1 Processor -- start 2024-09-19T05:42:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/2204911343 shutdown_connections 2024-09-19T05:42:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/2204911343 wait complete. 2024-09-19T05:42:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f96055f8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f9600151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f6adffff640 1 -- 172.21.15.2:0/1523760448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0008040 con 0x7f6ae8150d20 2024-09-19T05:42:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f8768991640 1 -- start start 2024-09-19T05:42:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f2afb242640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af414d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f96055f8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9600159b10 0x7f96001582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f6adffff640 1 -- 172.21.15.2:0/1523760448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae0007950 con 0x7f6ae8150d20 2024-09-19T05:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f2afb242640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2af40c7020 0x7f2af414d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f95feffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f9600151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f95feffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f9600151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39294/0 (socket says 172.21.15.2:39294) 2024-09-19T05:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f6aed9df640 1 -- 172.21.15.2:0/1523760448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae814b6e0 con 0x7f6ae8150d20 2024-09-19T05:42:39.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f6aed9df640 1 -- 172.21.15.2:0/1523760448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae8149a40 con 0x7f6ae8150d20 2024-09-19T05:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f95fe7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9600159b10 0x7f96001582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f2afb242640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af4153410 0x7f2af4150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27ed423640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e80c6ed0 0x7f27e8200e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27ed423640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e80c7660 0x7f27e8160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f95feffd640 1 -- 172.21.15.2:0/301580979 learned_addr learned my addr 172.21.15.2:0/301580979 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27e6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e80c6ed0 0x7f27e8200e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27e6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e80c6ed0 0x7f27e8200e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44162/0 (socket says 172.21.15.2:44162) 2024-09-19T05:42:39.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27e6ffd640 1 -- 172.21.15.2:0/1202260090 learned_addr learned my addr 172.21.15.2:0/1202260090 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f2afb242640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2af415fa50 con 0x7f2af40c7020 2024-09-19T05:42:39.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f96055f8640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96001f4740 0x7f96001587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f2afb242640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2af415f770 con 0x7f2af4153410 2024-09-19T05:42:39.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f2afb242640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2af4072fb0 con 0x7f2af40c6c40 2024-09-19T05:42:39.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f96055f8640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f960006dad0 con 0x7f96001f4740 2024-09-19T05:42:39.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f96055f8640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96000c9a00 con 0x7f9600159b10 2024-09-19T05:42:39.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f96055f8640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9600150a10 con 0x7f95e80063e0 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f2afa240640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af414d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f2afa240640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af414d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:39302/0 (socket says 172.21.15.2:39302) 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f2afa240640 1 -- 172.21.15.2:0/2866555295 learned_addr learned my addr 172.21.15.2:0/2866555295 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f2afaa41640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af4153410 0x7f2af4150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.840+0000 7fc4f7fff640 1 Processor -- start 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7fc4f7fff640 1 -- start start 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4f806c700 0x7fc4f8146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f80c6fb0 0x7fc4f81504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4f81577f0 0x7fc4f8150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f7fff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f81572d0 con 0x7fc4f80c6fb0 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f7fff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8156ff0 con 0x7fc4f81577f0 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f7fff640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f8071f30 con 0x7fc4f806c700 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4f81577f0 0x7fc4f8150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4f81577f0 0x7fc4f8150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44192/0 (socket says 172.21.15.2:44192) 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7fc4f77fe640 1 -- 172.21.15.2:0/1766141146 learned_addr learned my addr 172.21.15.2:0/1766141146 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f95feffd640 1 -- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9600159b10 msgr2=0x7f96001582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f95feffd640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9600159b10 0x7f96001582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f95feffd640 1 -- 172.21.15.2:0/301580979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96001f4740 msgr2=0x7f96001587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f95feffd640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96001f4740 0x7f96001587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f95feffd640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9600151e40 con 0x7f95e80063e0 2024-09-19T05:42:39.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f2afa240640 1 -- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af4153410 msgr2=0x7f2af4150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.842+0000 7f2afa240640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af4153410 0x7f2af4150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f2afa240640 1 -- 172.21.15.2:0/2866555295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2af40c7020 msgr2=0x7f2af414d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f2af9a3f640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2af40c7020 0x7f2af414d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f2afa240640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2af40c7020 0x7f2af414d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fc4f6ffd640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4f806c700 0x7fc4f8146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f2afa240640 1 -- 172.21.15.2:0/2866555295 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af41549f0 con 0x7f2af40c6c40 2024-09-19T05:42:39.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27ed423640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e8155700 0x7f27e81ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f2af9a3f640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2af40c7020 0x7f2af414d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:39.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fe1b5e4c640 1 Processor -- start 2024-09-19T05:42:39.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fe1b5e4c640 1 -- start start 2024-09-19T05:42:39.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f0513fff640 1 Processor -- start 2024-09-19T05:42:39.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f0513fff640 1 -- start start 2024-09-19T05:42:39.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6acc005290 con 0x7f6ae8150d20 2024-09-19T05:42:39.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f95feffd640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f9600151420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c530 tx=0x7f95f800c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fc4f77fe640 1 -- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4f806c700 msgr2=0x7fc4f8146460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fc4f77fe640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4f806c700 0x7fc4f8146460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fc4f77fe640 1 -- 172.21.15.2:0/1766141146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f80c6fb0 msgr2=0x7fc4f81504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fc4f77fe640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f80c6fb0 0x7fc4f81504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fc4f77fe640 1 -- 172.21.15.2:0/1766141146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f81ec3d0 con 0x7fc4f81577f0 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8768991640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87641594f0 0x7f87641fde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8768991640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8764162790 0x7f8764154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8768991640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87641630b0 0x7f8764161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8768991640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f87640725c0 con 0x7f8764162790 2024-09-19T05:42:39.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8768991640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87640722e0 con 0x7f87641594f0 2024-09-19T05:42:39.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8768991640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f876406f640 con 0x7f87641630b0 2024-09-19T05:42:39.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7fc4f77fe640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4f81577f0 0x7fc4f8150a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c3e0 tx=0x7fc4ec00c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f95e77fe640 1 -- 172.21.15.2:0/301580979 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8016020 con 0x7f95e80063e0 2024-09-19T05:42:39.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8761d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8764162790 0x7f8764154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8761d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8764162790 0x7f8764154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37724/0 (socket says 172.21.15.2:37724) 2024-09-19T05:42:39.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f8761d9b640 1 -- 172.21.15.2:0/3177849243 learned_addr learned my addr 172.21.15.2:0/3177849243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.843+0000 7f2afa240640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af414d1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2af002a910 tx=0x7f2af00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f6adffff640 1 -- 172.21.15.2:0/1523760448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0007d40 con 0x7f6ae8150d20 2024-09-19T05:42:39.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f95e77fe640 1 -- 172.21.15.2:0/301580979 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f80052f0 con 0x7f95e80063e0 2024-09-19T05:42:39.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f96055f8640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9600152100 con 0x7f95e80063e0 2024-09-19T05:42:39.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f2aeaffd640 1 -- 172.21.15.2:0/2866555295 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0041070 con 0x7f2af40c6c40 2024-09-19T05:42:39.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f96055f8640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001526c0 con 0x7f95e80063e0 2024-09-19T05:42:39.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af4154cd0 con 0x7f2af40c6c40 2024-09-19T05:42:39.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fc4df7fe640 1 -- 172.21.15.2:0/1766141146 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec011790 con 0x7fc4f81577f0 2024-09-19T05:42:39.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fc4f7fff640 1 -- 172.21.15.2:0/1766141146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f81ec6b0 con 0x7fc4f81577f0 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fc4f7fff640 1 -- 172.21.15.2:0/1766141146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f81ecc70 con 0x7fc4f81577f0 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fc4df7fe640 1 -- 172.21.15.2:0/1766141146 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ec017070 con 0x7fc4f81577f0 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fc4df7fe640 1 -- 172.21.15.2:0/1766141146 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec011790 con 0x7fc4f81577f0 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f876259c640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87641594f0 0x7f87641fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f8762d9d640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87641630b0 0x7f8764161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f8761d9b640 1 -- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87641630b0 msgr2=0x7f8764161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f8761d9b640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87641630b0 0x7f8764161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f8761d9b640 1 -- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87641594f0 msgr2=0x7f87641fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af4155290 con 0x7f2af40c6c40 2024-09-19T05:42:39.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f2aeaffd640 1 -- 172.21.15.2:0/2866555295 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af003c870 con 0x7f2af40c6c40 2024-09-19T05:42:39.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f8761d9b640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87641594f0 0x7f87641fde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27e67fc640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e80c7660 0x7f27e8160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80c9140 con 0x7f27e8155700 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80c8e60 con 0x7f27e80c6ed0 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.841+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f27e8155ae0 con 0x7f27e80c7660 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f27e67fc640 1 -- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e80c6ed0 msgr2=0x7f27e8200e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f27e67fc640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e80c6ed0 0x7f27e8200e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f27e67fc640 1 -- 172.21.15.2:0/1202260090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e8155700 msgr2=0x7f27e81ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f27e67fc640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e8155700 0x7f27e81ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f27e67fc640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e8156220 con 0x7f27e80c7660 2024-09-19T05:42:39.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f8761d9b640 1 -- 172.21.15.2:0/3177849243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87641fd3e0 con 0x7f8764162790 2024-09-19T05:42:39.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f876259c640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87641594f0 0x7f87641fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:39.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f95e77fe640 1 -- 172.21.15.2:0/301580979 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8007260 con 0x7f95e80063e0 2024-09-19T05:42:39.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fe1b5e4c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8049b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fe1b5e4c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a80e6270 0x7fe1a80e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fe1b5e4c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a80e6650 0x7fe1a80d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fe1b5e4c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a80457a0 con 0x7fe1a8045c30 2024-09-19T05:42:39.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fe1b5e4c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a804c460 con 0x7fe1a80e6270 2024-09-19T05:42:39.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7fe1b5e4c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a8044e70 con 0x7fe1a80e6650 2024-09-19T05:42:39.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b4e4a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8049b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b4e4a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8049b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37742/0 (socket says 172.21.15.2:37742) 2024-09-19T05:42:39.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b4e4a640 1 -- 172.21.15.2:0/917705410 learned_addr learned my addr 172.21.15.2:0/917705410 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b564b640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a80e6650 0x7fe1a80d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f27e67fc640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e80c7660 0x7f27e8160370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27e0007d80 tx=0x7f27e000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f2aeaffd640 1 -- 172.21.15.2:0/2866555295 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0037040 con 0x7f2af40c6c40 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0513fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc0039b0 0x7f050c053870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0513fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0ba390 0x7f050c049250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0513fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f050c0d5230 0x7f050c0d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0512ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc0039b0 0x7f050c053870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0513fff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f050c0dcf40 con 0x7f04fc0039b0 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0513fff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f050c006af0 con 0x7f050c0d5230 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0512ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc0039b0 0x7f050c053870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37728/0 (socket says 172.21.15.2:37728) 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0513fff640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f050c038300 con 0x7f050c0ba390 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f0512ffd640 1 -- 172.21.15.2:0/3008870988 learned_addr learned my addr 172.21.15.2:0/3008870988 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.844+0000 7f05137fe640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f050c0d5230 0x7f050c0d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f05127fc640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0ba390 0x7f050c049250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f0512ffd640 1 -- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0ba390 msgr2=0x7f050c049250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f0512ffd640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0ba390 0x7f050c049250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f0512ffd640 1 -- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f050c0d5230 msgr2=0x7f050c0d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f0512ffd640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f050c0d5230 0x7f050c0d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7f0512ffd640 1 -- 172.21.15.2:0/3008870988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c04df80 con 0x7f04fc0039b0 2024-09-19T05:42:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f0512ffd640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc0039b0 0x7f050c053870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f050402a510 tx=0x7f0504004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f04fb7fe640 1 -- 172.21.15.2:0/3008870988 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504042020 con 0x7f04fc0039b0 2024-09-19T05:42:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f04fb7fe640 1 -- 172.21.15.2:0/3008870988 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0504033bd0 con 0x7f04fc0039b0 2024-09-19T05:42:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b564b640 1 -- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a80e6270 msgr2=0x7fe1a80e4e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:42:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b564b640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a80e6270 0x7fe1a80e4e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b564b640 1 -- 172.21.15.2:0/917705410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 msgr2=0x7fe1a8049b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:39.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b564b640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8049b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:39.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f2aeaffd640 1 -- 172.21.15.2:0/2866555295 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2af0044070 con 0x7f2af40c6c40 2024-09-19T05:42:39.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.845+0000 7fe1b564b640 1 -- 172.21.15.2:0/917705410 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a80d6f80 con 0x7fe1a80e6650 2024-09-19T05:42:39.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7fe1b564b640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a80e6650 0x7fe1a80d7cb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe1a001d3d0 tx=0x7fe1a001d890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc005290 con 0x7f2af40c6c40 2024-09-19T05:42:39.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f8761d9b640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8764162790 0x7f8764154cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c7b0 tx=0x7f875c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f8752ffd640 1 -- 172.21.15.2:0/3177849243 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c018070 con 0x7f8764162790 2024-09-19T05:42:39.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7fe1ad7fa640 1 -- 172.21.15.2:0/917705410 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00050d0 con 0x7fe1a80e6650 2024-09-19T05:42:39.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f8752ffd640 1 -- 172.21.15.2:0/3177849243 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f875c00a640 con 0x7f8764162790 2024-09-19T05:42:39.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f8752ffd640 1 -- 172.21.15.2:0/3177849243 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c00aa30 con 0x7f8764162790 2024-09-19T05:42:39.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87641e9a50 con 0x7f8764162790 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87641ea010 con 0x7f8764162790 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f27cf7fe640 1 -- 172.21.15.2:0/1202260090 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e001a070 con 0x7f27e80c7660 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.846+0000 7f27cf7fe640 1 -- 172.21.15.2:0/1202260090 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e0003990 con 0x7f27e80c7660 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f27cf7fe640 1 -- 172.21.15.2:0/1202260090 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0003dc0 con 0x7f27e80c7660 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7fe1ad7fa640 1 -- 172.21.15.2:0/917705410 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a0005270 con 0x7fe1a80e6650 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7fe1ad7fa640 1 -- 172.21.15.2:0/917705410 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0007020 con 0x7fe1a80e6650 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a81791d0 con 0x7fe1a80e6650 2024-09-19T05:42:39.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a8179470 con 0x7fe1a80e6650 2024-09-19T05:42:39.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f96055f8640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96000c4010 con 0x7f95e80063e0 2024-09-19T05:42:39.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e8157070 con 0x7f27e80c7660 2024-09-19T05:42:39.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e8160d10 con 0x7f27e80c7660 2024-09-19T05:42:39.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f04fb7fe640 1 -- 172.21.15.2:0/3008870988 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504037040 con 0x7f04fc0039b0 2024-09-19T05:42:39.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7fc4f7fff640 1 -- 172.21.15.2:0/1766141146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d8005290 con 0x7fc4f81577f0 2024-09-19T05:42:39.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7fc4df7fe640 1 -- 172.21.15.2:0/1766141146 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc4ec011930 con 0x7fc4f81577f0 2024-09-19T05:42:39.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.847+0000 7f95e77fe640 1 -- 172.21.15.2:0/301580979 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f95f80074c0 con 0x7f95e80063e0 2024-09-19T05:42:39.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7f6adffff640 1 -- 172.21.15.2:0/1523760448 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6ae0018400 con 0x7f6ae8150d20 2024-09-19T05:42:39.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7f2aeaffd640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2ad806c3b0 0x7f2ad806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7fc4df7fe640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4e406c6b0 0x7fc4e406eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7fc4df7fe640 1 -- 172.21.15.2:0/1766141146 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc4ec011590 con 0x7fc4f81577f0 2024-09-19T05:42:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7f2aeaffd640 1 -- 172.21.15.2:0/2866555295 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2af00abbf0 con 0x7f2af40c6c40 2024-09-19T05:42:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7f6adffff640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ac806c380 0x7f6ac806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876406c730 con 0x7f8764162790 2024-09-19T05:42:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7fc4f6ffd640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4e406c6b0 0x7fc4e406eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050c053db0 con 0x7f04fc0039b0 2024-09-19T05:42:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7f6adffff640 1 -- 172.21.15.2:0/1523760448 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6ae008b470 con 0x7f6ae8150d20 2024-09-19T05:42:39.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7f6ae6ffd640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ac806c380 0x7f6ac806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7f95e77fe640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95ec06c3b0 0x7f95ec06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.848+0000 7f95fe7fc640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95ec06c3b0 0x7f95ec06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7f95e77fe640 1 -- 172.21.15.2:0/301580979 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f95f808ae70 con 0x7f95e80063e0 2024-09-19T05:42:39.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7f2af9a3f640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2ad806c3b0 0x7f2ad806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050c0d6880 con 0x7f04fc0039b0 2024-09-19T05:42:39.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7fe1ad7fa640 1 -- 172.21.15.2:0/917705410 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe1a00071c0 con 0x7fe1a80e6650 2024-09-19T05:42:39.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7fc4f6ffd640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4e406c6b0 0x7fc4e406eb70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8005880 tx=0x7fc4e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.849+0000 7f95fe7fc640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95ec06c3b0 0x7f95ec06e870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f960006e230 tx=0x7f95f4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7f2af9a3f640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2ad806c3b0 0x7f2ad806e870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007fd0 tx=0x7f2aec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7f6ae6ffd640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ac806c380 0x7f6ac806e840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000b7f0 tx=0x7f6ad0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7fe1ad7fa640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe19c06c380 0x7fe19c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7fe1ad7fa640 1 -- 172.21.15.2:0/917705410 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe1a009b430 con 0x7fe1a80e6650 2024-09-19T05:42:39.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e8162d90 con 0x7f27e80c7660 2024-09-19T05:42:39.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7fe1b4e4a640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe19c06c380 0x7fe19c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe190005290 con 0x7fe1a80e6650 2024-09-19T05:42:39.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7fe1b4e4a640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe19c06c380 0x7fe19c06e840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0076640 tx=0x7fe1b006e040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.850+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050c04e150 con 0x7f04fc0039b0 2024-09-19T05:42:39.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.853+0000 7fc4df7fe640 1 -- 172.21.15.2:0/1766141146 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc4ec0592c0 con 0x7fc4f81577f0 2024-09-19T05:42:39.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.853+0000 7f95e77fe640 1 -- 172.21.15.2:0/301580979 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f96000c4010 con 0x7f95e80063e0 2024-09-19T05:42:39.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.854+0000 7f6adffff640 1 -- 172.21.15.2:0/1523760448 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6ae0014030 con 0x7f6ae8150d20 2024-09-19T05:42:39.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.855+0000 7f27cf7fe640 1 -- 172.21.15.2:0/1202260090 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f27e0005d10 con 0x7f27e80c7660 2024-09-19T05:42:39.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.855+0000 7f27cf7fe640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f27d406c330 0x7f27d406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.856+0000 7f27e6ffd640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f27d406c330 0x7f27d406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.856+0000 7f27e6ffd640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f27d406c330 0x7f27d406e7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009a30 tx=0x7f27dc009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.856+0000 7f8752ffd640 1 -- 172.21.15.2:0/3177849243 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f875c01b070 con 0x7f8764162790 2024-09-19T05:42:39.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.857+0000 7f04fb7fe640 1 -- 172.21.15.2:0/3008870988 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0504045020 con 0x7f04fc0039b0 2024-09-19T05:42:39.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.857+0000 7f8752ffd640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f874406c330 0x7f874406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.857+0000 7f8752ffd640 1 -- 172.21.15.2:0/3177849243 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f875c08af10 con 0x7f8764162790 2024-09-19T05:42:39.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.857+0000 7f876259c640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f874406c330 0x7f874406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.857+0000 7f04fb7fe640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f050006c380 0x7f050006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:39.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.857+0000 7f04fb7fe640 1 -- 172.21.15.2:0/3008870988 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f05040abcf0 con 0x7f04fc0039b0 2024-09-19T05:42:39.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.858+0000 7f05127fc640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f050006c380 0x7f050006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:39.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.858+0000 7f2aeaffd640 1 -- 172.21.15.2:0/2866555295 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2af0079cf0 con 0x7f2af40c6c40 2024-09-19T05:42:39.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.858+0000 7f05127fc640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f050006c380 0x7f050006e840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05140504d0 tx=0x7f051406e040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.858+0000 7f876259c640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f874406c330 0x7f874406e7f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8754034000 tx=0x7f875402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:39.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.863+0000 7f27cf7fe640 1 -- 172.21.15.2:0/1202260090 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f27e0057df0 con 0x7f27e80c7660 2024-09-19T05:42:39.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.863+0000 7fe1ad7fa640 1 -- 172.21.15.2:0/917705410 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe1a80d6f80 con 0x7fe1a80e6650 2024-09-19T05:42:39.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.866+0000 7f8752ffd640 1 -- 172.21.15.2:0/3177849243 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f875c022020 con 0x7f8764162790 2024-09-19T05:42:39.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.867+0000 7f27cf7fe640 1 -- 172.21.15.2:0/1202260090 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f27e0059d50 con 0x7f27e80c7660 2024-09-19T05:42:39.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:39.874+0000 7f04fb7fe640 1 -- 172.21.15.2:0/3008870988 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f050404c030 con 0x7f04fc0039b0 2024-09-19T05:42:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.262+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2adc005580 con 0x7f2af40c6c40 2024-09-19T05:42:40.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.262+0000 7f2aeaffd640 1 -- 172.21.15.2:0/2866555295 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2af007dff0 con 0x7f2af40c6c40 2024-09-19T05:42:40.265 INFO:teuthology.orchestra.run.smithi002.stdout:64424509446 2024-09-19T05:42:40.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.264+0000 7fc4f7fff640 1 -- 172.21.15.2:0/1766141146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc4d8005580 con 0x7fc4f81577f0 2024-09-19T05:42:40.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.264+0000 7fc4df7fe640 1 -- 172.21.15.2:0/1766141146 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4ec058bc0 con 0x7fc4f81577f0 2024-09-19T05:42:40.266 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.266+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2ad806c3b0 msgr2=0x7f2ad806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.266+0000 7f2afb242640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2ad806c3b0 0x7f2ad806e870 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2aec007fd0 tx=0x7f2aec009040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.266+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 msgr2=0x7f2af414d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.266+0000 7f2afb242640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af414d1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2af002a910 tx=0x7f2af00046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.266+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 shutdown_connections 2024-09-19T05:42:40.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.266+0000 7f2afb242640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2ad806c3b0 0x7f2ad806e870 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7f2afb242640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2af4153410 0x7f2af4150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7f2afb242640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2af40c7020 0x7f2af414d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7f2afb242640 1 --2- 172.21.15.2:0/2866555295 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2af40c6c40 0x7f2af414d1a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 >> 172.21.15.2:0/2866555295 conn(0x7f2af406bc10 msgr2=0x7f2af415e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7f96055f8640 1 -- 172.21.15.2:0/301580979 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f9600150da0 con 0x7f95e80063e0 2024-09-19T05:42:40.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 shutdown_connections 2024-09-19T05:42:40.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7f2afb242640 1 -- 172.21.15.2:0/2866555295 wait complete. 2024-09-19T05:42:40.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7fc4dd7fa640 1 -- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4e406c6b0 msgr2=0x7fc4e406eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.267+0000 7fc4dd7fa640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4e406c6b0 0x7fc4e406eb70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8005880 tx=0x7fc4e8009040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 -- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4f81577f0 msgr2=0x7fc4f8150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7f95e77fe640 1 -- 172.21.15.2:0/301580979 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f95f8020020 con 0x7f95e80063e0 2024-09-19T05:42:40.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4f81577f0 0x7fc4f8150a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c3e0 tx=0x7fc4ec00c8a0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.272 INFO:teuthology.orchestra.run.smithi002.stdout:55834574854 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 -- 172.21.15.2:0/1766141146 shutdown_connections 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4e406c6b0 0x7fc4e406eb70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4f81577f0 0x7fc4f8150a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4f80c6fb0 0x7fc4f81504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 --2- 172.21.15.2:0/1766141146 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4f806c700 0x7fc4f8146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 -- 172.21.15.2:0/1766141146 >> 172.21.15.2:0/1766141146 conn(0x7fc4f806bc10 msgr2=0x7fc4f8155410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.268+0000 7fc4dd7fa640 1 -- 172.21.15.2:0/1766141146 shutdown_connections 2024-09-19T05:42:40.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.269+0000 7fc4dd7fa640 1 -- 172.21.15.2:0/1766141146 wait complete. 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.272+0000 7f95e57fa640 1 -- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95ec06c3b0 msgr2=0x7f95ec06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.272+0000 7f95e57fa640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95ec06c3b0 0x7f95ec06e870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f960006e230 tx=0x7f95f4008040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.272+0000 7f95e57fa640 1 -- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 msgr2=0x7f9600151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.272+0000 7f95e57fa640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f9600151420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c530 tx=0x7f95f800c9f0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.272+0000 7f95e57fa640 1 -- 172.21.15.2:0/301580979 shutdown_connections 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f95e57fa640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95ec06c3b0 0x7f95ec06e870 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f95e57fa640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9600159b10 0x7f96001582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f95e57fa640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96001f4740 0x7f96001587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f27e80c8100 con 0x7f27e80c7660 2024-09-19T05:42:40.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f95e57fa640 1 --2- 172.21.15.2:0/301580979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e80063e0 0x7f9600151420 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f95e57fa640 1 -- 172.21.15.2:0/301580979 >> 172.21.15.2:0/301580979 conn(0x7f960006f980 msgr2=0x7f9600074d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f95e57fa640 1 -- 172.21.15.2:0/301580979 shutdown_connections 2024-09-19T05:42:40.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f27cf7fe640 1 -- 172.21.15.2:0/1202260090 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27e005e050 con 0x7f27e80c7660 2024-09-19T05:42:40.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.273+0000 7f95e57fa640 1 -- 172.21.15.2:0/301580979 wait complete. 2024-09-19T05:42:40.277 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:40.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.274+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6acc005580 con 0x7f6ae8150d20 2024-09-19T05:42:40.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.274+0000 7f6adffff640 1 -- 172.21.15.2:0/1523760448 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ae0059bc0 con 0x7f6ae8150d20 2024-09-19T05:42:40.278 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-19T05:42:40.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.277+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f27d406c330 msgr2=0x7f27d406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.277+0000 7f27ed423640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f27d406c330 0x7f27d406e7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009a30 tx=0x7f27dc009190 comp rx=0 tx=0).stop 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e80c7660 msgr2=0x7f27e8160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e80c7660 0x7f27e8160370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f27e0007d80 tx=0x7f27e000e4b0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 shutdown_connections 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f27d406c330 0x7f27d406e7f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27e8155700 0x7f27e81ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f27e80c7660 0x7f27e8160370 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 --2- 172.21.15.2:0/1202260090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27e80c6ed0 0x7f27e8200e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 >> 172.21.15.2:0/1202260090 conn(0x7f27e80ccc60 msgr2=0x7f27e81503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 shutdown_connections 2024-09-19T05:42:40.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.278+0000 7f27ed423640 1 -- 172.21.15.2:0/1202260090 wait complete. 2024-09-19T05:42:40.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.279+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8764073b90 con 0x7f8764162790 2024-09-19T05:42:40.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.279+0000 7f8752ffd640 1 -- 172.21.15.2:0/3177849243 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f87641fd3e0 con 0x7f8764162790 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.279+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ac806c380 msgr2=0x7f6ac806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.279+0000 7f6addffb640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ac806c380 0x7f6ac806e840 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000b7f0 tx=0x7f6ad0008040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.279+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8150d20 msgr2=0x7f6ae814f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.279+0000 7f6addffb640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8150d20 0x7f6ae814f5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c880 tx=0x7f6ae000cd40 comp rx=0 tx=0).stop 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.280+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 shutdown_connections 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.280+0000 7f6addffb640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ac806c380 0x7f6ac806e840 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000b7f0 tx=0x7f6ad0008040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.280+0000 7f6addffb640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8072650 0x7f6ae814bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.280+0000 7f6addffb640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ae8150d20 0x7f6ae814f5c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.280+0000 7f6addffb640 1 --2- 172.21.15.2:0/1523760448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6ae8072090 0x7f6ae81f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.280+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 >> 172.21.15.2:0/1523760448 conn(0x7f6ae806bc10 msgr2=0x7f6ae80c2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.284 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:40.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.281+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 shutdown_connections 2024-09-19T05:42:40.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.281+0000 7f6addffb640 1 -- 172.21.15.2:0/1523760448 wait complete. 2024-09-19T05:42:40.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f874406c330 msgr2=0x7f874406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f8768991640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f874406c330 0x7f874406e7f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8754034000 tx=0x7f875402e000 comp rx=0 tx=0).stop 2024-09-19T05:42:40.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f050c04fbb0 con 0x7f04fc0039b0 2024-09-19T05:42:40.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8764162790 msgr2=0x7f8764154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f8768991640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8764162790 0x7f8764154cf0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f875c00c7b0 tx=0x7f875c00cc70 comp rx=0 tx=0).stop 2024-09-19T05:42:40.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 shutdown_connections 2024-09-19T05:42:40.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f8768991640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f874406c330 0x7f874406e7f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7f8768991640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f87641630b0 0x7f8764161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7f8768991640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8764162790 0x7f8764154cf0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7f8768991640 1 --2- 172.21.15.2:0/3177849243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87641594f0 0x7f87641fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 >> 172.21.15.2:0/3177849243 conn(0x7f876406bc10 msgr2=0x7f8764156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.283+0000 7f04fb7fe640 1 -- 172.21.15.2:0/3008870988 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0504079d60 con 0x7f04fc0039b0 2024-09-19T05:42:40.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe190005580 con 0x7fe1a80e6650 2024-09-19T05:42:40.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 shutdown_connections 2024-09-19T05:42:40.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7f8768991640 1 -- 172.21.15.2:0/3177849243 wait complete. 2024-09-19T05:42:40.288 INFO:teuthology.orchestra.run.smithi002.stdout:60129542150 2024-09-19T05:42:40.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.284+0000 7fe1ad7fa640 1 -- 172.21.15.2:0/917705410 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1a0069530 con 0x7fe1a80e6650 2024-09-19T05:42:40.289 INFO:teuthology.orchestra.run.smithi002.stdout:60129542149 2024-09-19T05:42:40.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.288+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe19c06c380 msgr2=0x7fe19c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.288+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe19c06c380 0x7fe19c06e840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0076640 tx=0x7fe1b006e040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.288+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a80e6650 msgr2=0x7fe1a80d7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.288+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a80e6650 0x7fe1a80d7cb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe1a001d3d0 tx=0x7fe1a001d890 comp rx=0 tx=0).stop 2024-09-19T05:42:40.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-09-19T05:42:40.292 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-19T05:42:40.293 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 shutdown_connections 2024-09-19T05:42:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe19c06c380 0x7fe19c06e840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1a80e6270 0x7fe1a80e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1a80e6650 0x7fe1a80d7cb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 --2- 172.21.15.2:0/917705410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1a8045c30 0x7fe1a8049b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 >> 172.21.15.2:0/917705410 conn(0x7fe1a804fc50 msgr2=0x7fe1a8006560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f050006c380 msgr2=0x7f050006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f050006c380 0x7f050006e840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05140504d0 tx=0x7f051406e040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 shutdown_connections 2024-09-19T05:42:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7fe1b5e4c640 1 -- 172.21.15.2:0/917705410 wait complete. 2024-09-19T05:42:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc0039b0 msgr2=0x7f050c053870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc0039b0 0x7f050c053870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f050402a510 tx=0x7f0504004730 comp rx=0 tx=0).stop 2024-09-19T05:42:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 shutdown_connections 2024-09-19T05:42:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f050006c380 0x7f050006e840 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f050c0d5230 0x7f050c0d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f050c0ba390 0x7f050c049250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 --2- 172.21.15.2:0/3008870988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04fc0039b0 0x7f050c053870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 >> 172.21.15.2:0/3008870988 conn(0x7f050c054060 msgr2=0x7f050c0d9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 shutdown_connections 2024-09-19T05:42:40.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.289+0000 7f0513fff640 1 -- 172.21.15.2:0/3008870988 wait complete. 2024-09-19T05:42:40.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-09-19T05:42:40.297 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-09-19T05:42:40.303 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2024-09-19T05:42:40.304 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2024-09-19T05:42:40.306 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2024-09-19T05:42:40.307 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-09-19T05:42:40.309 DEBUG:teuthology.parallel:result is None 2024-09-19T05:42:40.309 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T05:42:40.310 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:42:40.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.449+0000 7f732ac65640 1 Processor -- start 2024-09-19T05:42:40.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.449+0000 7f732ac65640 1 -- start start 2024-09-19T05:42:40.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f732ac65640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f73240cc5f0 0x7f732415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f732ac65640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73240734c0 con 0x7f7324154cb0 2024-09-19T05:42:40.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f732ac65640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7324073780 con 0x7f73240cd7c0 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f732ac65640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f73240cd330 con 0x7f73240cc5f0 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f7328a01640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f7324154cb0 0x7f73240c55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51614/0 (socket says 172.21.15.2:51614) 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f7328a01640 1 -- 172.21.15.2:0/2512814288 learned_addr learned my addr 172.21.15.2:0/2512814288 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731bfff640 1 --2- 172.21.15.2:0/2512814288 >> v2:172.21.15.123:3300/0 conn(0x7f73240cc5f0 0x7f732415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554044214 0 0) 0x7f73240734c0 con 0x7f7324154cb0 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f730c003520 con 0x7f7324154cb0 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1181587805 0 0) 0x7f730c003520 con 0x7f7324154cb0 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 >> v2:172.21.15.123:3300/0 conn(0x7f73240cc5f0 msgr2=0x7f732415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:40.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 --2- 172.21.15.2:0/2512814288 >> v2:172.21.15.123:3300/0 conn(0x7f73240cc5f0 0x7f732415a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 >> v1:172.21.15.123:6789/0 conn(0x7f73240cd7c0 legacy=0x7f73240cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7324073a60 con 0x7f7324154cb0 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.450+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f7320001670 con 0x7f7324154cb0 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 >> v1:172.21.15.2:6789/0 conn(0x7f7324154cb0 legacy=0x7f73240c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731b7fe640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c003a20 0x7f730c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731b7fe640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f730c004800 0x7f730c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731b7fe640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c0051c0 0x7f730c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f73240cd330 con 0x7f730c004800 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f730c00c0d0 con 0x7f730c003a20 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f730c00c2d0 con 0x7f730c0051c0 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731bfff640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f730c004800 0x7f730c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f7328a01640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c003a20 0x7f730c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f7329202640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c0051c0 0x7f730c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731bfff640 1 -- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c0051c0 msgr2=0x7f730c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731bfff640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c0051c0 0x7f730c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731bfff640 1 -- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c003a20 msgr2=0x7f730c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731bfff640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f730c003a20 0x7f730c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f731bfff640 1 -- 172.21.15.2:0/2512814288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c003520 con 0x7f730c004800 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.451+0000 7f7329202640 1 -- 172.21.15.2:0/2512814288 reap_dead start 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.452+0000 7f731bfff640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f730c004800 0x7f730c004c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7314009ac0 tx=0x7f730c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a740379a37aa93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.452+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314036070 con 0x7f730c004800 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.452+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7314031400 con 0x7f730c004800 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.452+0000 7f731b7fe640 1 -- 172.21.15.2:0/2512814288 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314009690 con 0x7f730c004800 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.452+0000 7f732ac65640 1 -- 172.21.15.2:0/2512814288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f730c004800 msgr2=0x7f730c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.452+0000 7f732ac65640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f730c004800 0x7f730c004c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7314009ac0 tx=0x7f730c005830 comp rx=0 tx=0).stop 2024-09-19T05:42:40.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.453+0000 7f732ac65640 1 -- 172.21.15.2:0/2512814288 shutdown_connections 2024-09-19T05:42:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.453+0000 7f732ac65640 1 --2- 172.21.15.2:0/2512814288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f730c004800 0x7f730c004c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.453+0000 7f732ac65640 1 -- 172.21.15.2:0/2512814288 >> 172.21.15.2:0/2512814288 conn(0x7f732406d0c0 msgr2=0x7f732406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.453+0000 7f732ac65640 1 -- 172.21.15.2:0/2512814288 shutdown_connections 2024-09-19T05:42:40.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.453+0000 7f732ac65640 1 -- 172.21.15.2:0/2512814288 wait complete. 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.454+0000 7f732ac65640 1 Processor -- start 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.454+0000 7f732ac65640 1 -- start start 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f732ac65640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c004800 0x7f73240c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f732ac65640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73240b5c10 0x7f73240cf480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f732ac65640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73241e5ef0 0x7f732414abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f732ac65640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f73240cdbf0 con 0x7f73241e5ef0 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f732ac65640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73240b30d0 con 0x7f73240b5c10 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f732ac65640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f732415a810 con 0x7f730c004800 2024-09-19T05:42:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73241e5ef0 0x7f732414abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73241e5ef0 0x7f732414abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37758/0 (socket says 172.21.15.2:37758) 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 -- 172.21.15.2:0/3756561472 learned_addr learned my addr 172.21.15.2:0/3756561472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f731bfff640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73240b5c10 0x7f73240cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7328a01640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c004800 0x7f73240c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 -- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c004800 msgr2=0x7f73240c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c004800 0x7f73240c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 -- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73240b5c10 msgr2=0x7f73240cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73240b5c10 0x7f73240cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 -- 172.21.15.2:0/3756561472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73240c96b0 con 0x7f73241e5ef0 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f7329202640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73241e5ef0 0x7f732414abe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f73240c6e50 tx=0x7f731c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f73197fa640 1 -- 172.21.15.2:0/3756561472 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c003970 con 0x7f73241e5ef0 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f73197fa640 1 -- 172.21.15.2:0/3756561472 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c018070 con 0x7f73241e5ef0 2024-09-19T05:42:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73240c8e10 con 0x7f73241e5ef0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.455+0000 7f73197fa640 1 -- 172.21.15.2:0/3756561472 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c003d70 con 0x7f73241e5ef0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.456+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73240c93d0 con 0x7f73241e5ef0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.456+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73240c55f0 con 0x7f73241e5ef0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.457+0000 7f73197fa640 1 -- 172.21.15.2:0/3756561472 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f731c0050d0 con 0x7f73241e5ef0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.457+0000 7f73197fa640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f731006c2b0 0x7f731006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.457+0000 7f73197fa640 1 -- 172.21.15.2:0/3756561472 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f731c08b110 con 0x7f73241e5ef0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.458+0000 7f7328a01640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f731006c2b0 0x7f731006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.458+0000 7f7328a01640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f731006c2b0 0x7f731006e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73200011d0 tx=0x7f7320009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:40.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.462+0000 7f73197fa640 1 -- 172.21.15.2:0/3756561472 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f731c059910 con 0x7f73241e5ef0 2024-09-19T05:42:40.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.634+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7324159830 con 0x7f731006c2b0 2024-09-19T05:42:40.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.641+0000 7f73197fa640 1 -- 172.21.15.2:0/3756561472 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252004 (secure 0 0 0) 0x7f7324159830 con 0x7f731006c2b0 2024-09-19T05:42:40.645 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:40.646 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:42:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f731006c2b0 msgr2=0x7f731006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f731006c2b0 0x7f731006e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f73200011d0 tx=0x7f7320009040 comp rx=0 tx=0).stop 2024-09-19T05:42:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73241e5ef0 msgr2=0x7f732414abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73241e5ef0 0x7f732414abe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f73240c6e50 tx=0x7f731c00eb90 comp rx=0 tx=0).stop 2024-09-19T05:42:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 shutdown_connections 2024-09-19T05:42:40.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f731006c2b0 0x7f731006e770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73240b5c10 0x7f73240cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73241e5ef0 0x7f732414abe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 --2- 172.21.15.2:0/3756561472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f730c004800 0x7f73240c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 >> 172.21.15.2:0/3756561472 conn(0x7f732406d0c0 msgr2=0x7f73240caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 shutdown_connections 2024-09-19T05:42:40.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.644+0000 7f732ac65640 1 -- 172.21.15.2:0/3756561472 wait complete. 2024-09-19T05:42:40.657 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-19T05:42:39.820905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234796,"kb_used_data":1952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1998848,"data_stored":1818484,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.851188"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058235+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:42:34.058235+0000","last_peered":"2024-09-19T05:42:34.058235+0000","last_clean":"2024-09-19T05:42:34.058235+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:42:34.058235+0000","last_undegraded":"2024-09-19T05:42:34.058235+0000","last_fullsized":"2024-09-19T05:42:34.058235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058494+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:42:34.058494+0000","last_peered":"2024-09-19T05:42:34.058494+0000","last_clean":"2024-09-19T05:42:34.058494+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:42:34.058494+0000","last_undegraded":"2024-09-19T05:42:34.058494+0000","last_fullsized":"2024-09-19T05:42:34.058494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058017+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:42:34.058017+0000","last_peered":"2024-09-19T05:42:34.058017+0000","last_clean":"2024-09-19T05:42:34.058017+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:42:34.058017+0000","last_undegraded":"2024-09-19T05:42:34.058017+0000","last_fullsized":"2024-09-19T05:42:34.058017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923708+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:42:33.923708+0000","last_peered":"2024-09-19T05:42:33.923708+0000","last_clean":"2024-09-19T05:42:33.923708+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:42:33.923708+0000","last_undegraded":"2024-09-19T05:42:33.923708+0000","last_fullsized":"2024-09-19T05:42:33.923708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057846+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:42:34.057846+0000","last_peered":"2024-09-19T05:42:34.057846+0000","last_clean":"2024-09-19T05:42:34.057846+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:42:34.057846+0000","last_undegraded":"2024-09-19T05:42:34.057846+0000","last_fullsized":"2024-09-19T05:42:34.057846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923679+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:42:33.923679+0000","last_peered":"2024-09-19T05:42:33.923679+0000","last_clean":"2024-09-19T05:42:33.923679+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:42:33.923679+0000","last_undegraded":"2024-09-19T05:42:33.923679+0000","last_fullsized":"2024-09-19T05:42:33.923679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057764+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:42:34.057764+0000","last_peered":"2024-09-19T05:42:34.057764+0000","last_clean":"2024-09-19T05:42:34.057764+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:42:34.057764+0000","last_undegraded":"2024-09-19T05:42:34.057764+0000","last_fullsized":"2024-09-19T05:42:34.057764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058259+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:42:34.058259+0000","last_peered":"2024-09-19T05:42:34.058259+0000","last_clean":"2024-09-19T05:42:34.058259+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:42:34.058259+0000","last_undegraded":"2024-09-19T05:42:34.058259+0000","last_fullsized":"2024-09-19T05:42:34.058259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058756+0000","last_change":"2024-09-19T05:42:32.945872+0000","last_active":"2024-09-19T05:42:34.058756+0000","last_peered":"2024-09-19T05:42:34.058756+0000","last_clean":"2024-09-19T05:42:34.058756+0000","last_became_active":"2024-09-19T05:42:32.945545+0000","last_became_peered":"2024-09-19T05:42:32.945545+0000","last_unstale":"2024-09-19T05:42:34.058756+0000","last_undegraded":"2024-09-19T05:42:34.058756+0000","last_fullsized":"2024-09-19T05:42:34.058756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058073+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:42:34.058073+0000","last_peered":"2024-09-19T05:42:34.058073+0000","last_clean":"2024-09-19T05:42:34.058073+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:42:34.058073+0000","last_undegraded":"2024-09-19T05:42:34.058073+0000","last_fullsized":"2024-09-19T05:42:34.058073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058096+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:42:34.058096+0000","last_peered":"2024-09-19T05:42:34.058096+0000","last_clean":"2024-09-19T05:42:34.058096+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:42:34.058096+0000","last_undegraded":"2024-09-19T05:42:34.058096+0000","last_fullsized":"2024-09-19T05:42:34.058096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058167+0000","last_change":"2024-09-19T05:42:32.941601+0000","last_active":"2024-09-19T05:42:34.058167+0000","last_peered":"2024-09-19T05:42:34.058167+0000","last_clean":"2024-09-19T05:42:34.058167+0000","last_became_active":"2024-09-19T05:42:32.941288+0000","last_became_peered":"2024-09-19T05:42:32.941288+0000","last_unstale":"2024-09-19T05:42:34.058167+0000","last_undegraded":"2024-09-19T05:42:34.058167+0000","last_fullsized":"2024-09-19T05:42:34.058167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:35.670719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058583+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:42:34.058583+0000","last_peered":"2024-09-19T05:42:34.058583+0000","last_clean":"2024-09-19T05:42:34.058583+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:42:34.058583+0000","last_undegraded":"2024-09-19T05:42:34.058583+0000","last_fullsized":"2024-09-19T05:42:34.058583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058482+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:42:34.058482+0000","last_peered":"2024-09-19T05:42:34.058482+0000","last_clean":"2024-09-19T05:42:34.058482+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:42:34.058482+0000","last_undegraded":"2024-09-19T05:42:34.058482+0000","last_fullsized":"2024-09-19T05:42:34.058482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058261+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:42:34.058261+0000","last_peered":"2024-09-19T05:42:34.058261+0000","last_clean":"2024-09-19T05:42:34.058261+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:42:34.058261+0000","last_undegraded":"2024-09-19T05:42:34.058261+0000","last_fullsized":"2024-09-19T05:42:34.058261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057516+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:42:34.057516+0000","last_peered":"2024-09-19T05:42:34.057516+0000","last_clean":"2024-09-19T05:42:34.057516+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:42:34.057516+0000","last_undegraded":"2024-09-19T05:42:34.057516+0000","last_fullsized":"2024-09-19T05:42:34.057516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059660+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:42:34.059660+0000","last_peered":"2024-09-19T05:42:34.059660+0000","last_clean":"2024-09-19T05:42:34.059660+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:42:34.059660+0000","last_undegraded":"2024-09-19T05:42:34.059660+0000","last_fullsized":"2024-09-19T05:42:34.059660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059534+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:42:34.059534+0000","last_peered":"2024-09-19T05:42:34.059534+0000","last_clean":"2024-09-19T05:42:34.059534+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:42:34.059534+0000","last_undegraded":"2024-09-19T05:42:34.059534+0000","last_fullsized":"2024-09-19T05:42:34.059534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058089+0000","last_change":"2024-09-19T05:42:32.940627+0000","last_active":"2024-09-19T05:42:34.058089+0000","last_peered":"2024-09-19T05:42:34.058089+0000","last_clean":"2024-09-19T05:42:34.058089+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058089+0000","last_undegraded":"2024-09-19T05:42:34.058089+0000","last_fullsized":"2024-09-19T05:42:34.058089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.616840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925755+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:42:33.925755+0000","last_peered":"2024-09-19T05:42:33.925755+0000","last_clean":"2024-09-19T05:42:33.925755+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:42:33.925755+0000","last_undegraded":"2024-09-19T05:42:33.925755+0000","last_fullsized":"2024-09-19T05:42:33.925755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058013+0000","last_change":"2024-09-19T05:42:32.940425+0000","last_active":"2024-09-19T05:42:34.058013+0000","last_peered":"2024-09-19T05:42:34.058013+0000","last_clean":"2024-09-19T05:42:34.058013+0000","last_became_active":"2024-09-19T05:42:32.940059+0000","last_became_peered":"2024-09-19T05:42:32.940059+0000","last_unstale":"2024-09-19T05:42:34.058013+0000","last_undegraded":"2024-09-19T05:42:34.058013+0000","last_fullsized":"2024-09-19T05:42:34.058013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058206+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:42:34.058206+0000","last_peered":"2024-09-19T05:42:34.058206+0000","last_clean":"2024-09-19T05:42:34.058206+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:42:34.058206+0000","last_undegraded":"2024-09-19T05:42:34.058206+0000","last_fullsized":"2024-09-19T05:42:34.058206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057512+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:42:34.057512+0000","last_peered":"2024-09-19T05:42:34.057512+0000","last_clean":"2024-09-19T05:42:34.057512+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:42:34.057512+0000","last_undegraded":"2024-09-19T05:42:34.057512+0000","last_fullsized":"2024-09-19T05:42:34.057512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058955+0000","last_change":"2024-09-19T05:42:32.941414+0000","last_active":"2024-09-19T05:42:34.058955+0000","last_peered":"2024-09-19T05:42:34.058955+0000","last_clean":"2024-09-19T05:42:34.058955+0000","last_became_active":"2024-09-19T05:42:32.941076+0000","last_became_peered":"2024-09-19T05:42:32.941076+0000","last_unstale":"2024-09-19T05:42:34.058955+0000","last_undegraded":"2024-09-19T05:42:34.058955+0000","last_fullsized":"2024-09-19T05:42:34.058955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:13.371046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923839+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:42:33.923839+0000","last_peered":"2024-09-19T05:42:33.923839+0000","last_clean":"2024-09-19T05:42:33.923839+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:42:33.923839+0000","last_undegraded":"2024-09-19T05:42:33.923839+0000","last_fullsized":"2024-09-19T05:42:33.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058867+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:42:34.058867+0000","last_peered":"2024-09-19T05:42:34.058867+0000","last_clean":"2024-09-19T05:42:34.058867+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:42:34.058867+0000","last_undegraded":"2024-09-19T05:42:34.058867+0000","last_fullsized":"2024-09-19T05:42:34.058867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058377+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:42:34.058377+0000","last_peered":"2024-09-19T05:42:34.058377+0000","last_clean":"2024-09-19T05:42:34.058377+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:42:34.058377+0000","last_undegraded":"2024-09-19T05:42:34.058377+0000","last_fullsized":"2024-09-19T05:42:34.058377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059114+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:42:34.059114+0000","last_peered":"2024-09-19T05:42:34.059114+0000","last_clean":"2024-09-19T05:42:34.059114+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:42:34.059114+0000","last_undegraded":"2024-09-19T05:42:34.059114+0000","last_fullsized":"2024-09-19T05:42:34.059114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924874+0000","last_change":"2024-09-19T05:42:32.927184+0000","last_active":"2024-09-19T05:42:33.924874+0000","last_peered":"2024-09-19T05:42:33.924874+0000","last_clean":"2024-09-19T05:42:33.924874+0000","last_became_active":"2024-09-19T05:42:32.926952+0000","last_became_peered":"2024-09-19T05:42:32.926952+0000","last_unstale":"2024-09-19T05:42:33.924874+0000","last_undegraded":"2024-09-19T05:42:33.924874+0000","last_fullsized":"2024-09-19T05:42:33.924874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923475+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:42:33.923475+0000","last_peered":"2024-09-19T05:42:33.923475+0000","last_clean":"2024-09-19T05:42:33.923475+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:42:33.923475+0000","last_undegraded":"2024-09-19T05:42:33.923475+0000","last_fullsized":"2024-09-19T05:42:33.923475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058161+0000","last_change":"2024-09-19T05:42:32.942584+0000","last_active":"2024-09-19T05:42:34.058161+0000","last_peered":"2024-09-19T05:42:34.058161+0000","last_clean":"2024-09-19T05:42:34.058161+0000","last_became_active":"2024-09-19T05:42:32.942274+0000","last_became_peered":"2024-09-19T05:42:32.942274+0000","last_unstale":"2024-09-19T05:42:34.058161+0000","last_undegraded":"2024-09-19T05:42:34.058161+0000","last_fullsized":"2024-09-19T05:42:34.058161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832292+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:42:34.832292+0000","last_peered":"2024-09-19T05:42:34.832292+0000","last_clean":"2024-09-19T05:42:34.832292+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:42:34.832292+0000","last_undegraded":"2024-09-19T05:42:34.832292+0000","last_fullsized":"2024-09-19T05:42:34.832292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925748+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:42:33.925748+0000","last_peered":"2024-09-19T05:42:33.925748+0000","last_clean":"2024-09-19T05:42:33.925748+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:42:33.925748+0000","last_undegraded":"2024-09-19T05:42:33.925748+0000","last_fullsized":"2024-09-19T05:42:33.925748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.839402+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:42:34.839402+0000","last_peered":"2024-09-19T05:42:34.839402+0000","last_clean":"2024-09-19T05:42:34.839402+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:42:34.839402+0000","last_undegraded":"2024-09-19T05:42:34.839402+0000","last_fullsized":"2024-09-19T05:42:34.839402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058211+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:42:34.058211+0000","last_peered":"2024-09-19T05:42:34.058211+0000","last_clean":"2024-09-19T05:42:34.058211+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:42:34.058211+0000","last_undegraded":"2024-09-19T05:42:34.058211+0000","last_fullsized":"2024-09-19T05:42:34.058211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058257+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:42:34.058257+0000","last_peered":"2024-09-19T05:42:34.058257+0000","last_clean":"2024-09-19T05:42:34.058257+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058257+0000","last_undegraded":"2024-09-19T05:42:34.058257+0000","last_fullsized":"2024-09-19T05:42:34.058257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057643+0000","last_change":"2024-09-19T05:42:31.995072+0000","last_active":"2024-09-19T05:42:34.057643+0000","last_peered":"2024-09-19T05:42:34.057643+0000","last_clean":"2024-09-19T05:42:34.057643+0000","last_became_active":"2024-09-19T05:42:31.994586+0000","last_became_peered":"2024-09-19T05:42:31.994586+0000","last_unstale":"2024-09-19T05:42:34.057643+0000","last_undegraded":"2024-09-19T05:42:34.057643+0000","last_fullsized":"2024-09-19T05:42:34.057643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925971+0000","last_change":"2024-09-19T05:42:32.928695+0000","last_active":"2024-09-19T05:42:33.925971+0000","last_peered":"2024-09-19T05:42:33.925971+0000","last_clean":"2024-09-19T05:42:33.925971+0000","last_became_active":"2024-09-19T05:42:32.928437+0000","last_became_peered":"2024-09-19T05:42:32.928437+0000","last_unstale":"2024-09-19T05:42:33.925971+0000","last_undegraded":"2024-09-19T05:42:33.925971+0000","last_fullsized":"2024-09-19T05:42:33.925971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058846+0000","last_change":"2024-09-19T05:42:31.985393+0000","last_active":"2024-09-19T05:42:34.058846+0000","last_peered":"2024-09-19T05:42:34.058846+0000","last_clean":"2024-09-19T05:42:34.058846+0000","last_became_active":"2024-09-19T05:42:31.985034+0000","last_became_peered":"2024-09-19T05:42:31.985034+0000","last_unstale":"2024-09-19T05:42:34.058846+0000","last_undegraded":"2024-09-19T05:42:34.058846+0000","last_fullsized":"2024-09-19T05:42:34.058846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:11.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923861+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:42:33.923861+0000","last_peered":"2024-09-19T05:42:33.923861+0000","last_clean":"2024-09-19T05:42:33.923861+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:42:33.923861+0000","last_undegraded":"2024-09-19T05:42:33.923861+0000","last_fullsized":"2024-09-19T05:42:33.923861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058193+0000","last_change":"2024-09-19T05:42:31.987634+0000","last_active":"2024-09-19T05:42:34.058193+0000","last_peered":"2024-09-19T05:42:34.058193+0000","last_clean":"2024-09-19T05:42:34.058193+0000","last_became_active":"2024-09-19T05:42:31.986452+0000","last_became_peered":"2024-09-19T05:42:31.986452+0000","last_unstale":"2024-09-19T05:42:34.058193+0000","last_undegraded":"2024-09-19T05:42:34.058193+0000","last_fullsized":"2024-09-19T05:42:34.058193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.981030+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:42:37.981030+0000","last_peered":"2024-09-19T05:42:37.981030+0000","last_clean":"2024-09-19T05:42:37.981030+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:42:37.981030+0000","last_undegraded":"2024-09-19T05:42:37.981030+0000","last_fullsized":"2024-09-19T05:42:37.981030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059270+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:42:34.059270+0000","last_peered":"2024-09-19T05:42:34.059270+0000","last_clean":"2024-09-19T05:42:34.059270+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:42:34.059270+0000","last_undegraded":"2024-09-19T05:42:34.059270+0000","last_fullsized":"2024-09-19T05:42:34.059270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.983225+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:42:37.983225+0000","last_peered":"2024-09-19T05:42:37.983225+0000","last_clean":"2024-09-19T05:42:37.983225+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:42:37.983225+0000","last_undegraded":"2024-09-19T05:42:37.983225+0000","last_fullsized":"2024-09-19T05:42:37.983225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058283+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:42:34.058283+0000","last_peered":"2024-09-19T05:42:34.058283+0000","last_clean":"2024-09-19T05:42:34.058283+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:42:34.058283+0000","last_undegraded":"2024-09-19T05:42:34.058283+0000","last_fullsized":"2024-09-19T05:42:34.058283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978898+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:42:37.978898+0000","last_peered":"2024-09-19T05:42:37.978898+0000","last_clean":"2024-09-19T05:42:37.978898+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:42:37.978898+0000","last_undegraded":"2024-09-19T05:42:37.978898+0000","last_fullsized":"2024-09-19T05:42:37.978898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058589+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:42:34.058589+0000","last_peered":"2024-09-19T05:42:34.058589+0000","last_clean":"2024-09-19T05:42:34.058589+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:42:34.058589+0000","last_undegraded":"2024-09-19T05:42:34.058589+0000","last_fullsized":"2024-09-19T05:42:34.058589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059325+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:42:34.059325+0000","last_peered":"2024-09-19T05:42:34.059325+0000","last_clean":"2024-09-19T05:42:34.059325+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:42:34.059325+0000","last_undegraded":"2024-09-19T05:42:34.059325+0000","last_fullsized":"2024-09-19T05:42:34.059325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059177+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:42:34.059177+0000","last_peered":"2024-09-19T05:42:34.059177+0000","last_clean":"2024-09-19T05:42:34.059177+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:42:34.059177+0000","last_undegraded":"2024-09-19T05:42:34.059177+0000","last_fullsized":"2024-09-19T05:42:34.059177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057760+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:42:34.057760+0000","last_peered":"2024-09-19T05:42:34.057760+0000","last_clean":"2024-09-19T05:42:34.057760+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:42:34.057760+0000","last_undegraded":"2024-09-19T05:42:34.057760+0000","last_fullsized":"2024-09-19T05:42:34.057760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058006+0000","last_change":"2024-09-19T05:42:32.938549+0000","last_active":"2024-09-19T05:42:34.058006+0000","last_peered":"2024-09-19T05:42:34.058006+0000","last_clean":"2024-09-19T05:42:34.058006+0000","last_became_active":"2024-09-19T05:42:32.938113+0000","last_became_peered":"2024-09-19T05:42:32.938113+0000","last_unstale":"2024-09-19T05:42:34.058006+0000","last_undegraded":"2024-09-19T05:42:34.058006+0000","last_fullsized":"2024-09-19T05:42:34.058006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:15.787698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.985533+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:42:37.985533+0000","last_peered":"2024-09-19T05:42:37.985533+0000","last_clean":"2024-09-19T05:42:37.985533+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:42:37.985533+0000","last_undegraded":"2024-09-19T05:42:37.985533+0000","last_fullsized":"2024-09-19T05:42:37.985533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.837827+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:42:34.837827+0000","last_peered":"2024-09-19T05:42:34.837827+0000","last_clean":"2024-09-19T05:42:34.837827+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:42:34.837827+0000","last_undegraded":"2024-09-19T05:42:34.837827+0000","last_fullsized":"2024-09-19T05:42:34.837827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923969+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:42:33.923969+0000","last_peered":"2024-09-19T05:42:33.923969+0000","last_clean":"2024-09-19T05:42:33.923969+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:42:33.923969+0000","last_undegraded":"2024-09-19T05:42:33.923969+0000","last_fullsized":"2024-09-19T05:42:33.923969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980561+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:42:37.980561+0000","last_peered":"2024-09-19T05:42:37.980561+0000","last_clean":"2024-09-19T05:42:37.980561+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:42:37.980561+0000","last_undegraded":"2024-09-19T05:42:37.980561+0000","last_fullsized":"2024-09-19T05:42:37.980561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058299+0000","last_change":"2024-09-19T05:42:32.945685+0000","last_active":"2024-09-19T05:42:34.058299+0000","last_peered":"2024-09-19T05:42:34.058299+0000","last_clean":"2024-09-19T05:42:34.058299+0000","last_became_active":"2024-09-19T05:42:32.945337+0000","last_became_peered":"2024-09-19T05:42:32.945337+0000","last_unstale":"2024-09-19T05:42:34.058299+0000","last_undegraded":"2024-09-19T05:42:34.058299+0000","last_fullsized":"2024-09-19T05:42:34.058299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832321+0000","last_change":"2024-09-19T05:42:31.991430+0000","last_active":"2024-09-19T05:42:34.832321+0000","last_peered":"2024-09-19T05:42:34.832321+0000","last_clean":"2024-09-19T05:42:34.832321+0000","last_became_active":"2024-09-19T05:42:31.991057+0000","last_became_peered":"2024-09-19T05:42:31.991057+0000","last_unstale":"2024-09-19T05:42:34.832321+0000","last_undegraded":"2024-09-19T05:42:34.832321+0000","last_fullsized":"2024-09-19T05:42:34.832321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:58.247495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058366+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:42:34.058366+0000","last_peered":"2024-09-19T05:42:34.058366+0000","last_clean":"2024-09-19T05:42:34.058366+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:42:34.058366+0000","last_undegraded":"2024-09-19T05:42:34.058366+0000","last_fullsized":"2024-09-19T05:42:34.058366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058851+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:42:34.058851+0000","last_peered":"2024-09-19T05:42:34.058851+0000","last_clean":"2024-09-19T05:42:34.058851+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:42:34.058851+0000","last_undegraded":"2024-09-19T05:42:34.058851+0000","last_fullsized":"2024-09-19T05:42:34.058851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059399+0000","last_change":"2024-09-19T05:42:32.941889+0000","last_active":"2024-09-19T05:42:34.059399+0000","last_peered":"2024-09-19T05:42:34.059399+0000","last_clean":"2024-09-19T05:42:34.059399+0000","last_became_active":"2024-09-19T05:42:32.941254+0000","last_became_peered":"2024-09-19T05:42:32.941254+0000","last_unstale":"2024-09-19T05:42:34.059399+0000","last_undegraded":"2024-09-19T05:42:34.059399+0000","last_fullsized":"2024-09-19T05:42:34.059399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.771973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057837+0000","last_change":"2024-09-19T05:42:23.458936+0000","last_active":"2024-09-19T05:42:34.057837+0000","last_peered":"2024-09-19T05:42:34.057837+0000","last_clean":"2024-09-19T05:42:34.057837+0000","last_became_active":"2024-09-19T05:42:23.457990+0000","last_became_peered":"2024-09-19T05:42:23.457990+0000","last_unstale":"2024-09-19T05:42:34.057837+0000","last_undegraded":"2024-09-19T05:42:34.057837+0000","last_fullsized":"2024-09-19T05:42:34.057837+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:45.259241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059347+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:42:34.059347+0000","last_peered":"2024-09-19T05:42:34.059347+0000","last_clean":"2024-09-19T05:42:34.059347+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:42:34.059347+0000","last_undegraded":"2024-09-19T05:42:34.059347+0000","last_fullsized":"2024-09-19T05:42:34.059347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925215+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:42:33.925215+0000","last_peered":"2024-09-19T05:42:33.925215+0000","last_clean":"2024-09-19T05:42:33.925215+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:42:33.925215+0000","last_undegraded":"2024-09-19T05:42:33.925215+0000","last_fullsized":"2024-09-19T05:42:33.925215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057735+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:42:34.057735+0000","last_peered":"2024-09-19T05:42:34.057735+0000","last_clean":"2024-09-19T05:42:34.057735+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:42:34.057735+0000","last_undegraded":"2024-09-19T05:42:34.057735+0000","last_fullsized":"2024-09-19T05:42:34.057735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058126+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:42:34.058126+0000","last_peered":"2024-09-19T05:42:34.058126+0000","last_clean":"2024-09-19T05:42:34.058126+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:42:34.058126+0000","last_undegraded":"2024-09-19T05:42:34.058126+0000","last_fullsized":"2024-09-19T05:42:34.058126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924861+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:42:33.924861+0000","last_peered":"2024-09-19T05:42:33.924861+0000","last_clean":"2024-09-19T05:42:33.924861+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:42:33.924861+0000","last_undegraded":"2024-09-19T05:42:33.924861+0000","last_fullsized":"2024-09-19T05:42:33.924861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057949+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:42:34.057949+0000","last_peered":"2024-09-19T05:42:34.057949+0000","last_clean":"2024-09-19T05:42:34.057949+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:42:34.057949+0000","last_undegraded":"2024-09-19T05:42:34.057949+0000","last_fullsized":"2024-09-19T05:42:34.057949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059345+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:42:34.059345+0000","last_peered":"2024-09-19T05:42:34.059345+0000","last_clean":"2024-09-19T05:42:34.059345+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:42:34.059345+0000","last_undegraded":"2024-09-19T05:42:34.059345+0000","last_fullsized":"2024-09-19T05:42:34.059345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057870+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:42:34.057870+0000","last_peered":"2024-09-19T05:42:34.057870+0000","last_clean":"2024-09-19T05:42:34.057870+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:42:34.057870+0000","last_undegraded":"2024-09-19T05:42:34.057870+0000","last_fullsized":"2024-09-19T05:42:34.057870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058164+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:42:34.058164+0000","last_peered":"2024-09-19T05:42:34.058164+0000","last_clean":"2024-09-19T05:42:34.058164+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:42:34.058164+0000","last_undegraded":"2024-09-19T05:42:34.058164+0000","last_fullsized":"2024-09-19T05:42:34.058164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923659+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:42:33.923659+0000","last_peered":"2024-09-19T05:42:33.923659+0000","last_clean":"2024-09-19T05:42:33.923659+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:42:33.923659+0000","last_undegraded":"2024-09-19T05:42:33.923659+0000","last_fullsized":"2024-09-19T05:42:33.923659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059484+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:42:34.059484+0000","last_peered":"2024-09-19T05:42:34.059484+0000","last_clean":"2024-09-19T05:42:34.059484+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:42:34.059484+0000","last_undegraded":"2024-09-19T05:42:34.059484+0000","last_fullsized":"2024-09-19T05:42:34.059484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058726+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:42:34.058726+0000","last_peered":"2024-09-19T05:42:34.058726+0000","last_clean":"2024-09-19T05:42:34.058726+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:42:34.058726+0000","last_undegraded":"2024-09-19T05:42:34.058726+0000","last_fullsized":"2024-09-19T05:42:34.058726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925942+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:42:33.925942+0000","last_peered":"2024-09-19T05:42:33.925942+0000","last_clean":"2024-09-19T05:42:33.925942+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:42:33.925942+0000","last_undegraded":"2024-09-19T05:42:33.925942+0000","last_fullsized":"2024-09-19T05:42:33.925942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925132+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:42:33.925132+0000","last_peered":"2024-09-19T05:42:33.925132+0000","last_clean":"2024-09-19T05:42:33.925132+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:42:33.925132+0000","last_undegraded":"2024-09-19T05:42:33.925132+0000","last_fullsized":"2024-09-19T05:42:33.925132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977979+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:42:37.977979+0000","last_peered":"2024-09-19T05:42:37.977979+0000","last_clean":"2024-09-19T05:42:37.977979+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:42:37.977979+0000","last_undegraded":"2024-09-19T05:42:37.977979+0000","last_fullsized":"2024-09-19T05:42:37.977979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925887+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:42:33.925887+0000","last_peered":"2024-09-19T05:42:33.925887+0000","last_clean":"2024-09-19T05:42:33.925887+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:42:33.925887+0000","last_undegraded":"2024-09-19T05:42:33.925887+0000","last_fullsized":"2024-09-19T05:42:33.925887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057349+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:42:34.057349+0000","last_peered":"2024-09-19T05:42:34.057349+0000","last_clean":"2024-09-19T05:42:34.057349+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:42:34.057349+0000","last_undegraded":"2024-09-19T05:42:34.057349+0000","last_fullsized":"2024-09-19T05:42:34.057349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059062+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:42:34.059062+0000","last_peered":"2024-09-19T05:42:34.059062+0000","last_clean":"2024-09-19T05:42:34.059062+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:42:34.059062+0000","last_undegraded":"2024-09-19T05:42:34.059062+0000","last_fullsized":"2024-09-19T05:42:34.059062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925905+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:42:33.925905+0000","last_peered":"2024-09-19T05:42:33.925905+0000","last_clean":"2024-09-19T05:42:33.925905+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:42:33.925905+0000","last_undegraded":"2024-09-19T05:42:33.925905+0000","last_fullsized":"2024-09-19T05:42:33.925905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925556+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:42:33.925556+0000","last_peered":"2024-09-19T05:42:33.925556+0000","last_clean":"2024-09-19T05:42:33.925556+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:42:33.925556+0000","last_undegraded":"2024-09-19T05:42:33.925556+0000","last_fullsized":"2024-09-19T05:42:33.925556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058603+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:42:34.058603+0000","last_peered":"2024-09-19T05:42:34.058603+0000","last_clean":"2024-09-19T05:42:34.058603+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:42:34.058603+0000","last_undegraded":"2024-09-19T05:42:34.058603+0000","last_fullsized":"2024-09-19T05:42:34.058603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925165+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:42:33.925165+0000","last_peered":"2024-09-19T05:42:33.925165+0000","last_clean":"2024-09-19T05:42:33.925165+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:42:33.925165+0000","last_undegraded":"2024-09-19T05:42:33.925165+0000","last_fullsized":"2024-09-19T05:42:33.925165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978153+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:42:37.978153+0000","last_peered":"2024-09-19T05:42:37.978153+0000","last_clean":"2024-09-19T05:42:37.978153+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:42:37.978153+0000","last_undegraded":"2024-09-19T05:42:37.978153+0000","last_fullsized":"2024-09-19T05:42:37.978153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059198+0000","last_change":"2024-09-19T05:42:32.941131+0000","last_active":"2024-09-19T05:42:34.059198+0000","last_peered":"2024-09-19T05:42:34.059198+0000","last_clean":"2024-09-19T05:42:34.059198+0000","last_became_active":"2024-09-19T05:42:32.940596+0000","last_became_peered":"2024-09-19T05:42:32.940596+0000","last_unstale":"2024-09-19T05:42:34.059198+0000","last_undegraded":"2024-09-19T05:42:34.059198+0000","last_fullsized":"2024-09-19T05:42:34.059198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:00.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059376+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:42:34.059376+0000","last_peered":"2024-09-19T05:42:34.059376+0000","last_clean":"2024-09-19T05:42:34.059376+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:42:34.059376+0000","last_undegraded":"2024-09-19T05:42:34.059376+0000","last_fullsized":"2024-09-19T05:42:34.059376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978276+0000","last_change":"2024-09-19T05:42:31.988393+0000","last_active":"2024-09-19T05:42:37.978276+0000","last_peered":"2024-09-19T05:42:37.978276+0000","last_clean":"2024-09-19T05:42:37.978276+0000","last_became_active":"2024-09-19T05:42:31.986557+0000","last_became_peered":"2024-09-19T05:42:31.986557+0000","last_unstale":"2024-09-19T05:42:37.978276+0000","last_undegraded":"2024-09-19T05:42:37.978276+0000","last_fullsized":"2024-09-19T05:42:37.978276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058853+0000","last_change":"2024-09-19T05:42:32.929140+0000","last_active":"2024-09-19T05:42:34.058853+0000","last_peered":"2024-09-19T05:42:34.058853+0000","last_clean":"2024-09-19T05:42:34.058853+0000","last_became_active":"2024-09-19T05:42:32.928811+0000","last_became_peered":"2024-09-19T05:42:32.928811+0000","last_unstale":"2024-09-19T05:42:34.058853+0000","last_undegraded":"2024-09-19T05:42:34.058853+0000","last_fullsized":"2024-09-19T05:42:34.058853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:29.306206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977607+0000","last_change":"2024-09-19T05:42:31.988083+0000","last_active":"2024-09-19T05:42:37.977607+0000","last_peered":"2024-09-19T05:42:37.977607+0000","last_clean":"2024-09-19T05:42:37.977607+0000","last_became_active":"2024-09-19T05:42:31.987088+0000","last_became_peered":"2024-09-19T05:42:31.987088+0000","last_unstale":"2024-09-19T05:42:37.977607+0000","last_undegraded":"2024-09-19T05:42:37.977607+0000","last_fullsized":"2024-09-19T05:42:37.977607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.941310+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.940997+0000","last_became_peered":"2024-09-19T05:42:32.940997+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:58.755996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832348+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:42:34.832348+0000","last_peered":"2024-09-19T05:42:34.832348+0000","last_clean":"2024-09-19T05:42:34.832348+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:42:34.832348+0000","last_undegraded":"2024-09-19T05:42:34.832348+0000","last_fullsized":"2024-09-19T05:42:34.832348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058604+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:42:34.058604+0000","last_peered":"2024-09-19T05:42:34.058604+0000","last_clean":"2024-09-19T05:42:34.058604+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:42:34.058604+0000","last_undegraded":"2024-09-19T05:42:34.058604+0000","last_fullsized":"2024-09-19T05:42:34.058604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980788+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:42:37.980788+0000","last_peered":"2024-09-19T05:42:37.980788+0000","last_clean":"2024-09-19T05:42:37.980788+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:42:37.980788+0000","last_undegraded":"2024-09-19T05:42:37.980788+0000","last_fullsized":"2024-09-19T05:42:37.980788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":80728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669576,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":81400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":79698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":294912,"data_stored":669554,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:42:40.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:42:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.797+0000 7f8174282640 1 Processor -- start 2024-09-19T05:42:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.797+0000 7f8174282640 1 -- start start 2024-09-19T05:42:40.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.798+0000 7f8174282640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f816c0d06e0 0x7f816c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.798+0000 7f8174282640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c15eb50 0x7f816c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.798+0000 7f8174282640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f816c0cb400 con 0x7f816c0d0ac0 2024-09-19T05:42:40.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.798+0000 7f8174282640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f816c0cb6e0 con 0x7f816c0d06e0 2024-09-19T05:42:40.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.798+0000 7f8174282640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f816c0cc970 con 0x7f816c15eb50 2024-09-19T05:42:40.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.798+0000 7f817201e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f816c0d06e0 0x7f816c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.798+0000 7f817181d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f816c0d0ac0 0x7f816c154af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51624/0 (socket says 172.21.15.2:51624) 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817181d640 1 -- 172.21.15.2:0/2997163257 learned_addr learned my addr 172.21.15.2:0/2997163257 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817281f640 1 --2- 172.21.15.2:0/2997163257 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c15eb50 0x7f816c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817201e640 1 -- 172.21.15.2:0/2997163257 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c15eb50 msgr2=0x7f816c0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817201e640 1 --2- 172.21.15.2:0/2997163257 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c15eb50 0x7f816c0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817201e640 1 -- 172.21.15.2:0/2997163257 >> v1:172.21.15.2:6789/0 conn(0x7f816c0d0ac0 legacy=0x7f816c154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817101c640 1 -- 172.21.15.2:0/2997163257 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1058699050 0 0) 0x7f816c0cb400 con 0x7f816c0d0ac0 2024-09-19T05:42:40.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817201e640 1 -- 172.21.15.2:0/2997163257 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c0ccc50 con 0x7f816c0d06e0 2024-09-19T05:42:40.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.799+0000 7f817201e640 1 --2- 172.21.15.2:0/2997163257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d06e0 0x7f816c0cbb40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81680097a0 tx=0x7f816802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a58524d6fd3022 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f817101c640 1 -- 172.21.15.2:0/2997163257 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168037070 con 0x7f816c0d06e0 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f817101c640 1 -- 172.21.15.2:0/2997163257 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8168032440 con 0x7f816c0d06e0 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f817101c640 1 -- 172.21.15.2:0/2997163257 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168032810 con 0x7f816c0d06e0 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 -- 172.21.15.2:0/2997163257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d06e0 msgr2=0x7f816c0cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 --2- 172.21.15.2:0/2997163257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d06e0 0x7f816c0cbb40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f81680097a0 tx=0x7f816802a180 comp rx=0 tx=0).stop 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 -- 172.21.15.2:0/2997163257 shutdown_connections 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 --2- 172.21.15.2:0/2997163257 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c15eb50 0x7f816c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 --2- 172.21.15.2:0/2997163257 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d06e0 0x7f816c0cbb40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 -- 172.21.15.2:0/2997163257 >> 172.21.15.2:0/2997163257 conn(0x7f816c06d740 msgr2=0x7f816c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 -- 172.21.15.2:0/2997163257 shutdown_connections 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.800+0000 7f8174282640 1 -- 172.21.15.2:0/2997163257 wait complete. 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.801+0000 7f8174282640 1 Processor -- start 2024-09-19T05:42:40.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.801+0000 7f8174282640 1 -- start start 2024-09-19T05:42:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f8174282640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c0d06e0 0x7f816c158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f8174282640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d0ac0 0x7f816c157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f8174282640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816c15eb50 0x7f816c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f8174282640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0c5830 con 0x7f816c0d0ac0 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f8174282640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f816c15f000 con 0x7f816c15eb50 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f8174282640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f816c155200 con 0x7f816c0d06e0 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f817181d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d0ac0 0x7f816c157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f817181d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d0ac0 0x7f816c157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37770/0 (socket says 172.21.15.2:37770) 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f817181d640 1 -- 172.21.15.2:0/1103035202 learned_addr learned my addr 172.21.15.2:0/1103035202 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f817201e640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c0d06e0 0x7f816c158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.802+0000 7f817281f640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816c15eb50 0x7f816c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f817181d640 1 -- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c0d06e0 msgr2=0x7f816c158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f817181d640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c0d06e0 0x7f816c158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f817181d640 1 -- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816c15eb50 msgr2=0x7f816c157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f817181d640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816c15eb50 0x7f816c157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f817181d640 1 -- 172.21.15.2:0/1103035202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c15a0d0 con 0x7f816c0d0ac0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f817281f640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816c15eb50 0x7f816c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f817181d640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d0ac0 0x7f816c157020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f815c00c460 tx=0x7f815c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f8162ffd640 1 -- 172.21.15.2:0/1103035202 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c018070 con 0x7f816c0d0ac0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f8162ffd640 1 -- 172.21.15.2:0/1103035202 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815c007520 con 0x7f816c0d0ac0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c0ca560 con 0x7f816c0d0ac0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f8162ffd640 1 -- 172.21.15.2:0/1103035202 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c004a20 con 0x7f816c0d0ac0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.803+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c152270 con 0x7f816c0d0ac0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.804+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c156d00 con 0x7f816c0d0ac0 2024-09-19T05:42:40.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.805+0000 7f8162ffd640 1 -- 172.21.15.2:0/1103035202 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f815c01b070 con 0x7f816c0d0ac0 2024-09-19T05:42:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.805+0000 7f8162ffd640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f815006c2e0 0x7f815006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.805+0000 7f8162ffd640 1 -- 172.21.15.2:0/1103035202 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f815c08a720 con 0x7f816c0d0ac0 2024-09-19T05:42:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.805+0000 7f817201e640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f815006c2e0 0x7f815006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:40.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.806+0000 7f817201e640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f815006c2e0 0x7f815006e7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8168034000 tx=0x7f816802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:40.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.809+0000 7f8162ffd640 1 -- 172.21.15.2:0/1103035202 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f816c156d00 con 0x7f816c0d0ac0 2024-09-19T05:42:40.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.986+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f816c151d40 con 0x7f815006c2e0 2024-09-19T05:42:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.992+0000 7f8162ffd640 1 -- 172.21.15.2:0/1103035202 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252004 (secure 0 0 0) 0x7f816c151d40 con 0x7f815006c2e0 2024-09-19T05:42:40.997 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:40.998 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:42:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f815006c2e0 msgr2=0x7f815006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f815006c2e0 0x7f815006e7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8168034000 tx=0x7f816802e000 comp rx=0 tx=0).stop 2024-09-19T05:42:40.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d0ac0 msgr2=0x7f816c157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d0ac0 0x7f816c157020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f815c00c460 tx=0x7f815c00c920 comp rx=0 tx=0).stop 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 shutdown_connections 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f815006c2e0 0x7f815006e7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f816c15eb50 0x7f816c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f816c0d0ac0 0x7f816c157020 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 --2- 172.21.15.2:0/1103035202 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f816c0d06e0 0x7f816c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 >> 172.21.15.2:0/1103035202 conn(0x7f816c06d740 msgr2=0x7f816c0c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 shutdown_connections 2024-09-19T05:42:41.000 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:40.995+0000 7f8174282640 1 -- 172.21.15.2:0/1103035202 wait complete. 2024-09-19T05:42:41.008 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-19T05:42:39.820905+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234796,"kb_used_data":1952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1998848,"data_stored":1818484,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.851188"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058235+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:42:34.058235+0000","last_peered":"2024-09-19T05:42:34.058235+0000","last_clean":"2024-09-19T05:42:34.058235+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:42:34.058235+0000","last_undegraded":"2024-09-19T05:42:34.058235+0000","last_fullsized":"2024-09-19T05:42:34.058235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058494+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:42:34.058494+0000","last_peered":"2024-09-19T05:42:34.058494+0000","last_clean":"2024-09-19T05:42:34.058494+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:42:34.058494+0000","last_undegraded":"2024-09-19T05:42:34.058494+0000","last_fullsized":"2024-09-19T05:42:34.058494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058017+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:42:34.058017+0000","last_peered":"2024-09-19T05:42:34.058017+0000","last_clean":"2024-09-19T05:42:34.058017+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:42:34.058017+0000","last_undegraded":"2024-09-19T05:42:34.058017+0000","last_fullsized":"2024-09-19T05:42:34.058017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923708+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:42:33.923708+0000","last_peered":"2024-09-19T05:42:33.923708+0000","last_clean":"2024-09-19T05:42:33.923708+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:42:33.923708+0000","last_undegraded":"2024-09-19T05:42:33.923708+0000","last_fullsized":"2024-09-19T05:42:33.923708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057846+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:42:34.057846+0000","last_peered":"2024-09-19T05:42:34.057846+0000","last_clean":"2024-09-19T05:42:34.057846+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:42:34.057846+0000","last_undegraded":"2024-09-19T05:42:34.057846+0000","last_fullsized":"2024-09-19T05:42:34.057846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923679+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:42:33.923679+0000","last_peered":"2024-09-19T05:42:33.923679+0000","last_clean":"2024-09-19T05:42:33.923679+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:42:33.923679+0000","last_undegraded":"2024-09-19T05:42:33.923679+0000","last_fullsized":"2024-09-19T05:42:33.923679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057764+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:42:34.057764+0000","last_peered":"2024-09-19T05:42:34.057764+0000","last_clean":"2024-09-19T05:42:34.057764+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:42:34.057764+0000","last_undegraded":"2024-09-19T05:42:34.057764+0000","last_fullsized":"2024-09-19T05:42:34.057764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058259+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:42:34.058259+0000","last_peered":"2024-09-19T05:42:34.058259+0000","last_clean":"2024-09-19T05:42:34.058259+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:42:34.058259+0000","last_undegraded":"2024-09-19T05:42:34.058259+0000","last_fullsized":"2024-09-19T05:42:34.058259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058756+0000","last_change":"2024-09-19T05:42:32.945872+0000","last_active":"2024-09-19T05:42:34.058756+0000","last_peered":"2024-09-19T05:42:34.058756+0000","last_clean":"2024-09-19T05:42:34.058756+0000","last_became_active":"2024-09-19T05:42:32.945545+0000","last_became_peered":"2024-09-19T05:42:32.945545+0000","last_unstale":"2024-09-19T05:42:34.058756+0000","last_undegraded":"2024-09-19T05:42:34.058756+0000","last_fullsized":"2024-09-19T05:42:34.058756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058073+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:42:34.058073+0000","last_peered":"2024-09-19T05:42:34.058073+0000","last_clean":"2024-09-19T05:42:34.058073+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:42:34.058073+0000","last_undegraded":"2024-09-19T05:42:34.058073+0000","last_fullsized":"2024-09-19T05:42:34.058073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058096+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:42:34.058096+0000","last_peered":"2024-09-19T05:42:34.058096+0000","last_clean":"2024-09-19T05:42:34.058096+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:42:34.058096+0000","last_undegraded":"2024-09-19T05:42:34.058096+0000","last_fullsized":"2024-09-19T05:42:34.058096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058167+0000","last_change":"2024-09-19T05:42:32.941601+0000","last_active":"2024-09-19T05:42:34.058167+0000","last_peered":"2024-09-19T05:42:34.058167+0000","last_clean":"2024-09-19T05:42:34.058167+0000","last_became_active":"2024-09-19T05:42:32.941288+0000","last_became_peered":"2024-09-19T05:42:32.941288+0000","last_unstale":"2024-09-19T05:42:34.058167+0000","last_undegraded":"2024-09-19T05:42:34.058167+0000","last_fullsized":"2024-09-19T05:42:34.058167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:35.670719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058583+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:42:34.058583+0000","last_peered":"2024-09-19T05:42:34.058583+0000","last_clean":"2024-09-19T05:42:34.058583+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:42:34.058583+0000","last_undegraded":"2024-09-19T05:42:34.058583+0000","last_fullsized":"2024-09-19T05:42:34.058583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058482+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:42:34.058482+0000","last_peered":"2024-09-19T05:42:34.058482+0000","last_clean":"2024-09-19T05:42:34.058482+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:42:34.058482+0000","last_undegraded":"2024-09-19T05:42:34.058482+0000","last_fullsized":"2024-09-19T05:42:34.058482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058261+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:42:34.058261+0000","last_peered":"2024-09-19T05:42:34.058261+0000","last_clean":"2024-09-19T05:42:34.058261+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:42:34.058261+0000","last_undegraded":"2024-09-19T05:42:34.058261+0000","last_fullsized":"2024-09-19T05:42:34.058261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057516+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:42:34.057516+0000","last_peered":"2024-09-19T05:42:34.057516+0000","last_clean":"2024-09-19T05:42:34.057516+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:42:34.057516+0000","last_undegraded":"2024-09-19T05:42:34.057516+0000","last_fullsized":"2024-09-19T05:42:34.057516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059660+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:42:34.059660+0000","last_peered":"2024-09-19T05:42:34.059660+0000","last_clean":"2024-09-19T05:42:34.059660+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:42:34.059660+0000","last_undegraded":"2024-09-19T05:42:34.059660+0000","last_fullsized":"2024-09-19T05:42:34.059660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059534+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:42:34.059534+0000","last_peered":"2024-09-19T05:42:34.059534+0000","last_clean":"2024-09-19T05:42:34.059534+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:42:34.059534+0000","last_undegraded":"2024-09-19T05:42:34.059534+0000","last_fullsized":"2024-09-19T05:42:34.059534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058089+0000","last_change":"2024-09-19T05:42:32.940627+0000","last_active":"2024-09-19T05:42:34.058089+0000","last_peered":"2024-09-19T05:42:34.058089+0000","last_clean":"2024-09-19T05:42:34.058089+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058089+0000","last_undegraded":"2024-09-19T05:42:34.058089+0000","last_fullsized":"2024-09-19T05:42:34.058089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.616840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925755+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:42:33.925755+0000","last_peered":"2024-09-19T05:42:33.925755+0000","last_clean":"2024-09-19T05:42:33.925755+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:42:33.925755+0000","last_undegraded":"2024-09-19T05:42:33.925755+0000","last_fullsized":"2024-09-19T05:42:33.925755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058013+0000","last_change":"2024-09-19T05:42:32.940425+0000","last_active":"2024-09-19T05:42:34.058013+0000","last_peered":"2024-09-19T05:42:34.058013+0000","last_clean":"2024-09-19T05:42:34.058013+0000","last_became_active":"2024-09-19T05:42:32.940059+0000","last_became_peered":"2024-09-19T05:42:32.940059+0000","last_unstale":"2024-09-19T05:42:34.058013+0000","last_undegraded":"2024-09-19T05:42:34.058013+0000","last_fullsized":"2024-09-19T05:42:34.058013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058206+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:42:34.058206+0000","last_peered":"2024-09-19T05:42:34.058206+0000","last_clean":"2024-09-19T05:42:34.058206+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:42:34.058206+0000","last_undegraded":"2024-09-19T05:42:34.058206+0000","last_fullsized":"2024-09-19T05:42:34.058206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057512+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:42:34.057512+0000","last_peered":"2024-09-19T05:42:34.057512+0000","last_clean":"2024-09-19T05:42:34.057512+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:42:34.057512+0000","last_undegraded":"2024-09-19T05:42:34.057512+0000","last_fullsized":"2024-09-19T05:42:34.057512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058955+0000","last_change":"2024-09-19T05:42:32.941414+0000","last_active":"2024-09-19T05:42:34.058955+0000","last_peered":"2024-09-19T05:42:34.058955+0000","last_clean":"2024-09-19T05:42:34.058955+0000","last_became_active":"2024-09-19T05:42:32.941076+0000","last_became_peered":"2024-09-19T05:42:32.941076+0000","last_unstale":"2024-09-19T05:42:34.058955+0000","last_undegraded":"2024-09-19T05:42:34.058955+0000","last_fullsized":"2024-09-19T05:42:34.058955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:13.371046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923839+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:42:33.923839+0000","last_peered":"2024-09-19T05:42:33.923839+0000","last_clean":"2024-09-19T05:42:33.923839+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:42:33.923839+0000","last_undegraded":"2024-09-19T05:42:33.923839+0000","last_fullsized":"2024-09-19T05:42:33.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058867+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:42:34.058867+0000","last_peered":"2024-09-19T05:42:34.058867+0000","last_clean":"2024-09-19T05:42:34.058867+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:42:34.058867+0000","last_undegraded":"2024-09-19T05:42:34.058867+0000","last_fullsized":"2024-09-19T05:42:34.058867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058377+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:42:34.058377+0000","last_peered":"2024-09-19T05:42:34.058377+0000","last_clean":"2024-09-19T05:42:34.058377+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:42:34.058377+0000","last_undegraded":"2024-09-19T05:42:34.058377+0000","last_fullsized":"2024-09-19T05:42:34.058377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059114+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:42:34.059114+0000","last_peered":"2024-09-19T05:42:34.059114+0000","last_clean":"2024-09-19T05:42:34.059114+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:42:34.059114+0000","last_undegraded":"2024-09-19T05:42:34.059114+0000","last_fullsized":"2024-09-19T05:42:34.059114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924874+0000","last_change":"2024-09-19T05:42:32.927184+0000","last_active":"2024-09-19T05:42:33.924874+0000","last_peered":"2024-09-19T05:42:33.924874+0000","last_clean":"2024-09-19T05:42:33.924874+0000","last_became_active":"2024-09-19T05:42:32.926952+0000","last_became_peered":"2024-09-19T05:42:32.926952+0000","last_unstale":"2024-09-19T05:42:33.924874+0000","last_undegraded":"2024-09-19T05:42:33.924874+0000","last_fullsized":"2024-09-19T05:42:33.924874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923475+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:42:33.923475+0000","last_peered":"2024-09-19T05:42:33.923475+0000","last_clean":"2024-09-19T05:42:33.923475+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:42:33.923475+0000","last_undegraded":"2024-09-19T05:42:33.923475+0000","last_fullsized":"2024-09-19T05:42:33.923475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058161+0000","last_change":"2024-09-19T05:42:32.942584+0000","last_active":"2024-09-19T05:42:34.058161+0000","last_peered":"2024-09-19T05:42:34.058161+0000","last_clean":"2024-09-19T05:42:34.058161+0000","last_became_active":"2024-09-19T05:42:32.942274+0000","last_became_peered":"2024-09-19T05:42:32.942274+0000","last_unstale":"2024-09-19T05:42:34.058161+0000","last_undegraded":"2024-09-19T05:42:34.058161+0000","last_fullsized":"2024-09-19T05:42:34.058161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832292+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:42:34.832292+0000","last_peered":"2024-09-19T05:42:34.832292+0000","last_clean":"2024-09-19T05:42:34.832292+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:42:34.832292+0000","last_undegraded":"2024-09-19T05:42:34.832292+0000","last_fullsized":"2024-09-19T05:42:34.832292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925748+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:42:33.925748+0000","last_peered":"2024-09-19T05:42:33.925748+0000","last_clean":"2024-09-19T05:42:33.925748+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:42:33.925748+0000","last_undegraded":"2024-09-19T05:42:33.925748+0000","last_fullsized":"2024-09-19T05:42:33.925748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.839402+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:42:34.839402+0000","last_peered":"2024-09-19T05:42:34.839402+0000","last_clean":"2024-09-19T05:42:34.839402+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:42:34.839402+0000","last_undegraded":"2024-09-19T05:42:34.839402+0000","last_fullsized":"2024-09-19T05:42:34.839402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058211+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:42:34.058211+0000","last_peered":"2024-09-19T05:42:34.058211+0000","last_clean":"2024-09-19T05:42:34.058211+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:42:34.058211+0000","last_undegraded":"2024-09-19T05:42:34.058211+0000","last_fullsized":"2024-09-19T05:42:34.058211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058257+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:42:34.058257+0000","last_peered":"2024-09-19T05:42:34.058257+0000","last_clean":"2024-09-19T05:42:34.058257+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058257+0000","last_undegraded":"2024-09-19T05:42:34.058257+0000","last_fullsized":"2024-09-19T05:42:34.058257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057643+0000","last_change":"2024-09-19T05:42:31.995072+0000","last_active":"2024-09-19T05:42:34.057643+0000","last_peered":"2024-09-19T05:42:34.057643+0000","last_clean":"2024-09-19T05:42:34.057643+0000","last_became_active":"2024-09-19T05:42:31.994586+0000","last_became_peered":"2024-09-19T05:42:31.994586+0000","last_unstale":"2024-09-19T05:42:34.057643+0000","last_undegraded":"2024-09-19T05:42:34.057643+0000","last_fullsized":"2024-09-19T05:42:34.057643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925971+0000","last_change":"2024-09-19T05:42:32.928695+0000","last_active":"2024-09-19T05:42:33.925971+0000","last_peered":"2024-09-19T05:42:33.925971+0000","last_clean":"2024-09-19T05:42:33.925971+0000","last_became_active":"2024-09-19T05:42:32.928437+0000","last_became_peered":"2024-09-19T05:42:32.928437+0000","last_unstale":"2024-09-19T05:42:33.925971+0000","last_undegraded":"2024-09-19T05:42:33.925971+0000","last_fullsized":"2024-09-19T05:42:33.925971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058846+0000","last_change":"2024-09-19T05:42:31.985393+0000","last_active":"2024-09-19T05:42:34.058846+0000","last_peered":"2024-09-19T05:42:34.058846+0000","last_clean":"2024-09-19T05:42:34.058846+0000","last_became_active":"2024-09-19T05:42:31.985034+0000","last_became_peered":"2024-09-19T05:42:31.985034+0000","last_unstale":"2024-09-19T05:42:34.058846+0000","last_undegraded":"2024-09-19T05:42:34.058846+0000","last_fullsized":"2024-09-19T05:42:34.058846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:11.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923861+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:42:33.923861+0000","last_peered":"2024-09-19T05:42:33.923861+0000","last_clean":"2024-09-19T05:42:33.923861+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:42:33.923861+0000","last_undegraded":"2024-09-19T05:42:33.923861+0000","last_fullsized":"2024-09-19T05:42:33.923861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058193+0000","last_change":"2024-09-19T05:42:31.987634+0000","last_active":"2024-09-19T05:42:34.058193+0000","last_peered":"2024-09-19T05:42:34.058193+0000","last_clean":"2024-09-19T05:42:34.058193+0000","last_became_active":"2024-09-19T05:42:31.986452+0000","last_became_peered":"2024-09-19T05:42:31.986452+0000","last_unstale":"2024-09-19T05:42:34.058193+0000","last_undegraded":"2024-09-19T05:42:34.058193+0000","last_fullsized":"2024-09-19T05:42:34.058193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.981030+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:42:37.981030+0000","last_peered":"2024-09-19T05:42:37.981030+0000","last_clean":"2024-09-19T05:42:37.981030+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:42:37.981030+0000","last_undegraded":"2024-09-19T05:42:37.981030+0000","last_fullsized":"2024-09-19T05:42:37.981030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059270+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:42:34.059270+0000","last_peered":"2024-09-19T05:42:34.059270+0000","last_clean":"2024-09-19T05:42:34.059270+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:42:34.059270+0000","last_undegraded":"2024-09-19T05:42:34.059270+0000","last_fullsized":"2024-09-19T05:42:34.059270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.983225+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:42:37.983225+0000","last_peered":"2024-09-19T05:42:37.983225+0000","last_clean":"2024-09-19T05:42:37.983225+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:42:37.983225+0000","last_undegraded":"2024-09-19T05:42:37.983225+0000","last_fullsized":"2024-09-19T05:42:37.983225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058283+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:42:34.058283+0000","last_peered":"2024-09-19T05:42:34.058283+0000","last_clean":"2024-09-19T05:42:34.058283+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:42:34.058283+0000","last_undegraded":"2024-09-19T05:42:34.058283+0000","last_fullsized":"2024-09-19T05:42:34.058283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978898+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:42:37.978898+0000","last_peered":"2024-09-19T05:42:37.978898+0000","last_clean":"2024-09-19T05:42:37.978898+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:42:37.978898+0000","last_undegraded":"2024-09-19T05:42:37.978898+0000","last_fullsized":"2024-09-19T05:42:37.978898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058589+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:42:34.058589+0000","last_peered":"2024-09-19T05:42:34.058589+0000","last_clean":"2024-09-19T05:42:34.058589+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:42:34.058589+0000","last_undegraded":"2024-09-19T05:42:34.058589+0000","last_fullsized":"2024-09-19T05:42:34.058589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059325+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:42:34.059325+0000","last_peered":"2024-09-19T05:42:34.059325+0000","last_clean":"2024-09-19T05:42:34.059325+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:42:34.059325+0000","last_undegraded":"2024-09-19T05:42:34.059325+0000","last_fullsized":"2024-09-19T05:42:34.059325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059177+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:42:34.059177+0000","last_peered":"2024-09-19T05:42:34.059177+0000","last_clean":"2024-09-19T05:42:34.059177+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:42:34.059177+0000","last_undegraded":"2024-09-19T05:42:34.059177+0000","last_fullsized":"2024-09-19T05:42:34.059177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057760+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:42:34.057760+0000","last_peered":"2024-09-19T05:42:34.057760+0000","last_clean":"2024-09-19T05:42:34.057760+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:42:34.057760+0000","last_undegraded":"2024-09-19T05:42:34.057760+0000","last_fullsized":"2024-09-19T05:42:34.057760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058006+0000","last_change":"2024-09-19T05:42:32.938549+0000","last_active":"2024-09-19T05:42:34.058006+0000","last_peered":"2024-09-19T05:42:34.058006+0000","last_clean":"2024-09-19T05:42:34.058006+0000","last_became_active":"2024-09-19T05:42:32.938113+0000","last_became_peered":"2024-09-19T05:42:32.938113+0000","last_unstale":"2024-09-19T05:42:34.058006+0000","last_undegraded":"2024-09-19T05:42:34.058006+0000","last_fullsized":"2024-09-19T05:42:34.058006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:15.787698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.985533+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:42:37.985533+0000","last_peered":"2024-09-19T05:42:37.985533+0000","last_clean":"2024-09-19T05:42:37.985533+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:42:37.985533+0000","last_undegraded":"2024-09-19T05:42:37.985533+0000","last_fullsized":"2024-09-19T05:42:37.985533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.837827+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:42:34.837827+0000","last_peered":"2024-09-19T05:42:34.837827+0000","last_clean":"2024-09-19T05:42:34.837827+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:42:34.837827+0000","last_undegraded":"2024-09-19T05:42:34.837827+0000","last_fullsized":"2024-09-19T05:42:34.837827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923969+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:42:33.923969+0000","last_peered":"2024-09-19T05:42:33.923969+0000","last_clean":"2024-09-19T05:42:33.923969+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:42:33.923969+0000","last_undegraded":"2024-09-19T05:42:33.923969+0000","last_fullsized":"2024-09-19T05:42:33.923969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980561+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:42:37.980561+0000","last_peered":"2024-09-19T05:42:37.980561+0000","last_clean":"2024-09-19T05:42:37.980561+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:42:37.980561+0000","last_undegraded":"2024-09-19T05:42:37.980561+0000","last_fullsized":"2024-09-19T05:42:37.980561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058299+0000","last_change":"2024-09-19T05:42:32.945685+0000","last_active":"2024-09-19T05:42:34.058299+0000","last_peered":"2024-09-19T05:42:34.058299+0000","last_clean":"2024-09-19T05:42:34.058299+0000","last_became_active":"2024-09-19T05:42:32.945337+0000","last_became_peered":"2024-09-19T05:42:32.945337+0000","last_unstale":"2024-09-19T05:42:34.058299+0000","last_undegraded":"2024-09-19T05:42:34.058299+0000","last_fullsized":"2024-09-19T05:42:34.058299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832321+0000","last_change":"2024-09-19T05:42:31.991430+0000","last_active":"2024-09-19T05:42:34.832321+0000","last_peered":"2024-09-19T05:42:34.832321+0000","last_clean":"2024-09-19T05:42:34.832321+0000","last_became_active":"2024-09-19T05:42:31.991057+0000","last_became_peered":"2024-09-19T05:42:31.991057+0000","last_unstale":"2024-09-19T05:42:34.832321+0000","last_undegraded":"2024-09-19T05:42:34.832321+0000","last_fullsized":"2024-09-19T05:42:34.832321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:58.247495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058366+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:42:34.058366+0000","last_peered":"2024-09-19T05:42:34.058366+0000","last_clean":"2024-09-19T05:42:34.058366+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:42:34.058366+0000","last_undegraded":"2024-09-19T05:42:34.058366+0000","last_fullsized":"2024-09-19T05:42:34.058366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058851+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:42:34.058851+0000","last_peered":"2024-09-19T05:42:34.058851+0000","last_clean":"2024-09-19T05:42:34.058851+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:42:34.058851+0000","last_undegraded":"2024-09-19T05:42:34.058851+0000","last_fullsized":"2024-09-19T05:42:34.058851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059399+0000","last_change":"2024-09-19T05:42:32.941889+0000","last_active":"2024-09-19T05:42:34.059399+0000","last_peered":"2024-09-19T05:42:34.059399+0000","last_clean":"2024-09-19T05:42:34.059399+0000","last_became_active":"2024-09-19T05:42:32.941254+0000","last_became_peered":"2024-09-19T05:42:32.941254+0000","last_unstale":"2024-09-19T05:42:34.059399+0000","last_undegraded":"2024-09-19T05:42:34.059399+0000","last_fullsized":"2024-09-19T05:42:34.059399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.771973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057837+0000","last_change":"2024-09-19T05:42:23.458936+0000","last_active":"2024-09-19T05:42:34.057837+0000","last_peered":"2024-09-19T05:42:34.057837+0000","last_clean":"2024-09-19T05:42:34.057837+0000","last_became_active":"2024-09-19T05:42:23.457990+0000","last_became_peered":"2024-09-19T05:42:23.457990+0000","last_unstale":"2024-09-19T05:42:34.057837+0000","last_undegraded":"2024-09-19T05:42:34.057837+0000","last_fullsized":"2024-09-19T05:42:34.057837+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:45.259241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059347+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:42:34.059347+0000","last_peered":"2024-09-19T05:42:34.059347+0000","last_clean":"2024-09-19T05:42:34.059347+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:42:34.059347+0000","last_undegraded":"2024-09-19T05:42:34.059347+0000","last_fullsized":"2024-09-19T05:42:34.059347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925215+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:42:33.925215+0000","last_peered":"2024-09-19T05:42:33.925215+0000","last_clean":"2024-09-19T05:42:33.925215+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:42:33.925215+0000","last_undegraded":"2024-09-19T05:42:33.925215+0000","last_fullsized":"2024-09-19T05:42:33.925215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057735+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:42:34.057735+0000","last_peered":"2024-09-19T05:42:34.057735+0000","last_clean":"2024-09-19T05:42:34.057735+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:42:34.057735+0000","last_undegraded":"2024-09-19T05:42:34.057735+0000","last_fullsized":"2024-09-19T05:42:34.057735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058126+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:42:34.058126+0000","last_peered":"2024-09-19T05:42:34.058126+0000","last_clean":"2024-09-19T05:42:34.058126+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:42:34.058126+0000","last_undegraded":"2024-09-19T05:42:34.058126+0000","last_fullsized":"2024-09-19T05:42:34.058126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924861+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:42:33.924861+0000","last_peered":"2024-09-19T05:42:33.924861+0000","last_clean":"2024-09-19T05:42:33.924861+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:42:33.924861+0000","last_undegraded":"2024-09-19T05:42:33.924861+0000","last_fullsized":"2024-09-19T05:42:33.924861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057949+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:42:34.057949+0000","last_peered":"2024-09-19T05:42:34.057949+0000","last_clean":"2024-09-19T05:42:34.057949+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:42:34.057949+0000","last_undegraded":"2024-09-19T05:42:34.057949+0000","last_fullsized":"2024-09-19T05:42:34.057949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059345+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:42:34.059345+0000","last_peered":"2024-09-19T05:42:34.059345+0000","last_clean":"2024-09-19T05:42:34.059345+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:42:34.059345+0000","last_undegraded":"2024-09-19T05:42:34.059345+0000","last_fullsized":"2024-09-19T05:42:34.059345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057870+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:42:34.057870+0000","last_peered":"2024-09-19T05:42:34.057870+0000","last_clean":"2024-09-19T05:42:34.057870+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:42:34.057870+0000","last_undegraded":"2024-09-19T05:42:34.057870+0000","last_fullsized":"2024-09-19T05:42:34.057870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058164+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:42:34.058164+0000","last_peered":"2024-09-19T05:42:34.058164+0000","last_clean":"2024-09-19T05:42:34.058164+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:42:34.058164+0000","last_undegraded":"2024-09-19T05:42:34.058164+0000","last_fullsized":"2024-09-19T05:42:34.058164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923659+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:42:33.923659+0000","last_peered":"2024-09-19T05:42:33.923659+0000","last_clean":"2024-09-19T05:42:33.923659+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:42:33.923659+0000","last_undegraded":"2024-09-19T05:42:33.923659+0000","last_fullsized":"2024-09-19T05:42:33.923659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059484+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:42:34.059484+0000","last_peered":"2024-09-19T05:42:34.059484+0000","last_clean":"2024-09-19T05:42:34.059484+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:42:34.059484+0000","last_undegraded":"2024-09-19T05:42:34.059484+0000","last_fullsized":"2024-09-19T05:42:34.059484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058726+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:42:34.058726+0000","last_peered":"2024-09-19T05:42:34.058726+0000","last_clean":"2024-09-19T05:42:34.058726+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:42:34.058726+0000","last_undegraded":"2024-09-19T05:42:34.058726+0000","last_fullsized":"2024-09-19T05:42:34.058726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925942+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:42:33.925942+0000","last_peered":"2024-09-19T05:42:33.925942+0000","last_clean":"2024-09-19T05:42:33.925942+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:42:33.925942+0000","last_undegraded":"2024-09-19T05:42:33.925942+0000","last_fullsized":"2024-09-19T05:42:33.925942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925132+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:42:33.925132+0000","last_peered":"2024-09-19T05:42:33.925132+0000","last_clean":"2024-09-19T05:42:33.925132+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:42:33.925132+0000","last_undegraded":"2024-09-19T05:42:33.925132+0000","last_fullsized":"2024-09-19T05:42:33.925132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977979+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:42:37.977979+0000","last_peered":"2024-09-19T05:42:37.977979+0000","last_clean":"2024-09-19T05:42:37.977979+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:42:37.977979+0000","last_undegraded":"2024-09-19T05:42:37.977979+0000","last_fullsized":"2024-09-19T05:42:37.977979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925887+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:42:33.925887+0000","last_peered":"2024-09-19T05:42:33.925887+0000","last_clean":"2024-09-19T05:42:33.925887+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:42:33.925887+0000","last_undegraded":"2024-09-19T05:42:33.925887+0000","last_fullsized":"2024-09-19T05:42:33.925887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057349+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:42:34.057349+0000","last_peered":"2024-09-19T05:42:34.057349+0000","last_clean":"2024-09-19T05:42:34.057349+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:42:34.057349+0000","last_undegraded":"2024-09-19T05:42:34.057349+0000","last_fullsized":"2024-09-19T05:42:34.057349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059062+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:42:34.059062+0000","last_peered":"2024-09-19T05:42:34.059062+0000","last_clean":"2024-09-19T05:42:34.059062+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:42:34.059062+0000","last_undegraded":"2024-09-19T05:42:34.059062+0000","last_fullsized":"2024-09-19T05:42:34.059062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925905+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:42:33.925905+0000","last_peered":"2024-09-19T05:42:33.925905+0000","last_clean":"2024-09-19T05:42:33.925905+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:42:33.925905+0000","last_undegraded":"2024-09-19T05:42:33.925905+0000","last_fullsized":"2024-09-19T05:42:33.925905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925556+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:42:33.925556+0000","last_peered":"2024-09-19T05:42:33.925556+0000","last_clean":"2024-09-19T05:42:33.925556+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:42:33.925556+0000","last_undegraded":"2024-09-19T05:42:33.925556+0000","last_fullsized":"2024-09-19T05:42:33.925556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058603+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:42:34.058603+0000","last_peered":"2024-09-19T05:42:34.058603+0000","last_clean":"2024-09-19T05:42:34.058603+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:42:34.058603+0000","last_undegraded":"2024-09-19T05:42:34.058603+0000","last_fullsized":"2024-09-19T05:42:34.058603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925165+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:42:33.925165+0000","last_peered":"2024-09-19T05:42:33.925165+0000","last_clean":"2024-09-19T05:42:33.925165+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:42:33.925165+0000","last_undegraded":"2024-09-19T05:42:33.925165+0000","last_fullsized":"2024-09-19T05:42:33.925165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978153+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:42:37.978153+0000","last_peered":"2024-09-19T05:42:37.978153+0000","last_clean":"2024-09-19T05:42:37.978153+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:42:37.978153+0000","last_undegraded":"2024-09-19T05:42:37.978153+0000","last_fullsized":"2024-09-19T05:42:37.978153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059198+0000","last_change":"2024-09-19T05:42:32.941131+0000","last_active":"2024-09-19T05:42:34.059198+0000","last_peered":"2024-09-19T05:42:34.059198+0000","last_clean":"2024-09-19T05:42:34.059198+0000","last_became_active":"2024-09-19T05:42:32.940596+0000","last_became_peered":"2024-09-19T05:42:32.940596+0000","last_unstale":"2024-09-19T05:42:34.059198+0000","last_undegraded":"2024-09-19T05:42:34.059198+0000","last_fullsized":"2024-09-19T05:42:34.059198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:00.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059376+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:42:34.059376+0000","last_peered":"2024-09-19T05:42:34.059376+0000","last_clean":"2024-09-19T05:42:34.059376+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:42:34.059376+0000","last_undegraded":"2024-09-19T05:42:34.059376+0000","last_fullsized":"2024-09-19T05:42:34.059376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978276+0000","last_change":"2024-09-19T05:42:31.988393+0000","last_active":"2024-09-19T05:42:37.978276+0000","last_peered":"2024-09-19T05:42:37.978276+0000","last_clean":"2024-09-19T05:42:37.978276+0000","last_became_active":"2024-09-19T05:42:31.986557+0000","last_became_peered":"2024-09-19T05:42:31.986557+0000","last_unstale":"2024-09-19T05:42:37.978276+0000","last_undegraded":"2024-09-19T05:42:37.978276+0000","last_fullsized":"2024-09-19T05:42:37.978276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058853+0000","last_change":"2024-09-19T05:42:32.929140+0000","last_active":"2024-09-19T05:42:34.058853+0000","last_peered":"2024-09-19T05:42:34.058853+0000","last_clean":"2024-09-19T05:42:34.058853+0000","last_became_active":"2024-09-19T05:42:32.928811+0000","last_became_peered":"2024-09-19T05:42:32.928811+0000","last_unstale":"2024-09-19T05:42:34.058853+0000","last_undegraded":"2024-09-19T05:42:34.058853+0000","last_fullsized":"2024-09-19T05:42:34.058853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:29.306206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977607+0000","last_change":"2024-09-19T05:42:31.988083+0000","last_active":"2024-09-19T05:42:37.977607+0000","last_peered":"2024-09-19T05:42:37.977607+0000","last_clean":"2024-09-19T05:42:37.977607+0000","last_became_active":"2024-09-19T05:42:31.987088+0000","last_became_peered":"2024-09-19T05:42:31.987088+0000","last_unstale":"2024-09-19T05:42:37.977607+0000","last_undegraded":"2024-09-19T05:42:37.977607+0000","last_fullsized":"2024-09-19T05:42:37.977607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.941310+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.940997+0000","last_became_peered":"2024-09-19T05:42:32.940997+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:58.755996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832348+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:42:34.832348+0000","last_peered":"2024-09-19T05:42:34.832348+0000","last_clean":"2024-09-19T05:42:34.832348+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:42:34.832348+0000","last_undegraded":"2024-09-19T05:42:34.832348+0000","last_fullsized":"2024-09-19T05:42:34.832348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058604+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:42:34.058604+0000","last_peered":"2024-09-19T05:42:34.058604+0000","last_clean":"2024-09-19T05:42:34.058604+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:42:34.058604+0000","last_undegraded":"2024-09-19T05:42:34.058604+0000","last_fullsized":"2024-09-19T05:42:34.058604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980788+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:42:37.980788+0000","last_peered":"2024-09-19T05:42:37.980788+0000","last_clean":"2024-09-19T05:42:37.980788+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:42:37.980788+0000","last_undegraded":"2024-09-19T05:42:37.980788+0000","last_fullsized":"2024-09-19T05:42:37.980788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":80728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669576,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":81400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":79698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":294912,"data_stored":669554,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:42:41.012 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T05:42:41.012 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T05:42:41.012 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-19T05:42:41.012 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T05:42:41.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.147+0000 7f6bc8d18640 1 Processor -- start 2024-09-19T05:42:41.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.147+0000 7f6bc8d18640 1 -- start start 2024-09-19T05:42:41.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc8d18640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6bc41529b0 0x7f6bc415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc8d18640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6bc40cb0c0 0x7f6bc40cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc8d18640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc4158880 con 0x7f6bc4159320 2024-09-19T05:42:41.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc8d18640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc4159770 con 0x7f6bc41529b0 2024-09-19T05:42:41.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc8d18640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc40cba50 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6bc41529b0 0x7f6bc415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc1d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6bc4159320 0x7f6bc40ca490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:51634/0 (socket says 172.21.15.2:51634) 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.148+0000 7f6bc1d9b640 1 -- 172.21.15.2:0/568995175 learned_addr learned my addr 172.21.15.2:0/568995175 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc2d9d640 1 --2- 172.21.15.2:0/568995175 >> v2:172.21.15.123:3300/0 conn(0x7f6bc40cb0c0 0x7f6bc40cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc259c640 1 -- 172.21.15.2:0/568995175 >> v2:172.21.15.123:3300/0 conn(0x7f6bc40cb0c0 msgr2=0x7f6bc40cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc259c640 1 --2- 172.21.15.2:0/568995175 >> v2:172.21.15.123:3300/0 conn(0x7f6bc40cb0c0 0x7f6bc40cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc259c640 1 -- 172.21.15.2:0/568995175 >> v1:172.21.15.2:6789/0 conn(0x7f6bc4159320 legacy=0x7f6bc40ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc159a640 1 -- 172.21.15.2:0/568995175 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1067520786 0 0) 0x7f6bc4158880 con 0x7f6bc4159320 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc259c640 1 -- 172.21.15.2:0/568995175 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc40c3730 con 0x7f6bc41529b0 2024-09-19T05:42:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc2d9d640 1 --2- 172.21.15.2:0/568995175 >> v2:172.21.15.123:3300/0 conn(0x7f6bc40cb0c0 0x7f6bc40cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:41.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.149+0000 7f6bc259c640 1 --2- 172.21.15.2:0/568995175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc41529b0 0x7f6bc415bd40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40cc1c0 tx=0x7f6bbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927d627081a067d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc159a640 1 -- 172.21.15.2:0/568995175 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc037070 con 0x7f6bc41529b0 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc159a640 1 -- 172.21.15.2:0/568995175 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc032440 con 0x7f6bc41529b0 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc159a640 1 -- 172.21.15.2:0/568995175 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc032810 con 0x7f6bc41529b0 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 -- 172.21.15.2:0/568995175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc41529b0 msgr2=0x7f6bc415bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/568995175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc41529b0 0x7f6bc415bd40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40cc1c0 tx=0x7f6bbc02a1a0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 -- 172.21.15.2:0/568995175 shutdown_connections 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/568995175 >> v2:172.21.15.123:3300/0 conn(0x7f6bc40cb0c0 0x7f6bc40cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/568995175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc41529b0 0x7f6bc415bd40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 -- 172.21.15.2:0/568995175 >> 172.21.15.2:0/568995175 conn(0x7f6bc406bc10 msgr2=0x7f6bc406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 -- 172.21.15.2:0/568995175 shutdown_connections 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.150+0000 7f6bc8d18640 1 -- 172.21.15.2:0/568995175 wait complete. 2024-09-19T05:42:41.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 Processor -- start 2024-09-19T05:42:41.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 -- start start 2024-09-19T05:42:41.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc40cb0c0 0x7f6bc41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc41529b0 0x7f6bc41f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc4159320 0x7f6bc4159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4158e00 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4158b20 con 0x7f6bc41529b0 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc8d18640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc40c3cf0 con 0x7f6bc4159320 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc40cb0c0 0x7f6bc41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc259c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc40cb0c0 0x7f6bc41568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:37776/0 (socket says 172.21.15.2:37776) 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc259c640 1 -- 172.21.15.2:0/57686683 learned_addr learned my addr 172.21.15.2:0/57686683 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.152+0000 7f6bc1d9b640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc41529b0 0x7f6bc41f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc2d9d640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc4159320 0x7f6bc4159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc259c640 1 -- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc4159320 msgr2=0x7f6bc4159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc259c640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc4159320 0x7f6bc4159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc259c640 1 -- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc41529b0 msgr2=0x7f6bc41f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc259c640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc41529b0 0x7f6bc41f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc259c640 1 -- 172.21.15.2:0/57686683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc414c380 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc2d9d640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc4159320 0x7f6bc4159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc259c640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc40cb0c0 0x7f6bc41568b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc000c00 tx=0x7f6bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bbaffd640 1 -- 172.21.15.2:0/57686683 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc037040 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bbaffd640 1 -- 172.21.15.2:0/57686683 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc042070 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bbaffd640 1 -- 172.21.15.2:0/57686683 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc030a50 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc415c030 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.153+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc414c8d0 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.154+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc4152d90 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.155+0000 7f6bbaffd640 1 -- 172.21.15.2:0/57686683 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6bbc030bf0 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.155+0000 7f6bbaffd640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2e0 0x7f6ba406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.155+0000 7f6bbaffd640 1 -- 172.21.15.2:0/57686683 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6bbc0abe00 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.155+0000 7f6bc1d9b640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2e0 0x7f6ba406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.156+0000 7f6bc1d9b640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2e0 0x7f6ba406e7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0011e0 tx=0x7f6bac001130 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.159+0000 7f6bbaffd640 1 -- 172.21.15.2:0/57686683 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6bbc07a4e0 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.376+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6bc4158710 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.377+0000 7f6bbaffd640 1 -- 172.21.15.2:0/57686683 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6bbc079de0 con 0x7f6bc40cb0c0 2024-09-19T05:42:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:41.379 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T05:42:41.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2e0 msgr2=0x7f6ba406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2e0 0x7f6ba406e7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0011e0 tx=0x7f6bac001130 comp rx=0 tx=0).stop 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc40cb0c0 msgr2=0x7f6bc41568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc40cb0c0 0x7f6bc41568b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc000c00 tx=0x7f6bbc0046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 shutdown_connections 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2e0 0x7f6ba406e7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc41529b0 0x7f6bc41f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc4159320 0x7f6bc4159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 --2- 172.21.15.2:0/57686683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc40cb0c0 0x7f6bc41568b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 >> 172.21.15.2:0/57686683 conn(0x7f6bc406bc10 msgr2=0x7f6bc40c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 shutdown_connections 2024-09-19T05:42:41.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.379+0000 7f6bc8d18640 1 -- 172.21.15.2:0/57686683 wait complete. 2024-09-19T05:42:41.392 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-19T05:42:41.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:42:41.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.492+0000 7fcbc540e640 1 Processor -- start 2024-09-19T05:42:41.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.492+0000 7fcbc540e640 1 -- start start 2024-09-19T05:42:41.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.492+0000 7fcbc540e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcbc014e640 0x7fcbc00c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.492+0000 7fcbc540e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c8160 0x7fcbc00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.492+0000 7fcbc540e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbc00c8b10 con 0x7fcbc00c6c70 2024-09-19T05:42:41.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.492+0000 7fcbc540e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc00c7390 con 0x7fcbc014e640 2024-09-19T05:42:41.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.492+0000 7fcbc540e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00c3770 con 0x7fcbc00c8160 2024-09-19T05:42:41.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbeffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcbc014e640 0x7fcbc00c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbeffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcbc014e640 0x7fcbc00c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47528/0 (socket says 172.21.15.2:47528) 2024-09-19T05:42:41.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbeffd640 1 -- 172.21.15.2:0/958374853 learned_addr learned my addr 172.21.15.2:0/958374853 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:41.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbf7fe640 1 --2- 172.21.15.2:0/958374853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c8160 0x7fcbc00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbeffd640 1 -- 172.21.15.2:0/958374853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c8160 msgr2=0x7fcbc00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:41.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbeffd640 1 --2- 172.21.15.2:0/958374853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c8160 0x7fcbc00c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbeffd640 1 -- 172.21.15.2:0/958374853 >> v1:172.21.15.123:6789/0 conn(0x7fcbc00c6c70 legacy=0x7fcbc006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:41.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.493+0000 7fcbbeffd640 1 -- 172.21.15.2:0/958374853 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc00c3a30 con 0x7fcbc014e640 2024-09-19T05:42:41.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbbeffd640 1 --2- 172.21.15.2:0/958374853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc00c63f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009870 tx=0x7fcbb002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19f4d68fcac7273 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbbe7fc640 1 -- 172.21.15.2:0/958374853 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0037070 con 0x7fcbc014e640 2024-09-19T05:42:41.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbbe7fc640 1 -- 172.21.15.2:0/958374853 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb0032440 con 0x7fcbc014e640 2024-09-19T05:42:41.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbbe7fc640 1 -- 172.21.15.2:0/958374853 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0032810 con 0x7fcbc014e640 2024-09-19T05:42:41.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 -- 172.21.15.2:0/958374853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 msgr2=0x7fcbc00c63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 --2- 172.21.15.2:0/958374853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc00c63f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009870 tx=0x7fcbb002a2a0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 -- 172.21.15.2:0/958374853 shutdown_connections 2024-09-19T05:42:41.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 --2- 172.21.15.2:0/958374853 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c8160 0x7fcbc00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 --2- 172.21.15.2:0/958374853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc00c63f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 -- 172.21.15.2:0/958374853 >> 172.21.15.2:0/958374853 conn(0x7fcbc00cc270 msgr2=0x7fcbc00cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:41.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 -- 172.21.15.2:0/958374853 shutdown_connections 2024-09-19T05:42:41.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.494+0000 7fcbc540e640 1 -- 172.21.15.2:0/958374853 wait complete. 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.495+0000 7fcbc540e640 1 Processor -- start 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbc540e640 1 -- start start 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbc540e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c6c70 0x7fcbc01eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbc540e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc00c8160 0x7fcbc0157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbc540e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc0156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbc540e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00c7bf0 con 0x7fcbc014e640 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbc540e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00c7910 con 0x7fcbc00c8160 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbc540e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00c3d10 con 0x7fcbc00c6c70 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.496+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc0156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc0156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47544/0 (socket says 172.21.15.2:47544) 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 -- 172.21.15.2:0/2257114148 learned_addr learned my addr 172.21.15.2:0/2257114148 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbeffd640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c6c70 0x7fcbc01eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbb67fc640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc00c8160 0x7fcbc0157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 -- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c6c70 msgr2=0x7fcbc01eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c6c70 0x7fcbc01eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 -- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc00c8160 msgr2=0x7fcbc0157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc00c8160 0x7fcbc0157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 -- 172.21.15.2:0/2257114148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc01e2e20 con 0x7fcbc014e640 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbbf7fe640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc0156990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8007e40 tx=0x7fcbb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbb7fff640 1 -- 172.21.15.2:0/2257114148 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8014620 con 0x7fcbc014e640 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbb7fff640 1 -- 172.21.15.2:0/2257114148 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb8019070 con 0x7fcbc014e640 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc01e3100 con 0x7fcbc014e640 2024-09-19T05:42:41.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbb7fff640 1 -- 172.21.15.2:0/2257114148 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8014a20 con 0x7fcbc014e640 2024-09-19T05:42:41.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.497+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01e36c0 con 0x7fcbc014e640 2024-09-19T05:42:41.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.498+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc006d550 con 0x7fcbc014e640 2024-09-19T05:42:41.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.499+0000 7fcbb7fff640 1 -- 172.21.15.2:0/2257114148 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcbb8014bc0 con 0x7fcbc014e640 2024-09-19T05:42:41.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.499+0000 7fcbb7fff640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcba006c2e0 0x7fcba006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.499+0000 7fcbb7fff640 1 -- 172.21.15.2:0/2257114148 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcbb808bac0 con 0x7fcbc014e640 2024-09-19T05:42:41.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.499+0000 7fcbbeffd640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcba006c2e0 0x7fcba006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.500+0000 7fcbbeffd640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcba006c2e0 0x7fcba006e7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0034000 tx=0x7fcbb002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.504+0000 7fcbb7fff640 1 -- 172.21.15.2:0/2257114148 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcbc006d550 con 0x7fcbc014e640 2024-09-19T05:42:41.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.714+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbc00cb5c0 con 0x7fcbc014e640 2024-09-19T05:42:41.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.714+0000 7fcbb7fff640 1 -- 172.21.15.2:0/2257114148 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6863 (secure 0 0 0) 0x7fcbc00cb5c0 con 0x7fcbc014e640 2024-09-19T05:42:41.717 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:41.717 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"btime":"2024-09-19T05:42:38:988574+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/121690198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":121690198},{"type":"v1","addr":"172.21.15.2:6835","nonce":121690198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1118046326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1118046326},{"type":"v1","addr":"172.21.15.123:6837","nonce":1118046326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3264935286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3264935286},{"type":"v1","addr":"172.21.15.2:6837","nonce":3264935286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/4246839966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4246839966},{"type":"v1","addr":"172.21.15.123:6836","nonce":4246839966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:42:32.334667+0000","modified":"2024-09-19T05:42:38.988568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/4093916922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4093916922},{"type":"v1","addr":"172.21.15.123:6839","nonce":4093916922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4312,"qdb_cluster":[4312]},"id":1}]} 2024-09-19T05:42:41.717 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcba006c2e0 msgr2=0x7fcba006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcba006c2e0 0x7fcba006e7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0034000 tx=0x7fcbb002e000 comp rx=0 tx=0).stop 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 msgr2=0x7fcbc0156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc0156990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8007e40 tx=0x7fcbb800e4b0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 shutdown_connections 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcba006c2e0 0x7fcba006e7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbc00c8160 0x7fcbc0157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcbc014e640 0x7fcbc0156990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 --2- 172.21.15.2:0/2257114148 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbc00c6c70 0x7fcbc01eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 >> 172.21.15.2:0/2257114148 conn(0x7fcbc00cc270 msgr2=0x7fcbc014c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:41.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 shutdown_connections 2024-09-19T05:42:41.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.716+0000 7fcbc540e640 1 -- 172.21.15.2:0/2257114148 wait complete. 2024-09-19T05:42:41.730 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:42:41.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.830+0000 7fb3c56af640 1 Processor -- start 2024-09-19T05:42:41.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.830+0000 7fb3c56af640 1 -- start start 2024-09-19T05:42:41.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3c56af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3c56af640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3c00c9000 0x7fb3c015e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3c56af640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3c0151bb0 con 0x7fb3c00cbcc0 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3c56af640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c015b870 con 0x7fb3c00c9000 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3c56af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c015ac60 con 0x7fb3c00c5620 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3beffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3bf7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3c00c9000 0x7fb3c015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3bf7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb3c00c9000 0x7fb3c015e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:48040/0 (socket says 172.21.15.2:48040) 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3beffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c00c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:54098/0 (socket says 172.21.15.2:54098) 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3bf7fe640 1 -- 172.21.15.2:0/760443352 learned_addr learned my addr 172.21.15.2:0/760443352 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3be7fc640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb3c00cbcc0 0x7fb3c00ca810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:58704/0 (socket says 172.21.15.2:58704) 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3bf7fe640 1 -- 172.21.15.2:0/760443352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 msgr2=0x7fb3c00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3bf7fe640 1 --2- 172.21.15.2:0/760443352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c00c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3bf7fe640 1 -- 172.21.15.2:0/760443352 >> v1:172.21.15.123:6789/0 conn(0x7fb3c00cbcc0 legacy=0x7fb3c00ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.831+0000 7fb3bf7fe640 1 -- 172.21.15.2:0/760443352 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c015af40 con 0x7fb3c00c9000 2024-09-19T05:42:41.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.832+0000 7fb3bf7fe640 1 --2- 172.21.15.2:0/760443352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00c9000 0x7fb3c015e150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009e50 tx=0x7fb3b802a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ec395b6da5d1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.832+0000 7fb3bdffb640 1 -- 172.21.15.2:0/760443352 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8036070 con 0x7fb3c00c9000 2024-09-19T05:42:41.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.832+0000 7fb3bdffb640 1 -- 172.21.15.2:0/760443352 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b8031600 con 0x7fb3c00c9000 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.832+0000 7fb3bdffb640 1 -- 172.21.15.2:0/760443352 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80319d0 con 0x7fb3c00c9000 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 -- 172.21.15.2:0/760443352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00c9000 msgr2=0x7fb3c015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 --2- 172.21.15.2:0/760443352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00c9000 0x7fb3c015e150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009e50 tx=0x7fb3b802a7a0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 -- 172.21.15.2:0/760443352 shutdown_connections 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 --2- 172.21.15.2:0/760443352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00c9000 0x7fb3c015e150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 --2- 172.21.15.2:0/760443352 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c00c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 -- 172.21.15.2:0/760443352 >> 172.21.15.2:0/760443352 conn(0x7fb3c0070f90 msgr2=0x7fb3c006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 -- 172.21.15.2:0/760443352 shutdown_connections 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.833+0000 7fb3c56af640 1 -- 172.21.15.2:0/760443352 wait complete. 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.834+0000 7fb3c56af640 1 Processor -- start 2024-09-19T05:42:41.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.834+0000 7fb3c56af640 1 -- start start 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3c56af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c0153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3c56af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c00c9000 0x7fb3c01f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3c56af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00cbcc0 0x7fb3c01f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3c56af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c015a910 con 0x7fb3c00c9000 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3c56af640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c015b500 con 0x7fb3c00cbcc0 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3c56af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c00caf20 con 0x7fb3c00c5620 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c00c9000 0x7fb3c01f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c00c9000 0x7fb3c01f04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47548/0 (socket says 172.21.15.2:47548) 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 -- 172.21.15.2:0/2670236459 learned_addr learned my addr 172.21.15.2:0/2670236459 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:41.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3beffd640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c0153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3bf7fe640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00cbcc0 0x7fb3c01f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 -- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 msgr2=0x7fb3c0153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c0153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 -- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00cbcc0 msgr2=0x7fb3c01f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00cbcc0 0x7fb3c01f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3be7fc640 1 -- 172.21.15.2:0/2670236459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c0151d80 con 0x7fb3c00c9000 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.835+0000 7fb3beffd640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.836+0000 7fb3be7fc640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c00c9000 0x7fb3c01f04e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c470 tx=0x7fb3b400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.836+0000 7fb3a77fe640 1 -- 172.21.15.2:0/2670236459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b40074e0 con 0x7fb3c00c9000 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.836+0000 7fb3a77fe640 1 -- 172.21.15.2:0/2670236459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b4007680 con 0x7fb3c00c9000 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.836+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c0152060 con 0x7fb3c00c9000 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.836+0000 7fb3a77fe640 1 -- 172.21.15.2:0/2670236459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4004a00 con 0x7fb3c00c9000 2024-09-19T05:42:41.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.836+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c01f3ca0 con 0x7fb3c00c9000 2024-09-19T05:42:41.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.837+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c0152580 con 0x7fb3c00c9000 2024-09-19T05:42:41.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.837+0000 7fb3a77fe640 1 -- 172.21.15.2:0/2670236459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb3b401a020 con 0x7fb3c00c9000 2024-09-19T05:42:41.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.838+0000 7fb3a77fe640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb3ac06c2b0 0x7fb3ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:41.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.838+0000 7fb3a77fe640 1 -- 172.21.15.2:0/2670236459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb3b408ab30 con 0x7fb3c00c9000 2024-09-19T05:42:41.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.838+0000 7fb3beffd640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb3ac06c2b0 0x7fb3ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:41.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.839+0000 7fb3beffd640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb3ac06c2b0 0x7fb3ac06e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009b90 tx=0x7fb3b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:41.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:41.842+0000 7fb3a77fe640 1 -- 172.21.15.2:0/2670236459 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb3b4010300 con 0x7fb3c00c9000 2024-09-19T05:42:42.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.054+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3c015ee10 con 0x7fb3c00c9000 2024-09-19T05:42:42.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.055+0000 7fb3a77fe640 1 -- 172.21.15.2:0/2670236459 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6863 (secure 0 0 0) 0x7fb3b4059330 con 0x7fb3c00c9000 2024-09-19T05:42:42.057 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:42.058 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"btime":"2024-09-19T05:42:38:988574+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/121690198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":121690198},{"type":"v1","addr":"172.21.15.2:6835","nonce":121690198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1118046326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1118046326},{"type":"v1","addr":"172.21.15.123:6837","nonce":1118046326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3264935286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3264935286},{"type":"v1","addr":"172.21.15.2:6837","nonce":3264935286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/4246839966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4246839966},{"type":"v1","addr":"172.21.15.123:6836","nonce":4246839966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:42:32.334667+0000","modified":"2024-09-19T05:42:38.988568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/4093916922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4093916922},{"type":"v1","addr":"172.21.15.123:6839","nonce":4093916922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4312,"qdb_cluster":[4312]},"id":1}]} 2024-09-19T05:42:42.058 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb3ac06c2b0 msgr2=0x7fb3ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb3ac06c2b0 0x7fb3ac06e770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009b90 tx=0x7fb3b0008040 comp rx=0 tx=0).stop 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c00c9000 msgr2=0x7fb3c01f04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c00c9000 0x7fb3c01f04e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3b400c470 tx=0x7fb3b400c930 comp rx=0 tx=0).stop 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 shutdown_connections 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb3ac06c2b0 0x7fb3ac06e770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3c00cbcc0 0x7fb3c01f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c00c9000 0x7fb3c01f04e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 --2- 172.21.15.2:0/2670236459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb3c00c5620 0x7fb3c0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 >> 172.21.15.2:0/2670236459 conn(0x7fb3c0070f90 msgr2=0x7fb3c00ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:42.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 shutdown_connections 2024-09-19T05:42:42.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.057+0000 7fb3c56af640 1 -- 172.21.15.2:0/2670236459 wait complete. 2024-09-19T05:42:42.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:42:42.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.169+0000 7fa3808ac640 1 Processor -- start 2024-09-19T05:42:42.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.169+0000 7fa3808ac640 1 -- start start 2024-09-19T05:42:42.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.170+0000 7fa3808ac640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa37c1529b0 0x7fa37c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.170+0000 7fa3808ac640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa37c158880 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.170+0000 7fa3808ac640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa37c0c3730 con 0x7fa37c159320 2024-09-19T05:42:42.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.170+0000 7fa3808ac640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fa37c159770 con 0x7fa37c1529b0 2024-09-19T05:42:42.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.171+0000 7fa37a59c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa37c1529b0 0x7fa37c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.171+0000 7fa37ad9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa37c0cb0c0 0x7fa37c1630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41148/0 (socket says 172.21.15.2:41148) 2024-09-19T05:42:42.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.171+0000 7fa37ad9d640 1 -- 172.21.15.2:0/3254868800 learned_addr learned my addr 172.21.15.2:0/3254868800 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:42.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.171+0000 7fa37a59c640 1 -- 172.21.15.2:0/3254868800 >> v1:172.21.15.123:6789/0 conn(0x7fa37c159320 legacy=0x7fa37c0ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.171+0000 7fa37a59c640 1 -- 172.21.15.2:0/3254868800 >> v1:172.21.15.2:6789/0 conn(0x7fa37c0cb0c0 legacy=0x7fa37c1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.171+0000 7fa37a59c640 1 -- 172.21.15.2:0/3254868800 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c0c3a10 con 0x7fa37c1529b0 2024-09-19T05:42:42.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa37a59c640 1 --2- 172.21.15.2:0/3254868800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c1529b0 0x7fa37c15bd40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b655105cbbf91a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:42.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa37959a640 1 -- 172.21.15.2:0/3254868800 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c037070 con 0x7fa37c1529b0 2024-09-19T05:42:42.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa37959a640 1 -- 172.21.15.2:0/3254868800 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36c032440 con 0x7fa37c1529b0 2024-09-19T05:42:42.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa37959a640 1 -- 172.21.15.2:0/3254868800 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c032810 con 0x7fa37c1529b0 2024-09-19T05:42:42.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa3808ac640 1 -- 172.21.15.2:0/3254868800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c1529b0 msgr2=0x7fa37c15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa3808ac640 1 --2- 172.21.15.2:0/3254868800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c1529b0 0x7fa37c15bd40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0097a0 tx=0x7fa36c02a230 comp rx=0 tx=0).stop 2024-09-19T05:42:42.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa3808ac640 1 -- 172.21.15.2:0/3254868800 shutdown_connections 2024-09-19T05:42:42.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa3808ac640 1 --2- 172.21.15.2:0/3254868800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c1529b0 0x7fa37c15bd40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa3808ac640 1 -- 172.21.15.2:0/3254868800 >> 172.21.15.2:0/3254868800 conn(0x7fa37c06bc10 msgr2=0x7fa37c06f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:42.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa3808ac640 1 -- 172.21.15.2:0/3254868800 shutdown_connections 2024-09-19T05:42:42.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.172+0000 7fa3808ac640 1 -- 172.21.15.2:0/3254868800 wait complete. 2024-09-19T05:42:42.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 Processor -- start 2024-09-19T05:42:42.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 -- start start 2024-09-19T05:42:42.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c0cb0c0 0x7fa37c1e20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa37c1529b0 0x7fa37c149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa37c159320 0x7fa37c14dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c0c3cf0 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c158e00 con 0x7fa37c159320 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa3808ac640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa37c0c3e70 con 0x7fa37c1529b0 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa37a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c0cb0c0 0x7fa37c1e20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.174+0000 7fa37a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c0cb0c0 0x7fa37c1e20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47560/0 (socket says 172.21.15.2:47560) 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37a59c640 1 -- 172.21.15.2:0/831913619 learned_addr learned my addr 172.21.15.2:0/831913619 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37ad9d640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa37c159320 0x7fa37c14dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa379d9b640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa37c1529b0 0x7fa37c149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37a59c640 1 -- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa37c1529b0 msgr2=0x7fa37c149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37a59c640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa37c1529b0 0x7fa37c149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37a59c640 1 -- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa37c159320 msgr2=0x7fa37c14dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37a59c640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa37c159320 0x7fa37c14dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37a59c640 1 -- 172.21.15.2:0/831913619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c1eeea0 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37ad9d640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa37c159320 0x7fa37c14dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa37a59c640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c0cb0c0 0x7fa37c1e20c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0098c0 tx=0x7fa36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa36affd640 1 -- 172.21.15.2:0/831913619 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c037040 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa36affd640 1 -- 172.21.15.2:0/831913619 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36c030770 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c1ef180 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa36affd640 1 -- 172.21.15.2:0/831913619 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c030b40 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.175+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c0c5a10 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.176+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c152d90 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.177+0000 7fa36affd640 1 -- 172.21.15.2:0/831913619 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa36c044020 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.177+0000 7fa36affd640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa35c06c2e0 0x7fa35c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.177+0000 7fa36affd640 1 -- 172.21.15.2:0/831913619 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa36c0ab8e0 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.177+0000 7fa379d9b640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa35c06c2e0 0x7fa35c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.178+0000 7fa379d9b640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa35c06c2e0 0x7fa35c06e7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa370007fd0 tx=0x7fa370009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:42.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.182+0000 7fa36affd640 1 -- 172.21.15.2:0/831913619 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa36c032e20 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.358+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa37c0cbbe0 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.360+0000 7fa36affd640 1 -- 172.21.15.2:0/831913619 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16621 (secure 0 0 0) 0x7fa36c07a0e0 con 0x7fa37c0cb0c0 2024-09-19T05:42:42.362 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:42.362 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:42:33.917577+0000","last_up_change":"2024-09-19T05:42:20.842106+0000","last_in_change":"2024-09-19T05:42:11.870630+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6801","nonce":3690741328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6803","nonce":3690741328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6807","nonce":3690741328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3690741328},{"type":"v1","addr":"172.21.15.123:6805","nonce":3690741328}]},"public_addr":"172.21.15.123:6801/3690741328","cluster_addr":"172.21.15.123:6803/3690741328","heartbeat_back_addr":"172.21.15.123:6807/3690741328","heartbeat_front_addr":"172.21.15.123:6805/3690741328","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:42:42.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa35c06c2e0 msgr2=0x7fa35c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa35c06c2e0 0x7fa35c06e7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa370007fd0 tx=0x7fa370009040 comp rx=0 tx=0).stop 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c0cb0c0 msgr2=0x7fa37c1e20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c0cb0c0 0x7fa37c1e20c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0098c0 tx=0x7fa36c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 shutdown_connections 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa35c06c2e0 0x7fa35c06e7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa37c159320 0x7fa37c14dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa37c1529b0 0x7fa37c149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 --2- 172.21.15.2:0/831913619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa37c0cb0c0 0x7fa37c1e20c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 >> 172.21.15.2:0/831913619 conn(0x7fa37c06bc10 msgr2=0x7fa37c071630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 shutdown_connections 2024-09-19T05:42:42.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.362+0000 7fa3808ac640 1 -- 172.21.15.2:0/831913619 wait complete. 2024-09-19T05:42:42.375 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:42:42.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.475+0000 7fbcd0bf6640 1 Processor -- start 2024-09-19T05:42:42.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.475+0000 7fbcd0bf6640 1 -- start start 2024-09-19T05:42:42.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcd0bf6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbccc154e70 0x7fbccc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcd0bf6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc0cadd0 0x7fbccc0cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcd0bf6640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbccc0cd780 con 0x7fbccc0cc670 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcd0bf6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fbccc155250 con 0x7fbccc154e70 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcd0bf6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbccc0cd4a0 con 0x7fbccc0cadd0 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcca59c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbccc154e70 0x7fbccc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcc9d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fbccc0cc670 0x7fbccc15aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41154/0 (socket says 172.21.15.2:41154) 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcc9d9b640 1 -- 172.21.15.2:0/3038638502 learned_addr learned my addr 172.21.15.2:0/3038638502 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcca59c640 1 -- 172.21.15.2:0/3038638502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc0cadd0 msgr2=0x7fbccc0cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcca59c640 1 --2- 172.21.15.2:0/3038638502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc0cadd0 0x7fbccc0cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcca59c640 1 -- 172.21.15.2:0/3038638502 >> v1:172.21.15.2:6789/0 conn(0x7fbccc0cc670 legacy=0x7fbccc15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcc959a640 1 -- 172.21.15.2:0/3038638502 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820762879 0 0) 0x7fbccc0cd780 con 0x7fbccc0cc670 2024-09-19T05:42:42.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.476+0000 7fbcca59c640 1 -- 172.21.15.2:0/3038638502 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc15e0b0 con 0x7fbccc154e70 2024-09-19T05:42:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.477+0000 7fbcca59c640 1 --2- 172.21.15.2:0/3038638502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc154e70 0x7fbccc0cce60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40097a0 tx=0x7fbcc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc362cefc0aaedb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.477+0000 7fbcc959a640 1 -- 172.21.15.2:0/3038638502 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4037070 con 0x7fbccc154e70 2024-09-19T05:42:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.477+0000 7fbcc959a640 1 -- 172.21.15.2:0/3038638502 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc4032440 con 0x7fbccc154e70 2024-09-19T05:42:42.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.477+0000 7fbcc959a640 1 -- 172.21.15.2:0/3038638502 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4032810 con 0x7fbccc154e70 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/3038638502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc154e70 msgr2=0x7fbccc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/3038638502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc154e70 0x7fbccc0cce60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40097a0 tx=0x7fbcc402a180 comp rx=0 tx=0).stop 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/3038638502 shutdown_connections 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/3038638502 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc0cadd0 0x7fbccc0cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/3038638502 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc154e70 0x7fbccc0cce60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/3038638502 >> 172.21.15.2:0/3038638502 conn(0x7fbccc06da40 msgr2=0x7fbccc073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/3038638502 shutdown_connections 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.478+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/3038638502 wait complete. 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.479+0000 7fbcd0bf6640 1 Processor -- start 2024-09-19T05:42:42.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.479+0000 7fbcd0bf6640 1 -- start start 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcd0bf6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbccc0cadd0 0x7fbccc14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcd0bf6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc0cc670 0x7fbccc14f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcd0bf6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc154e70 0x7fbccc0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcd0bf6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc15e950 con 0x7fbccc0cc670 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcd0bf6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc15e670 con 0x7fbccc0cadd0 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcd0bf6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbccc15ee70 con 0x7fbccc154e70 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc0cc670 0x7fbccc14f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc0cc670 0x7fbccc14f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47576/0 (socket says 172.21.15.2:47576) 2024-09-19T05:42:42.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 -- 172.21.15.2:0/2300228055 learned_addr learned my addr 172.21.15.2:0/2300228055 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcca59c640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbccc0cadd0 0x7fbccc14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbccad9d640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc154e70 0x7fbccc0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 -- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc154e70 msgr2=0x7fbccc0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc154e70 0x7fbccc0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 -- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbccc0cadd0 msgr2=0x7fbccc14fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbccc0cadd0 0x7fbccc14fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.480+0000 7fbcc9d9b640 1 -- 172.21.15.2:0/2300228055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc1f15a0 con 0x7fbccc0cc670 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.481+0000 7fbcc9d9b640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc0cc670 0x7fbccc14f010 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c4a0 tx=0x7fbcb400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.481+0000 7fbcc2ffd640 1 -- 172.21.15.2:0/2300228055 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4018070 con 0x7fbccc0cc670 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.481+0000 7fbcc2ffd640 1 -- 172.21.15.2:0/2300228055 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb40074e0 con 0x7fbccc0cc670 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.481+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc0c89d0 con 0x7fbccc0cc670 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.481+0000 7fbcc2ffd640 1 -- 172.21.15.2:0/2300228055 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4004a20 con 0x7fbccc0cc670 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.481+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc0c8f90 con 0x7fbccc0cc670 2024-09-19T05:42:42.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.482+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc0cca90 con 0x7fbccc0cc670 2024-09-19T05:42:42.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.482+0000 7fbcc2ffd640 1 -- 172.21.15.2:0/2300228055 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbcb401b070 con 0x7fbccc0cc670 2024-09-19T05:42:42.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.483+0000 7fbcc2ffd640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbcac06c2b0 0x7fbcac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:42.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.483+0000 7fbcc2ffd640 1 -- 172.21.15.2:0/2300228055 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbcb408a6a0 con 0x7fbccc0cc670 2024-09-19T05:42:42.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.483+0000 7fbcca59c640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbcac06c2b0 0x7fbcac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:42.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.483+0000 7fbcca59c640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbcac06c2b0 0x7fbcac06e770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4034000 tx=0x7fbcc402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:42.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.487+0000 7fbcc2ffd640 1 -- 172.21.15.2:0/2300228055 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbcb40587c0 con 0x7fbccc0cc670 2024-09-19T05:42:42.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.696+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbccc0cb910 con 0x7fbccc0cc670 2024-09-19T05:42:42.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.696+0000 7fbcc2ffd640 1 -- 172.21.15.2:0/2300228055 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6863 (secure 0 0 0) 0x7fbcb405cac0 con 0x7fbccc0cc670 2024-09-19T05:42:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:42.699 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"btime":"2024-09-19T05:42:38:988574+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/121690198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":121690198},{"type":"v1","addr":"172.21.15.2:6835","nonce":121690198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1118046326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1118046326},{"type":"v1","addr":"172.21.15.123:6837","nonce":1118046326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3264935286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3264935286},{"type":"v1","addr":"172.21.15.2:6837","nonce":3264935286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/4246839966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4246839966},{"type":"v1","addr":"172.21.15.123:6836","nonce":4246839966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:42:32.334667+0000","modified":"2024-09-19T05:42:38.988568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/4093916922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4093916922},{"type":"v1","addr":"172.21.15.123:6839","nonce":4093916922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4312,"qdb_cluster":[4312]},"id":1}]} 2024-09-19T05:42:42.699 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbcac06c2b0 msgr2=0x7fbcac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbcac06c2b0 0x7fbcac06e770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4034000 tx=0x7fbcc402e000 comp rx=0 tx=0).stop 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc0cc670 msgr2=0x7fbccc14f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc0cc670 0x7fbccc14f010 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c4a0 tx=0x7fbcb400c960 comp rx=0 tx=0).stop 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 shutdown_connections 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbcac06c2b0 0x7fbcac06e770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbccc154e70 0x7fbccc0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbccc0cc670 0x7fbccc14f010 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 --2- 172.21.15.2:0/2300228055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbccc0cadd0 0x7fbccc14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:42.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 >> 172.21.15.2:0/2300228055 conn(0x7fbccc06da40 msgr2=0x7fbccc0c5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:42.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 shutdown_connections 2024-09-19T05:42:42.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:42.698+0000 7fbcd0bf6640 1 -- 172.21.15.2:0/2300228055 wait complete. 2024-09-19T05:42:43.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T05:42:32.334667+0000', 'modified': '2024-09-19T05:42:38.988568+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4312': {'gid': 4312, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.123:6839/4093916922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 4093916922}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 4093916922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4312, 'qdb_cluster': [4312]} 2024-09-19T05:42:43.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T05:42:43.315 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-09-19T05:42:43.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.422+0000 7f8b041c4640 1 Processor -- start 2024-09-19T05:42:43.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.422+0000 7f8b041c4640 1 -- start start 2024-09-19T05:42:43.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b041c4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8afc154c20 0x7f8afc0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b041c4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8afc155000 0x7f8afc073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b041c4640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8afc0ccbc0 con 0x7f8afc0caf40 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b041c4640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8afc0cc600 con 0x7f8afc155000 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b041c4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8afc0cc8e0 con 0x7f8afc154c20 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b0175f640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8afc155000 0x7f8afc073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b02761640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8afc0caf40 0x7f8afc15aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41158/0 (socket says 172.21.15.2:41158) 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b02761640 1 -- 172.21.15.2:0/3693191934 learned_addr learned my addr 172.21.15.2:0/3693191934 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3693191934 >> v2:172.21.15.123:3300/0 conn(0x7f8afc154c20 0x7f8afc0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b0175f640 1 -- 172.21.15.2:0/3693191934 >> v2:172.21.15.123:3300/0 conn(0x7f8afc154c20 msgr2=0x7f8afc0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b0175f640 1 --2- 172.21.15.2:0/3693191934 >> v2:172.21.15.123:3300/0 conn(0x7f8afc154c20 0x7f8afc0cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b0175f640 1 -- 172.21.15.2:0/3693191934 >> v1:172.21.15.2:6789/0 conn(0x7f8afc0caf40 legacy=0x7f8afc15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:43.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.423+0000 7f8b0175f640 1 -- 172.21.15.2:0/3693191934 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc0cbc60 con 0x7f8afc155000 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.424+0000 7f8b0175f640 1 --2- 172.21.15.2:0/3693191934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc155000 0x7f8afc073340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8afc073ed0 tx=0x7f8af002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48be06b4445d5d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.424+0000 7f8b00f5e640 1 -- 172.21.15.2:0/3693191934 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0037070 con 0x7f8afc155000 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.424+0000 7f8b00f5e640 1 -- 172.21.15.2:0/3693191934 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af0032440 con 0x7f8afc155000 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.424+0000 7f8b00f5e640 1 -- 172.21.15.2:0/3693191934 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0032810 con 0x7f8afc155000 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 -- 172.21.15.2:0/3693191934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc155000 msgr2=0x7f8afc073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 --2- 172.21.15.2:0/3693191934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc155000 0x7f8afc073340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8afc073ed0 tx=0x7f8af002a1a0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 -- 172.21.15.2:0/3693191934 shutdown_connections 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 --2- 172.21.15.2:0/3693191934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc155000 0x7f8afc073340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 --2- 172.21.15.2:0/3693191934 >> v2:172.21.15.123:3300/0 conn(0x7f8afc154c20 0x7f8afc0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 -- 172.21.15.2:0/3693191934 >> 172.21.15.2:0/3693191934 conn(0x7f8afc06d0e0 msgr2=0x7f8afc06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 -- 172.21.15.2:0/3693191934 shutdown_connections 2024-09-19T05:42:43.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.425+0000 7f8b041c4640 1 -- 172.21.15.2:0/3693191934 wait complete. 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.426+0000 7f8b041c4640 1 Processor -- start 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.426+0000 7f8b041c4640 1 -- start start 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b041c4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc0caf40 0x7f8afc1e3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b041c4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8afc154c20 0x7f8afc158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b041c4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8afc155000 0x7f8afc1f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b041c4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc15b1f0 con 0x7f8afc0caf40 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b041c4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0cb430 con 0x7f8afc155000 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b041c4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc15b370 con 0x7f8afc154c20 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc0caf40 0x7f8afc1e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc0caf40 0x7f8afc1e3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47590/0 (socket says 172.21.15.2:47590) 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 -- 172.21.15.2:0/275914580 learned_addr learned my addr 172.21.15.2:0/275914580 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:43.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b02761640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8afc155000 0x7f8afc1f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b0175f640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8afc154c20 0x7f8afc158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 -- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8afc154c20 msgr2=0x7f8afc158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8afc154c20 0x7f8afc158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 -- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8afc155000 msgr2=0x7f8afc1f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8afc155000 0x7f8afc1f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.427+0000 7f8b01f60640 1 -- 172.21.15.2:0/275914580 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc0ce9a0 con 0x7f8afc0caf40 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.428+0000 7f8b01f60640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc0caf40 0x7f8afc1e3050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c820 tx=0x7f8af400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.428+0000 7f8afa7fc640 1 -- 172.21.15.2:0/275914580 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4015070 con 0x7f8afc0caf40 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.428+0000 7f8afa7fc640 1 -- 172.21.15.2:0/275914580 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af4004240 con 0x7f8afc0caf40 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.428+0000 7f8afa7fc640 1 -- 172.21.15.2:0/275914580 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af400ce10 con 0x7f8afc0caf40 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.428+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8afc1e4820 con 0x7f8afc0caf40 2024-09-19T05:42:43.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.428+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8afc0ce3e0 con 0x7f8afc0caf40 2024-09-19T05:42:43.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.430+0000 7f8afa7fc640 1 -- 172.21.15.2:0/275914580 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8af4007830 con 0x7f8afc0caf40 2024-09-19T05:42:43.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.430+0000 7f8afa7fc640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8ae806c2b0 0x7f8ae806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.430+0000 7f8afa7fc640 1 -- 172.21.15.2:0/275914580 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8af408b290 con 0x7f8afc0caf40 2024-09-19T05:42:43.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.430+0000 7f8b0175f640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8ae806c2b0 0x7f8ae806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.431+0000 7f8b0175f640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8ae806c2b0 0x7f8ae806e770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8af0034000 tx=0x7f8af002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:43.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.431+0000 7f8b07038740 1 Processor -- start 2024-09-19T05:42:43.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.431+0000 7f8b07038740 1 -- start start 2024-09-19T05:42:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791750 0x562b3c7c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b07038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791b30 0x562b3c78da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b07038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562b3c6f2520 con 0x562b3c78ef80 2024-09-19T05:42:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3c69a3e0 con 0x562b3c791b30 2024-09-19T05:42:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x562b3c6ee4e0 con 0x562b3c791750 2024-09-19T05:42:43.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47598/0 (socket says 172.21.15.2:47598) 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 -- 172.21.15.2:0/573969836 learned_addr learned my addr 172.21.15.2:0/573969836 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b02761640 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791b30 0x562b3c78da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b01f60640 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791750 0x562b3c7c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 -- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791750 msgr2=0x562b3c7c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791750 0x562b3c7c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 -- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791b30 msgr2=0x562b3c78da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791b30 0x562b3c78da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 -- 172.21.15.2:0/573969836 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b3c6b52f0 con 0x562b3c78ef80 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.432+0000 7f8b0175f640 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7c9f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009850 tx=0x7f8af002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465db0e3489a9964 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8ae37fe640 1 -- 172.21.15.2:0/573969836 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af004d070 con 0x562b3c78ef80 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8ae37fe640 1 -- 172.21.15.2:0/573969836 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af003e460 con 0x562b3c78ef80 2024-09-19T05:42:43.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8ae37fe640 1 -- 172.21.15.2:0/573969836 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0037040 con 0x562b3c78ef80 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 -- 172.21.15.2:0/573969836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 msgr2=0x562b3c7c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7c9f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8af0009850 tx=0x7f8af002c000 comp rx=0 tx=0).stop 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 -- 172.21.15.2:0/573969836 shutdown_connections 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791b30 0x562b3c78da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7c9f20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 --2- 172.21.15.2:0/573969836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791750 0x562b3c7c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 -- 172.21.15.2:0/573969836 >> 172.21.15.2:0/573969836 conn(0x562b3c78fe70 msgr2=0x562b3c790270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 -- 172.21.15.2:0/573969836 shutdown_connections 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.433+0000 7f8b07038740 1 -- 172.21.15.2:0/573969836 wait complete. 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.434+0000 7f8b07038740 1 -- start start 2024-09-19T05:42:43.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.434+0000 7f8b07038740 1 Processor -- start 2024-09-19T05:42:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b07038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c7b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c7b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b07038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562b3c790fe0 con 0x562b3c78ef80 2024-09-19T05:42:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3c67b450 con 0x562b3c791750 2024-09-19T05:42:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x562b3c7c9550 con 0x562b3c791b30 2024-09-19T05:42:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b07038740 10 client.0 _ll_register_callbacks cb 0x562b3c404ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7e1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47612/0 (socket says 172.21.15.2:47612) 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 -- 172.21.15.2:0/2802374176 learned_addr learned my addr 172.21.15.2:0/2802374176 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b0175f640 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c7b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b02761640 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c7b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 -- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 msgr2=0x562b3c7b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c7b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 -- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 msgr2=0x562b3c7b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c7b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 -- 172.21.15.2:0/2802374176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3c4052d0 con 0x562b3c78ef80 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8b01f60640 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7e1060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8af4004630 tx=0x7f8af40594d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba73d3472a9b759d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8ae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-19T05:42:43.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.435+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/2802374176 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af401e040 con 0x562b3c78ef80 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/2802374176 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8af4071380 con 0x562b3c78ef80 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af003e460 con 0x562b3c78ef80 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/2802374176 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f8af4061ae0 con 0x562b3c78ef80 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 10 client.4545 fetch_fsmap learned FSMap version 8 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x562b3c4cf6b0 con 0x562b3c78ef80 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/2802374176 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af003e460 con 0x562b3c78ef80 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/2802374176 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 8) ==== 4712+0+0 (secure 0 0 0) 0x7f8af40587e0 con 0x562b3c78ef80 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 10 client.4545 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 10 client.4545 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 20 client.4545 populate_metadata read hostname 'smithi002' 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c7b3520 0x562b3c7ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 4 client.4545 mds_command: new command op to 4312 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T05:42:43.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 --> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b3c6ffc50 con 0x562b3c7b3520 2024-09-19T05:42:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.436+0000 7f8b0175f640 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c7b3520 0x562b3c7ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.437+0000 7f8b0175f640 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c7b3520 0x562b3c7ef9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b7843cf7b9256 server_cookie=d6a1f9226d52181f in_seq=0 out_seq=0 2024-09-19T05:42:43.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.437+0000 7f8ae0ff9640 10 client.4545 ms_handle_connect on v2:172.21.15.123:6838/4093916922 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/2802374176 <== mds.0 v2:172.21.15.123:6838/4093916922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b3c6ffc50 con 0x562b3c7b3520 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8ae0ff9640 10 client.4545 handle_command_reply: tid=0 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 client.4545 shutdown 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 msgr2=0x562b3c7e1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7e1060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8af4004630 tx=0x7f8af40594d0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 shutdown_connections 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c7b3520 0x562b3c7ef9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c7b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c7b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 --2- 172.21.15.2:0/2802374176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7e1060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 >> 172.21.15.2:0/2802374176 conn(0x562b3c78fe70 msgr2=0x562b3c7ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8ae0ff9640 0 client.4545 ms_handle_reset on v2:172.21.15.123:6838/4093916922 2024-09-19T05:42:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 shutdown_connections 2024-09-19T05:42:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.994+0000 7f8b07038740 1 -- 172.21.15.2:0/2802374176 wait complete. 2024-09-19T05:42:43.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:43.995+0000 7f8b07038740 20 client.4545 trim_cache size 0 max 16384 2024-09-19T05:42:44.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.014+0000 7f8b07038740 1 Processor -- start 2024-09-19T05:42:44.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.014+0000 7f8b07038740 1 -- start start 2024-09-19T05:42:44.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b07038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c78f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b07038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0b5c60 con 0x562b3c78ef80 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3c7c4290 con 0x562b3c791750 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x562b3c7b6470 con 0x562b3c791b30 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b01f60640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b01f60640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47620/0 (socket says 172.21.15.2:47620) 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b01f60640 1 -- 172.21.15.2:0/3046094525 learned_addr learned my addr 172.21.15.2:0/3046094525 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b02761640 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c78f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.015+0000 7f8b0175f640 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8b01f60640 1 -- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 msgr2=0x562b3c78f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c78f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8b01f60640 1 -- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 msgr2=0x562b3c78f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:44.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8b01f60640 1 -- 172.21.15.2:0/3046094525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b3c7c56d0 con 0x562b3c78ef80 2024-09-19T05:42:44.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7b5470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8af400cfd0 tx=0x7f8af408b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be658b167f89887e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:44.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8ae2ffd640 1 -- 172.21.15.2:0/3046094525 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af401e040 con 0x562b3c78ef80 2024-09-19T05:42:44.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8ae2ffd640 1 -- 172.21.15.2:0/3046094525 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af4078350 con 0x562b3c78ef80 2024-09-19T05:42:44.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.016+0000 7f8ae2ffd640 1 -- 172.21.15.2:0/3046094525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4071180 con 0x562b3c78ef80 2024-09-19T05:42:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 -- 172.21.15.2:0/3046094525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 msgr2=0x562b3c7b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7b5470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8af400cfd0 tx=0x7f8af408b050 comp rx=0 tx=0).stop 2024-09-19T05:42:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 -- 172.21.15.2:0/3046094525 shutdown_connections 2024-09-19T05:42:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c791b30 0x562b3c78f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 --2- 172.21.15.2:0/3046094525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c78ef80 0x562b3c7b5470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 -- 172.21.15.2:0/3046094525 >> 172.21.15.2:0/3046094525 conn(0x562b3c78fe70 msgr2=0x562b3c7c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 -- 172.21.15.2:0/3046094525 shutdown_connections 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.017+0000 7f8b07038740 1 -- 172.21.15.2:0/3046094525 wait complete. 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.018+0000 7f8b07038740 1 -- start start 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.018+0000 7f8b07038740 1 Processor -- start 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c78ef80 0x562b3c7dd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b07038740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b07038740 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c791b30 0x562b3c78e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b07038740 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x562b3c7b8190 con 0x562b3c791b30 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3c7b5cd0 con 0x562b3c791750 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b07038740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x562b3c7e4300 con 0x562b3c78ef80 2024-09-19T05:42:44.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b07038740 10 client.0 _ll_register_callbacks cb 0x7f8af40556d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c791b30 0x562b3c78e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c791b30 0x562b3c78e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47636/0 (socket says 172.21.15.2:47636) 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 -- 172.21.15.2:0/3623863951 learned_addr learned my addr 172.21.15.2:0/3623863951 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c78ef80 0x562b3c7dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b0175f640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 -- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c78ef80 msgr2=0x562b3c7dd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c78ef80 0x562b3c7dd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 -- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 msgr2=0x562b3c78db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 -- 172.21.15.2:0/3623863951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3c78d580 con 0x562b3c791b30 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c78ef80 0x562b3c7dd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b0175f640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:44.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8b02761640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c791b30 0x562b3c78e0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0012e0 tx=0x7f8aec016a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6033ebe560ba0a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.019+0000 7f8ae0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/3623863951 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec004d70 con 0x562b3c791b30 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/3623863951 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8aec017550 con 0x562b3c791b30 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8af4078350 con 0x562b3c791b30 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/3623863951 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f8aec027050 con 0x562b3c791b30 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 10 client.4551 fetch_fsmap learned FSMap version 8 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x562b3c7b4040 con 0x562b3c791b30 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/3623863951 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af4078350 con 0x562b3c791b30 2024-09-19T05:42:44.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/3623863951 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 8) ==== 4712+0+0 (secure 0 0 0) 0x7f8aec02a020 con 0x562b3c791b30 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 10 client.4551 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 10 client.4551 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 20 client.4551 populate_metadata read hostname 'smithi002' 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c84f1d0 0x562b3c84f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 4 client.4551 mds_command: new command op to 4312 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.020+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 --> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] -- command(tid 0: {"prefix": "status"}) -- 0x562b3c6f9fa0 con 0x562b3c84f1d0 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c84f1d0 0x562b3c84f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b01f60640 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c84f1d0 0x562b3c84f5b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5796132d1d69d4 server_cookie=901cb4bd58cb96a5 in_seq=0 out_seq=0 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8ae0ff9640 10 client.4551 ms_handle_connect on v2:172.21.15.123:6838/4093916922 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8ae0ff9640 1 -- 172.21.15.2:0/3623863951 <== mds.0 v2:172.21.15.123:6838/4093916922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x562b3c6f9fa0 con 0x562b3c84f1d0 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8ae0ff9640 10 client.4551 handle_command_reply: tid=0 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 client.4551 shutdown 2024-09-19T05:42:44.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c791b30 msgr2=0x562b3c78e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c791b30 0x562b3c78e0c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0012e0 tx=0x7f8aec016a80 comp rx=0 tx=0).stop 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 shutdown_connections 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:6838/4093916922,v1:172.21.15.123:6839/4093916922] conn(0x562b3c84f1d0 0x562b3c84f5b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3c791750 0x562b3c78db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562b3c791b30 0x562b3c78e0c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 --2- 172.21.15.2:0/3623863951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x562b3c78ef80 0x562b3c7dd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8ae0ff9640 0 client.4551 ms_handle_reset on v2:172.21.15.123:6838/4093916922 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 >> 172.21.15.2:0/3623863951 conn(0x562b3c78fe70 msgr2=0x562b3c7c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 shutdown_connections 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.021+0000 7f8b07038740 1 -- 172.21.15.2:0/3623863951 wait complete. 2024-09-19T05:42:44.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b07038740 20 client.4551 trim_cache size 0 max 16384 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "896e162c-4658-4743-b70e-7ec2b5bce950", 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4312, 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 11.051557492000001, 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 8, 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 22, 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 22, 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 16.350676419999999 2024-09-19T05:42:44.031 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:44.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8ae806c2b0 msgr2=0x7f8ae806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8ae806c2b0 0x7f8ae806e770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8af0034000 tx=0x7f8af002e000 comp rx=0 tx=0).stop 2024-09-19T05:42:44.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc0caf40 msgr2=0x7f8afc1e3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc0caf40 0x7f8afc1e3050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c820 tx=0x7f8af400cce0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 shutdown_connections 2024-09-19T05:42:44.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8ae806c2b0 0x7f8ae806e770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8afc155000 0x7f8afc1f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8afc154c20 0x7f8afc158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 --2- 172.21.15.2:0/275914580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8afc0caf40 0x7f8afc1e3050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 >> 172.21.15.2:0/275914580 conn(0x7f8afc06d0e0 msgr2=0x7f8afc153940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 shutdown_connections 2024-09-19T05:42:44.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.022+0000 7f8b041c4640 1 -- 172.21.15.2:0/275914580 wait complete. 2024-09-19T05:42:44.039 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T05:42:44.046 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T05:42:44.046 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T05:42:44.047 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T05:42:44.047 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T05:42:44.101 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T05:42:44.102 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T05:42:44.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:44.102 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-09-19T05:42:44.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:44.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-09-19T05:42:44.191 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2024-09-19T05:42:44.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:42:44.193 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-19T05:42:44.193 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T05:42:44.193 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:44.193 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T05:42:44.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.337+0000 7f3df638d640 1 Processor -- start 2024-09-19T05:42:44.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.337+0000 7f3df638d640 1 -- start start 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3df638d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3df00cc5f0 0x7f3df015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3df638d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df00734c0 con 0x7f3df0154cb0 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3df638d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df0073780 con 0x7f3df00cd7c0 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3df638d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f3df00cd330 con 0x7f3df00cc5f0 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3def7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3df00cc5f0 0x7f3df015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3deffff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3df0154cb0 0x7f3df00c55f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41168/0 (socket says 172.21.15.2:41168) 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3deffff640 1 -- 172.21.15.2:0/2437733321 learned_addr learned my addr 172.21.15.2:0/2437733321 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3def7fe640 1 -- 172.21.15.2:0/2437733321 >> v1:172.21.15.123:6789/0 conn(0x7f3df00cd7c0 legacy=0x7f3df00cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3def7fe640 1 -- 172.21.15.2:0/2437733321 >> v1:172.21.15.2:6789/0 conn(0x7f3df0154cb0 legacy=0x7f3df00c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3deeffd640 1 -- 172.21.15.2:0/2437733321 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297964529 0 0) 0x7f3df00734c0 con 0x7f3df0154cb0 2024-09-19T05:42:44.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3def7fe640 1 -- 172.21.15.2:0/2437733321 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df0073a60 con 0x7f3df00cc5f0 2024-09-19T05:42:44.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3def7fe640 1 --2- 172.21.15.2:0/2437733321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df015a810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3de40097a0 tx=0x7f3de402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af82e19cb58545 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3deeffd640 1 -- 172.21.15.2:0/2437733321 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4037070 con 0x7f3df00cc5f0 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3deeffd640 1 -- 172.21.15.2:0/2437733321 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de4032440 con 0x7f3df00cc5f0 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.338+0000 7f3deeffd640 1 -- 172.21.15.2:0/2437733321 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4032810 con 0x7f3df00cc5f0 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.340+0000 7f3df638d640 1 -- 172.21.15.2:0/2437733321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 msgr2=0x7f3df015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.340+0000 7f3df638d640 1 --2- 172.21.15.2:0/2437733321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df015a810 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3de40097a0 tx=0x7f3de402a270 comp rx=0 tx=0).stop 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.340+0000 7f3df638d640 1 -- 172.21.15.2:0/2437733321 shutdown_connections 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.340+0000 7f3df638d640 1 --2- 172.21.15.2:0/2437733321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df015a810 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.340+0000 7f3df638d640 1 -- 172.21.15.2:0/2437733321 >> 172.21.15.2:0/2437733321 conn(0x7f3df006d0c0 msgr2=0x7f3df006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.340+0000 7f3df638d640 1 -- 172.21.15.2:0/2437733321 shutdown_connections 2024-09-19T05:42:44.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.340+0000 7f3df638d640 1 -- 172.21.15.2:0/2437733321 wait complete. 2024-09-19T05:42:44.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.341+0000 7f3df638d640 1 Processor -- start 2024-09-19T05:42:44.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.341+0000 7f3df638d640 1 -- start start 2024-09-19T05:42:44.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3df638d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df0150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3df638d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00cd7c0 0x7f3df01533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3df638d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3df0154cb0 0x7f3df0153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3df638d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3df00cdbf0 con 0x7f3df00cc5f0 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3df638d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3df00b30d0 con 0x7f3df00cd7c0 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3df638d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3df00cb4d0 con 0x7f3df0154cb0 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df0150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df0150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47654/0 (socket says 172.21.15.2:47654) 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 -- 172.21.15.2:0/1897774553 learned_addr learned my addr 172.21.15.2:0/1897774553 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 -- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3df0154cb0 msgr2=0x7f3df0153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3df0154cb0 0x7f3df0153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 -- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00cd7c0 msgr2=0x7f3df01533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3def7fe640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00cd7c0 0x7f3df01533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00cd7c0 0x7f3df01533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3deffff640 1 -- 172.21.15.2:0/1897774553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df014fa00 con 0x7f3df00cc5f0 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.342+0000 7f3def7fe640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00cd7c0 0x7f3df01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.343+0000 7f3deffff640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df0150ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c430 tx=0x7f3de000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.343+0000 7f3decff9640 1 -- 172.21.15.2:0/1897774553 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00074e0 con 0x7f3df00cc5f0 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.343+0000 7f3decff9640 1 -- 172.21.15.2:0/1897774553 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de00047e0 con 0x7f3df00cc5f0 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.343+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df0151520 con 0x7f3df00cc5f0 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.343+0000 7f3decff9640 1 -- 172.21.15.2:0/1897774553 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0004bd0 con 0x7f3df00cc5f0 2024-09-19T05:42:44.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.343+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01e4b10 con 0x7f3df00cc5f0 2024-09-19T05:42:44.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.344+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df015a810 con 0x7f3df00cc5f0 2024-09-19T05:42:44.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.344+0000 7f3decff9640 1 -- 172.21.15.2:0/1897774553 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3de0011070 con 0x7f3df00cc5f0 2024-09-19T05:42:44.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.345+0000 7f3decff9640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3ddc06c2b0 0x7f3ddc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:44.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.345+0000 7f3decff9640 1 -- 172.21.15.2:0/1897774553 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3de008afe0 con 0x7f3df00cc5f0 2024-09-19T05:42:44.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.345+0000 7f3def7fe640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3ddc06c2b0 0x7f3ddc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:44.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.346+0000 7f3def7fe640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3ddc06c2b0 0x7f3ddc06e770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3de4034000 tx=0x7f3de402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:44.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.349+0000 7f3decff9640 1 -- 172.21.15.2:0/1897774553 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3de00597e0 con 0x7f3df00cc5f0 2024-09-19T05:42:44.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.560+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3df00cc070 con 0x7f3df00cc5f0 2024-09-19T05:42:44.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.560+0000 7f3decff9640 1 -- 172.21.15.2:0/1897774553 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f3de00590e0 con 0x7f3df00cc5f0 2024-09-19T05:42:44.561 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T05:42:44.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3ddc06c2b0 msgr2=0x7f3ddc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3ddc06c2b0 0x7f3ddc06e770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3de4034000 tx=0x7f3de402e000 comp rx=0 tx=0).stop 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 msgr2=0x7f3df0150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df0150ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3de000c430 tx=0x7f3de000c8f0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 shutdown_connections 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3ddc06c2b0 0x7f3ddc06e770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00cd7c0 0x7f3df01533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3df0154cb0 0x7f3df0153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 --2- 172.21.15.2:0/1897774553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3df00cc5f0 0x7f3df0150ad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 >> 172.21.15.2:0/1897774553 conn(0x7f3df006d0c0 msgr2=0x7f3df0151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 shutdown_connections 2024-09-19T05:42:44.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44.562+0000 7f3df638d640 1 -- 172.21.15.2:0/1897774553 wait complete. 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T05:42:44.574 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:42:44.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:44.575 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2024-09-19T05:42:44.590 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f0 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp7s0f1 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:42:44.591 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f0 2024-09-19T05:42:44.592 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T05:42:44.592 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42766sec preferred_lft 42766sec 2024-09-19T05:42:44.592 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2024-09-19T05:42:44.592 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2024-09-19T05:42:44.592 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T05:42:44.592 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2024-09-19T05:42:44.592 INFO:teuthology.orchestra.run.smithi002.stdout: altname enp1s0f1 2024-09-19T05:42:44.592 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T05:42:44.593 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:42:44.593 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-19T05:42:44.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2024-09-19T05:42:44.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2024-09-19T05:42:44.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2024-09-19T05:42:44.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T05:42:44.593 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-19T05:42:44.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:42:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:44.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:44.801 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T05:42:44.828 INFO:teuthology.orchestra.run.smithi002.stdout:1 2024-09-19T05:42:44.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:44.829 DEBUG:teuthology.orchestra.run.smithi002:> route 2024-09-19T05:42:44.906 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2024-09-19T05:42:44.906 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T05:42:44.907 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T05:42:44.907 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T05:42:44.907 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T05:42:44.907 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:42:44.907 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-19T05:42:44.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T05:42:44.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T05:42:44.907 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T05:42:44.907 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-19T05:42:44.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:42:45.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:45.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:45.253 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-09-19T05:42:45.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:45.269 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2024-09-19T05:42:45.326 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:42:45.326 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-19T05:42:45.326 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:42:45.326 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T05:42:45.326 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-19T05:42:45.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:42:45.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:45.449 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T05:42:45.449 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:42:45.449 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-19T05:42:45.449 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T05:42:45.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T05:42:45.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T05:42:45.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T05:42:45.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T05:42:45.450 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-19T05:42:45.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:42:45.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:45.609 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T05:42:45.609 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2024-09-19T05:42:45.609 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2024-09-19T05:42:45.609 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T05:42:45.609 DEBUG:teuthology.orchestra.run.smithi002:> ') 2024-09-19T05:42:45.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:42:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:45.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:42:45.690 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:45.706 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T05:42:45.707 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T05:42:45.707 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:45.762 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-19T05:42:45.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:45.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-19T05:42:46.039 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-19T05:42:46.039 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-19T05:42:46.039 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2024-09-19T05:42:46.551 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-19T05:42:46.552 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-19T05:42:46.552 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2024-09-19T05:42:46.552 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.123:3300,172.21.15.123:3301" 2024-09-19T05:42:46.552 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-19T05:42:46.557 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-19T05:42:46.557 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-19T05:42:46.557 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2024-09-19T05:42:46.558 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.123:3300,172.21.15.123:3301" 2024-09-19T05:42:46.558 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-19T05:42:46.558 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.123:6789,172.21.15.123:6790" 2024-09-19T05:42:46.558 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.123:6789,172.21.15.123:6790:/ 2024-09-19T05:42:46.558 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=896e162c-4658-4743-b70e-7ec2b5bce950" will pass to kernel 2024-09-19T05:42:46.558 INFO:teuthology.orchestra.run.smithi002.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T05:42:46.558 INFO:teuthology.orchestra.run.smithi002.stdout: You just mounted a file system that supports labels which does not 2024-09-19T05:42:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T05:42:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T05:42:46.559 INFO:teuthology.orchestra.run.smithi002.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T05:42:46.560 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T05:42:46.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:42:46.561 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:46.589 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:42:46.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:42:46.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:46.696 INFO:teuthology.orchestra.run.smithi002.stdout:client4534 2024-09-19T05:42:46.697 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:42:46.697 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:42:46.714 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:42:46.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:42:46.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:46.817 INFO:teuthology.orchestra.run.smithi002.stdout:client4534 2024-09-19T05:42:46.817 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T05:42:46.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T05:42:46.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:46.880 INFO:teuthology.orchestra.run.smithi002.stdout:client4534 2024-09-19T05:42:46.881 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/896e162c-4658-4743-b70e-7ec2b5bce950.client4534/status') 2024-09-19T05:42:46.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T05:42:46.945 INFO:teuthology.orchestra.run.smithi002.stdout:instance: client.4534 (3)172.21.15.2:0/3907881870 2024-09-19T05:42:46.945 INFO:teuthology.orchestra.run.smithi002.stdout:blocklisted: false 2024-09-19T05:42:46.945 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2024-09-19T05:42:46.945 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2024-09-19T05:42:46.945 INFO:teuthology.orchestra.run.smithi002.stderr:69 bytes copied, 2.5812e-05 s, 2.7 MB/s 2024-09-19T05:42:46.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T05:42:46.952 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-19T05:42:46.997 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-19T05:42:46.997 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e'}} 2024-09-19T05:42:46.997 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-19T05:42:46.997 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-19T05:42:47.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.102+0000 7f12b1d0a640 1 Processor -- start 2024-09-19T05:42:47.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.102+0000 7f12b1d0a640 1 -- start start 2024-09-19T05:42:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12b1d0a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12ac0cb640 0x7f12ac0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12b1d0a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f12ac15ede0 0x7f12ac0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12b1d0a640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ac15ab40 con 0x7f12ac0cba20 2024-09-19T05:42:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12b1d0a640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac0c5ef0 con 0x7f12ac15ede0 2024-09-19T05:42:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12b1d0a640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac15ade0 con 0x7f12ac0cb640 2024-09-19T05:42:47.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12abfff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f12ac15ede0 0x7f12ac0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12aaffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f12ac0cba20 0x7f12ac1549e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41172/0 (socket says 172.21.15.2:41172) 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.103+0000 7f12aaffd640 1 -- 172.21.15.2:0/858003891 learned_addr learned my addr 172.21.15.2:0/858003891 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.104+0000 7f12ab7fe640 1 --2- 172.21.15.2:0/858003891 >> v2:172.21.15.123:3300/0 conn(0x7f12ac0cb640 0x7f12ac0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.104+0000 7f12abfff640 1 -- 172.21.15.2:0/858003891 >> v2:172.21.15.123:3300/0 conn(0x7f12ac0cb640 msgr2=0x7f12ac0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.104+0000 7f12abfff640 1 --2- 172.21.15.2:0/858003891 >> v2:172.21.15.123:3300/0 conn(0x7f12ac0cb640 0x7f12ac0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.104+0000 7f12abfff640 1 -- 172.21.15.2:0/858003891 >> v1:172.21.15.2:6789/0 conn(0x7f12ac0cba20 legacy=0x7f12ac1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.104+0000 7f12abfff640 1 -- 172.21.15.2:0/858003891 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac15b0c0 con 0x7f12ac15ede0 2024-09-19T05:42:47.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.104+0000 7f12aa7fc640 1 -- 172.21.15.2:0/858003891 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165249688 0 0) 0x7f12ac15ab40 con 0x7f12ac0cba20 2024-09-19T05:42:47.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.104+0000 7f12abfff640 1 --2- 172.21.15.2:0/858003891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac15ede0 0x7f12ac0c6380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f12ac073fd0 tx=0x7f12a402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34a702ecaeeeda0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12aa7fc640 1 -- 172.21.15.2:0/858003891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4007740 con 0x7f12ac15ede0 2024-09-19T05:42:47.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12aa7fc640 1 -- 172.21.15.2:0/858003891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a40078e0 con 0x7f12ac15ede0 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12aa7fc640 1 -- 172.21.15.2:0/858003891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4007cb0 con 0x7f12ac15ede0 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 -- 172.21.15.2:0/858003891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac15ede0 msgr2=0x7f12ac0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/858003891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac15ede0 0x7f12ac0c6380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f12ac073fd0 tx=0x7f12a402a710 comp rx=0 tx=0).stop 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 -- 172.21.15.2:0/858003891 shutdown_connections 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/858003891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac15ede0 0x7f12ac0c6380 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/858003891 >> v2:172.21.15.123:3300/0 conn(0x7f12ac0cb640 0x7f12ac0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 -- 172.21.15.2:0/858003891 >> 172.21.15.2:0/858003891 conn(0x7f12ac067950 msgr2=0x7f12ac067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 -- 172.21.15.2:0/858003891 shutdown_connections 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.105+0000 7f12b1d0a640 1 -- 172.21.15.2:0/858003891 wait complete. 2024-09-19T05:42:47.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.106+0000 7f12b1d0a640 1 Processor -- start 2024-09-19T05:42:47.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12b1d0a640 1 -- start start 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12b1d0a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12ac0cb640 0x7f12ac1f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12b1d0a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac0cba20 0x7f12ac0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12b1d0a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12ac15ede0 0x7f12ac0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12b1d0a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac15b960 con 0x7f12ac0cba20 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12b1d0a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac15b680 con 0x7f12ac0cb640 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12b1d0a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac1550f0 con 0x7f12ac15ede0 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.107+0000 7f12aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac0cba20 0x7f12ac0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac0cba20 0x7f12ac0b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47662/0 (socket says 172.21.15.2:47662) 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 -- 172.21.15.2:0/4230575509 learned_addr learned my addr 172.21.15.2:0/4230575509 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12ab7fe640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12ac0cb640 0x7f12ac1f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12abfff640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12ac15ede0 0x7f12ac0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 -- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12ac15ede0 msgr2=0x7f12ac0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12ac15ede0 0x7f12ac0ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 -- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12ac0cb640 msgr2=0x7f12ac1f05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12ac0cb640 0x7f12ac1f05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 -- 172.21.15.2:0/4230575509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac14e060 con 0x7f12ac0cba20 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12ab7fe640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12ac0cb640 0x7f12ac1f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12abfff640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12ac15ede0 0x7f12ac0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12aaffd640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac0cba20 0x7f12ac0b5a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c470 tx=0x7f12a000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f1293fff640 1 -- 172.21.15.2:0/4230575509 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0017070 con 0x7f12ac0cba20 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f1293fff640 1 -- 172.21.15.2:0/4230575509 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12a0005ce0 con 0x7f12ac0cba20 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f1293fff640 1 -- 172.21.15.2:0/4230575509 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0007310 con 0x7f12ac0cba20 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac14e300 con 0x7f12ac0cba20 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.108+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac1f40d0 con 0x7f12ac0cba20 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.109+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac06e090 con 0x7f12ac0cba20 2024-09-19T05:42:47.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.110+0000 7f1293fff640 1 -- 172.21.15.2:0/4230575509 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f12a001a070 con 0x7f12ac0cba20 2024-09-19T05:42:47.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.110+0000 7f1293fff640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f129806c2e0 0x7f129806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.110+0000 7f1293fff640 1 -- 172.21.15.2:0/4230575509 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f12a008a880 con 0x7f12ac0cba20 2024-09-19T05:42:47.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.110+0000 7f12ab7fe640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f129806c2e0 0x7f129806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.111+0000 7f12ab7fe640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f129806c2e0 0x7f129806e7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f129c009b90 tx=0x7f129c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.115+0000 7f1293fff640 1 -- 172.21.15.2:0/4230575509 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f12ac06e090 con 0x7f12ac0cba20 2024-09-19T05:42:47.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.292+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) -- 0x7f12ac1549e0 con 0x7f12ac0cba20 2024-09-19T05:42:47.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.295+0000 7f1293fff640 1 -- 172.21.15.2:0/4230575509 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v10)=0 v10) ==== 136+0+0 (secure 0 0 0) 0x7f12a0058ff0 con 0x7f12ac0cba20 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f129806c2e0 msgr2=0x7f129806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f129806c2e0 0x7f129806e7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f129c009b90 tx=0x7f129c008040 comp rx=0 tx=0).stop 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac0cba20 msgr2=0x7f12ac0b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac0cba20 0x7f12ac0b5a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f12a000c470 tx=0x7f12a000c930 comp rx=0 tx=0).stop 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 shutdown_connections 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f129806c2e0 0x7f129806e7a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12ac15ede0 0x7f12ac0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12ac0cba20 0x7f12ac0b5a50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 --2- 172.21.15.2:0/4230575509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12ac0cb640 0x7f12ac1f05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 >> 172.21.15.2:0/4230575509 conn(0x7f12ac067950 msgr2=0x7f12ac067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 shutdown_connections 2024-09-19T05:42:47.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.297+0000 7f12b1d0a640 1 -- 172.21.15.2:0/4230575509 wait complete. 2024-09-19T05:42:47.309 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-19T05:42:47.309 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T05:42:47.309 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:42:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.409+0000 7f59a405c640 1 Processor -- start 2024-09-19T05:42:47.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.410+0000 7f59a405c640 1 -- start start 2024-09-19T05:42:47.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.410+0000 7f59a405c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f599c0cce60 0x7f599c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.410+0000 7f59a405c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f599c0cd780 con 0x7f599c154e70 2024-09-19T05:42:47.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.410+0000 7f59a405c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f599c0cd4a0 con 0x7f599c15afe0 2024-09-19T05:42:47.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.410+0000 7f59a405c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f599c0cd260 con 0x7f599c0cce60 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a1df8640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f599c154e70 0x7f599c0cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41176/0 (socket says 172.21.15.2:41176) 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a1df8640 1 -- 172.21.15.2:0/4261766217 learned_addr learned my addr 172.21.15.2:0/4261766217 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a15f7640 1 --2- 172.21.15.2:0/4261766217 >> v2:172.21.15.123:3300/0 conn(0x7f599c0cce60 0x7f599c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3140318528 0 0) 0x7f599c0cd780 con 0x7f599c154e70 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5988003520 con 0x7f599c154e70 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1106660276 0 0) 0x7f5988003520 con 0x7f599c154e70 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 >> v2:172.21.15.123:3300/0 conn(0x7f599c0cce60 msgr2=0x7f599c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 --2- 172.21.15.2:0/4261766217 >> v2:172.21.15.123:3300/0 conn(0x7f599c0cce60 0x7f599c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 >> v1:172.21.15.123:6789/0 conn(0x7f599c15afe0 legacy=0x7f599c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c15e0b0 con 0x7f599c154e70 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f5998001670 con 0x7f599c154e70 2024-09-19T05:42:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 >> v1:172.21.15.2:6789/0 conn(0x7f599c154e70 legacy=0x7f599c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5988003a20 0x7f5988003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f5988004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59880051c0 0x7f598800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f599c0cd260 con 0x7f5988004800 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f598800c040 con 0x7f5988003a20 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f598800c240 con 0x7f59880051c0 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a15f7640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f5988004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a1df8640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5988003a20 0x7f5988003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.411+0000 7f59a25f9640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59880051c0 0x7f598800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a15f7640 1 -- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59880051c0 msgr2=0x7f598800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a15f7640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59880051c0 0x7f598800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a15f7640 1 -- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5988003a20 msgr2=0x7f5988003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:47.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a15f7640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5988003a20 0x7f5988003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a15f7640 1 -- 172.21.15.2:0/4261766217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5988003520 con 0x7f5988004800 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a25f9640 1 -- 172.21.15.2:0/4261766217 reap_dead start 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a15f7640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f5988004c80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f598c009ac0 tx=0x7f5988005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297a8ecbb480172b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c036070 con 0x7f5988004800 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.412+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598c031400 con 0x7f5988004800 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a0df6640 1 -- 172.21.15.2:0/4261766217 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c009690 con 0x7f5988004800 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a405c640 1 -- 172.21.15.2:0/4261766217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 msgr2=0x7f5988004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a405c640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f5988004c80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f598c009ac0 tx=0x7f5988005830 comp rx=0 tx=0).stop 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a405c640 1 -- 172.21.15.2:0/4261766217 shutdown_connections 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a405c640 1 --2- 172.21.15.2:0/4261766217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f5988004c80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a405c640 1 -- 172.21.15.2:0/4261766217 >> 172.21.15.2:0/4261766217 conn(0x7f599c06da40 msgr2=0x7f599c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a405c640 1 -- 172.21.15.2:0/4261766217 shutdown_connections 2024-09-19T05:42:47.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.413+0000 7f59a405c640 1 -- 172.21.15.2:0/4261766217 wait complete. 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.414+0000 7f59a405c640 1 Processor -- start 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.414+0000 7f59a405c640 1 -- start start 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a405c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f599c0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a405c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599c0d0f00 0x7f599c1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a405c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f599c158970 0x7f599c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a405c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f599c15e670 con 0x7f5988004800 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a405c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f599c155250 con 0x7f599c0d0f00 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a405c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f599c15e950 con 0x7f599c158970 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a1df8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f599c0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a1df8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f599c0ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47678/0 (socket says 172.21.15.2:47678) 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a1df8640 1 -- 172.21.15.2:0/535532885 learned_addr learned my addr 172.21.15.2:0/535532885 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a25f9640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f599c158970 0x7f599c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a15f7640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599c0d0f00 0x7f599c1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a1df8640 1 -- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f599c158970 msgr2=0x7f599c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a1df8640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f599c158970 0x7f599c0cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a1df8640 1 -- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599c0d0f00 msgr2=0x7f599c1f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.415+0000 7f59a1df8640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599c0d0f00 0x7f599c1f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59a1df8640 1 -- 172.21.15.2:0/535532885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c0b4280 con 0x7f5988004800 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59a15f7640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599c0d0f00 0x7f599c1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59a1df8640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f599c0ceb90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5998004f20 tx=0x7f59980074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59927fc640 1 -- 172.21.15.2:0/535532885 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599801a070 con 0x7f5988004800 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59927fc640 1 -- 172.21.15.2:0/535532885 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599800f750 con 0x7f5988004800 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599c0b3fb0 con 0x7f5988004800 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59927fc640 1 -- 172.21.15.2:0/535532885 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599800fb20 con 0x7f5988004800 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.416+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599c0b4800 con 0x7f5988004800 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.417+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f599c0caac0 con 0x7f5988004800 2024-09-19T05:42:47.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.417+0000 7f59927fc640 1 -- 172.21.15.2:0/535532885 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f599801d070 con 0x7f5988004800 2024-09-19T05:42:47.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.418+0000 7f59927fc640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f598006c2e0 0x7f598006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.418+0000 7f59927fc640 1 -- 172.21.15.2:0/535532885 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f599808aaa0 con 0x7f5988004800 2024-09-19T05:42:47.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.418+0000 7f59a15f7640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f598006c2e0 0x7f598006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.419+0000 7f59a15f7640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f598006c2e0 0x7f598006e7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f598c033000 tx=0x7f598c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.422+0000 7f59927fc640 1 -- 172.21.15.2:0/535532885 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5998058c50 con 0x7f5988004800 2024-09-19T05:42:47.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.594+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f599c0cd780 con 0x7f598006c2e0 2024-09-19T05:42:47.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.601+0000 7f59927fc640 1 -- 172.21.15.2:0/535532885 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251910 (secure 0 0 0) 0x7f599c0cd780 con 0x7f598006c2e0 2024-09-19T05:42:47.604 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:47.606 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:42:47.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f598006c2e0 msgr2=0x7f598006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f598006c2e0 0x7f598006e7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f598c033000 tx=0x7f598c008030 comp rx=0 tx=0).stop 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 msgr2=0x7f599c0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f599c0ceb90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5998004f20 tx=0x7f59980074f0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 shutdown_connections 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f598006c2e0 0x7f598006e7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f599c0d0f00 0x7f599c1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f599c158970 0x7f599c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 --2- 172.21.15.2:0/535532885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5988004800 0x7f599c0ceb90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 >> 172.21.15.2:0/535532885 conn(0x7f599c06da40 msgr2=0x7f599c152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 shutdown_connections 2024-09-19T05:42:47.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.604+0000 7f59a405c640 1 -- 172.21.15.2:0/535532885 wait complete. 2024-09-19T05:42:47.615 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-19T05:42:45.822458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234796,"kb_used_data":1952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1998848,"data_stored":1818484,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.899155"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058235+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:42:34.058235+0000","last_peered":"2024-09-19T05:42:34.058235+0000","last_clean":"2024-09-19T05:42:34.058235+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:42:34.058235+0000","last_undegraded":"2024-09-19T05:42:34.058235+0000","last_fullsized":"2024-09-19T05:42:34.058235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058494+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:42:34.058494+0000","last_peered":"2024-09-19T05:42:34.058494+0000","last_clean":"2024-09-19T05:42:34.058494+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:42:34.058494+0000","last_undegraded":"2024-09-19T05:42:34.058494+0000","last_fullsized":"2024-09-19T05:42:34.058494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058017+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:42:34.058017+0000","last_peered":"2024-09-19T05:42:34.058017+0000","last_clean":"2024-09-19T05:42:34.058017+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:42:34.058017+0000","last_undegraded":"2024-09-19T05:42:34.058017+0000","last_fullsized":"2024-09-19T05:42:34.058017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923708+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:42:33.923708+0000","last_peered":"2024-09-19T05:42:33.923708+0000","last_clean":"2024-09-19T05:42:33.923708+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:42:33.923708+0000","last_undegraded":"2024-09-19T05:42:33.923708+0000","last_fullsized":"2024-09-19T05:42:33.923708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057846+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:42:34.057846+0000","last_peered":"2024-09-19T05:42:34.057846+0000","last_clean":"2024-09-19T05:42:34.057846+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:42:34.057846+0000","last_undegraded":"2024-09-19T05:42:34.057846+0000","last_fullsized":"2024-09-19T05:42:34.057846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923679+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:42:33.923679+0000","last_peered":"2024-09-19T05:42:33.923679+0000","last_clean":"2024-09-19T05:42:33.923679+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:42:33.923679+0000","last_undegraded":"2024-09-19T05:42:33.923679+0000","last_fullsized":"2024-09-19T05:42:33.923679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057764+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:42:34.057764+0000","last_peered":"2024-09-19T05:42:34.057764+0000","last_clean":"2024-09-19T05:42:34.057764+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:42:34.057764+0000","last_undegraded":"2024-09-19T05:42:34.057764+0000","last_fullsized":"2024-09-19T05:42:34.057764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058259+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:42:34.058259+0000","last_peered":"2024-09-19T05:42:34.058259+0000","last_clean":"2024-09-19T05:42:34.058259+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:42:34.058259+0000","last_undegraded":"2024-09-19T05:42:34.058259+0000","last_fullsized":"2024-09-19T05:42:34.058259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058756+0000","last_change":"2024-09-19T05:42:32.945872+0000","last_active":"2024-09-19T05:42:34.058756+0000","last_peered":"2024-09-19T05:42:34.058756+0000","last_clean":"2024-09-19T05:42:34.058756+0000","last_became_active":"2024-09-19T05:42:32.945545+0000","last_became_peered":"2024-09-19T05:42:32.945545+0000","last_unstale":"2024-09-19T05:42:34.058756+0000","last_undegraded":"2024-09-19T05:42:34.058756+0000","last_fullsized":"2024-09-19T05:42:34.058756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058073+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:42:34.058073+0000","last_peered":"2024-09-19T05:42:34.058073+0000","last_clean":"2024-09-19T05:42:34.058073+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:42:34.058073+0000","last_undegraded":"2024-09-19T05:42:34.058073+0000","last_fullsized":"2024-09-19T05:42:34.058073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058096+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:42:34.058096+0000","last_peered":"2024-09-19T05:42:34.058096+0000","last_clean":"2024-09-19T05:42:34.058096+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:42:34.058096+0000","last_undegraded":"2024-09-19T05:42:34.058096+0000","last_fullsized":"2024-09-19T05:42:34.058096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058167+0000","last_change":"2024-09-19T05:42:32.941601+0000","last_active":"2024-09-19T05:42:34.058167+0000","last_peered":"2024-09-19T05:42:34.058167+0000","last_clean":"2024-09-19T05:42:34.058167+0000","last_became_active":"2024-09-19T05:42:32.941288+0000","last_became_peered":"2024-09-19T05:42:32.941288+0000","last_unstale":"2024-09-19T05:42:34.058167+0000","last_undegraded":"2024-09-19T05:42:34.058167+0000","last_fullsized":"2024-09-19T05:42:34.058167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:35.670719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058583+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:42:34.058583+0000","last_peered":"2024-09-19T05:42:34.058583+0000","last_clean":"2024-09-19T05:42:34.058583+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:42:34.058583+0000","last_undegraded":"2024-09-19T05:42:34.058583+0000","last_fullsized":"2024-09-19T05:42:34.058583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058482+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:42:34.058482+0000","last_peered":"2024-09-19T05:42:34.058482+0000","last_clean":"2024-09-19T05:42:34.058482+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:42:34.058482+0000","last_undegraded":"2024-09-19T05:42:34.058482+0000","last_fullsized":"2024-09-19T05:42:34.058482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058261+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:42:34.058261+0000","last_peered":"2024-09-19T05:42:34.058261+0000","last_clean":"2024-09-19T05:42:34.058261+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:42:34.058261+0000","last_undegraded":"2024-09-19T05:42:34.058261+0000","last_fullsized":"2024-09-19T05:42:34.058261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057516+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:42:34.057516+0000","last_peered":"2024-09-19T05:42:34.057516+0000","last_clean":"2024-09-19T05:42:34.057516+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:42:34.057516+0000","last_undegraded":"2024-09-19T05:42:34.057516+0000","last_fullsized":"2024-09-19T05:42:34.057516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059660+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:42:34.059660+0000","last_peered":"2024-09-19T05:42:34.059660+0000","last_clean":"2024-09-19T05:42:34.059660+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:42:34.059660+0000","last_undegraded":"2024-09-19T05:42:34.059660+0000","last_fullsized":"2024-09-19T05:42:34.059660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059534+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:42:34.059534+0000","last_peered":"2024-09-19T05:42:34.059534+0000","last_clean":"2024-09-19T05:42:34.059534+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:42:34.059534+0000","last_undegraded":"2024-09-19T05:42:34.059534+0000","last_fullsized":"2024-09-19T05:42:34.059534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058089+0000","last_change":"2024-09-19T05:42:32.940627+0000","last_active":"2024-09-19T05:42:34.058089+0000","last_peered":"2024-09-19T05:42:34.058089+0000","last_clean":"2024-09-19T05:42:34.058089+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058089+0000","last_undegraded":"2024-09-19T05:42:34.058089+0000","last_fullsized":"2024-09-19T05:42:34.058089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.616840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925755+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:42:33.925755+0000","last_peered":"2024-09-19T05:42:33.925755+0000","last_clean":"2024-09-19T05:42:33.925755+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:42:33.925755+0000","last_undegraded":"2024-09-19T05:42:33.925755+0000","last_fullsized":"2024-09-19T05:42:33.925755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058013+0000","last_change":"2024-09-19T05:42:32.940425+0000","last_active":"2024-09-19T05:42:34.058013+0000","last_peered":"2024-09-19T05:42:34.058013+0000","last_clean":"2024-09-19T05:42:34.058013+0000","last_became_active":"2024-09-19T05:42:32.940059+0000","last_became_peered":"2024-09-19T05:42:32.940059+0000","last_unstale":"2024-09-19T05:42:34.058013+0000","last_undegraded":"2024-09-19T05:42:34.058013+0000","last_fullsized":"2024-09-19T05:42:34.058013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058206+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:42:34.058206+0000","last_peered":"2024-09-19T05:42:34.058206+0000","last_clean":"2024-09-19T05:42:34.058206+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:42:34.058206+0000","last_undegraded":"2024-09-19T05:42:34.058206+0000","last_fullsized":"2024-09-19T05:42:34.058206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057512+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:42:34.057512+0000","last_peered":"2024-09-19T05:42:34.057512+0000","last_clean":"2024-09-19T05:42:34.057512+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:42:34.057512+0000","last_undegraded":"2024-09-19T05:42:34.057512+0000","last_fullsized":"2024-09-19T05:42:34.057512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058955+0000","last_change":"2024-09-19T05:42:32.941414+0000","last_active":"2024-09-19T05:42:34.058955+0000","last_peered":"2024-09-19T05:42:34.058955+0000","last_clean":"2024-09-19T05:42:34.058955+0000","last_became_active":"2024-09-19T05:42:32.941076+0000","last_became_peered":"2024-09-19T05:42:32.941076+0000","last_unstale":"2024-09-19T05:42:34.058955+0000","last_undegraded":"2024-09-19T05:42:34.058955+0000","last_fullsized":"2024-09-19T05:42:34.058955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:13.371046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923839+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:42:33.923839+0000","last_peered":"2024-09-19T05:42:33.923839+0000","last_clean":"2024-09-19T05:42:33.923839+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:42:33.923839+0000","last_undegraded":"2024-09-19T05:42:33.923839+0000","last_fullsized":"2024-09-19T05:42:33.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058867+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:42:34.058867+0000","last_peered":"2024-09-19T05:42:34.058867+0000","last_clean":"2024-09-19T05:42:34.058867+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:42:34.058867+0000","last_undegraded":"2024-09-19T05:42:34.058867+0000","last_fullsized":"2024-09-19T05:42:34.058867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058377+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:42:34.058377+0000","last_peered":"2024-09-19T05:42:34.058377+0000","last_clean":"2024-09-19T05:42:34.058377+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:42:34.058377+0000","last_undegraded":"2024-09-19T05:42:34.058377+0000","last_fullsized":"2024-09-19T05:42:34.058377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059114+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:42:34.059114+0000","last_peered":"2024-09-19T05:42:34.059114+0000","last_clean":"2024-09-19T05:42:34.059114+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:42:34.059114+0000","last_undegraded":"2024-09-19T05:42:34.059114+0000","last_fullsized":"2024-09-19T05:42:34.059114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924874+0000","last_change":"2024-09-19T05:42:32.927184+0000","last_active":"2024-09-19T05:42:33.924874+0000","last_peered":"2024-09-19T05:42:33.924874+0000","last_clean":"2024-09-19T05:42:33.924874+0000","last_became_active":"2024-09-19T05:42:32.926952+0000","last_became_peered":"2024-09-19T05:42:32.926952+0000","last_unstale":"2024-09-19T05:42:33.924874+0000","last_undegraded":"2024-09-19T05:42:33.924874+0000","last_fullsized":"2024-09-19T05:42:33.924874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923475+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:42:33.923475+0000","last_peered":"2024-09-19T05:42:33.923475+0000","last_clean":"2024-09-19T05:42:33.923475+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:42:33.923475+0000","last_undegraded":"2024-09-19T05:42:33.923475+0000","last_fullsized":"2024-09-19T05:42:33.923475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058161+0000","last_change":"2024-09-19T05:42:32.942584+0000","last_active":"2024-09-19T05:42:34.058161+0000","last_peered":"2024-09-19T05:42:34.058161+0000","last_clean":"2024-09-19T05:42:34.058161+0000","last_became_active":"2024-09-19T05:42:32.942274+0000","last_became_peered":"2024-09-19T05:42:32.942274+0000","last_unstale":"2024-09-19T05:42:34.058161+0000","last_undegraded":"2024-09-19T05:42:34.058161+0000","last_fullsized":"2024-09-19T05:42:34.058161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832292+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:42:34.832292+0000","last_peered":"2024-09-19T05:42:34.832292+0000","last_clean":"2024-09-19T05:42:34.832292+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:42:34.832292+0000","last_undegraded":"2024-09-19T05:42:34.832292+0000","last_fullsized":"2024-09-19T05:42:34.832292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925748+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:42:33.925748+0000","last_peered":"2024-09-19T05:42:33.925748+0000","last_clean":"2024-09-19T05:42:33.925748+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:42:33.925748+0000","last_undegraded":"2024-09-19T05:42:33.925748+0000","last_fullsized":"2024-09-19T05:42:33.925748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.839402+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:42:34.839402+0000","last_peered":"2024-09-19T05:42:34.839402+0000","last_clean":"2024-09-19T05:42:34.839402+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:42:34.839402+0000","last_undegraded":"2024-09-19T05:42:34.839402+0000","last_fullsized":"2024-09-19T05:42:34.839402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058211+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:42:34.058211+0000","last_peered":"2024-09-19T05:42:34.058211+0000","last_clean":"2024-09-19T05:42:34.058211+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:42:34.058211+0000","last_undegraded":"2024-09-19T05:42:34.058211+0000","last_fullsized":"2024-09-19T05:42:34.058211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058257+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:42:34.058257+0000","last_peered":"2024-09-19T05:42:34.058257+0000","last_clean":"2024-09-19T05:42:34.058257+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058257+0000","last_undegraded":"2024-09-19T05:42:34.058257+0000","last_fullsized":"2024-09-19T05:42:34.058257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057643+0000","last_change":"2024-09-19T05:42:31.995072+0000","last_active":"2024-09-19T05:42:34.057643+0000","last_peered":"2024-09-19T05:42:34.057643+0000","last_clean":"2024-09-19T05:42:34.057643+0000","last_became_active":"2024-09-19T05:42:31.994586+0000","last_became_peered":"2024-09-19T05:42:31.994586+0000","last_unstale":"2024-09-19T05:42:34.057643+0000","last_undegraded":"2024-09-19T05:42:34.057643+0000","last_fullsized":"2024-09-19T05:42:34.057643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925971+0000","last_change":"2024-09-19T05:42:32.928695+0000","last_active":"2024-09-19T05:42:33.925971+0000","last_peered":"2024-09-19T05:42:33.925971+0000","last_clean":"2024-09-19T05:42:33.925971+0000","last_became_active":"2024-09-19T05:42:32.928437+0000","last_became_peered":"2024-09-19T05:42:32.928437+0000","last_unstale":"2024-09-19T05:42:33.925971+0000","last_undegraded":"2024-09-19T05:42:33.925971+0000","last_fullsized":"2024-09-19T05:42:33.925971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058846+0000","last_change":"2024-09-19T05:42:31.985393+0000","last_active":"2024-09-19T05:42:34.058846+0000","last_peered":"2024-09-19T05:42:34.058846+0000","last_clean":"2024-09-19T05:42:34.058846+0000","last_became_active":"2024-09-19T05:42:31.985034+0000","last_became_peered":"2024-09-19T05:42:31.985034+0000","last_unstale":"2024-09-19T05:42:34.058846+0000","last_undegraded":"2024-09-19T05:42:34.058846+0000","last_fullsized":"2024-09-19T05:42:34.058846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:11.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923861+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:42:33.923861+0000","last_peered":"2024-09-19T05:42:33.923861+0000","last_clean":"2024-09-19T05:42:33.923861+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:42:33.923861+0000","last_undegraded":"2024-09-19T05:42:33.923861+0000","last_fullsized":"2024-09-19T05:42:33.923861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058193+0000","last_change":"2024-09-19T05:42:31.987634+0000","last_active":"2024-09-19T05:42:34.058193+0000","last_peered":"2024-09-19T05:42:34.058193+0000","last_clean":"2024-09-19T05:42:34.058193+0000","last_became_active":"2024-09-19T05:42:31.986452+0000","last_became_peered":"2024-09-19T05:42:31.986452+0000","last_unstale":"2024-09-19T05:42:34.058193+0000","last_undegraded":"2024-09-19T05:42:34.058193+0000","last_fullsized":"2024-09-19T05:42:34.058193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.981030+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:42:37.981030+0000","last_peered":"2024-09-19T05:42:37.981030+0000","last_clean":"2024-09-19T05:42:37.981030+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:42:37.981030+0000","last_undegraded":"2024-09-19T05:42:37.981030+0000","last_fullsized":"2024-09-19T05:42:37.981030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059270+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:42:34.059270+0000","last_peered":"2024-09-19T05:42:34.059270+0000","last_clean":"2024-09-19T05:42:34.059270+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:42:34.059270+0000","last_undegraded":"2024-09-19T05:42:34.059270+0000","last_fullsized":"2024-09-19T05:42:34.059270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.983225+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:42:37.983225+0000","last_peered":"2024-09-19T05:42:37.983225+0000","last_clean":"2024-09-19T05:42:37.983225+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:42:37.983225+0000","last_undegraded":"2024-09-19T05:42:37.983225+0000","last_fullsized":"2024-09-19T05:42:37.983225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058283+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:42:34.058283+0000","last_peered":"2024-09-19T05:42:34.058283+0000","last_clean":"2024-09-19T05:42:34.058283+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:42:34.058283+0000","last_undegraded":"2024-09-19T05:42:34.058283+0000","last_fullsized":"2024-09-19T05:42:34.058283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978898+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:42:37.978898+0000","last_peered":"2024-09-19T05:42:37.978898+0000","last_clean":"2024-09-19T05:42:37.978898+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:42:37.978898+0000","last_undegraded":"2024-09-19T05:42:37.978898+0000","last_fullsized":"2024-09-19T05:42:37.978898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058589+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:42:34.058589+0000","last_peered":"2024-09-19T05:42:34.058589+0000","last_clean":"2024-09-19T05:42:34.058589+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:42:34.058589+0000","last_undegraded":"2024-09-19T05:42:34.058589+0000","last_fullsized":"2024-09-19T05:42:34.058589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059325+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:42:34.059325+0000","last_peered":"2024-09-19T05:42:34.059325+0000","last_clean":"2024-09-19T05:42:34.059325+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:42:34.059325+0000","last_undegraded":"2024-09-19T05:42:34.059325+0000","last_fullsized":"2024-09-19T05:42:34.059325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059177+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:42:34.059177+0000","last_peered":"2024-09-19T05:42:34.059177+0000","last_clean":"2024-09-19T05:42:34.059177+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:42:34.059177+0000","last_undegraded":"2024-09-19T05:42:34.059177+0000","last_fullsized":"2024-09-19T05:42:34.059177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057760+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:42:34.057760+0000","last_peered":"2024-09-19T05:42:34.057760+0000","last_clean":"2024-09-19T05:42:34.057760+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:42:34.057760+0000","last_undegraded":"2024-09-19T05:42:34.057760+0000","last_fullsized":"2024-09-19T05:42:34.057760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058006+0000","last_change":"2024-09-19T05:42:32.938549+0000","last_active":"2024-09-19T05:42:34.058006+0000","last_peered":"2024-09-19T05:42:34.058006+0000","last_clean":"2024-09-19T05:42:34.058006+0000","last_became_active":"2024-09-19T05:42:32.938113+0000","last_became_peered":"2024-09-19T05:42:32.938113+0000","last_unstale":"2024-09-19T05:42:34.058006+0000","last_undegraded":"2024-09-19T05:42:34.058006+0000","last_fullsized":"2024-09-19T05:42:34.058006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:15.787698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.985533+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:42:37.985533+0000","last_peered":"2024-09-19T05:42:37.985533+0000","last_clean":"2024-09-19T05:42:37.985533+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:42:37.985533+0000","last_undegraded":"2024-09-19T05:42:37.985533+0000","last_fullsized":"2024-09-19T05:42:37.985533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.837827+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:42:34.837827+0000","last_peered":"2024-09-19T05:42:34.837827+0000","last_clean":"2024-09-19T05:42:34.837827+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:42:34.837827+0000","last_undegraded":"2024-09-19T05:42:34.837827+0000","last_fullsized":"2024-09-19T05:42:34.837827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923969+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:42:33.923969+0000","last_peered":"2024-09-19T05:42:33.923969+0000","last_clean":"2024-09-19T05:42:33.923969+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:42:33.923969+0000","last_undegraded":"2024-09-19T05:42:33.923969+0000","last_fullsized":"2024-09-19T05:42:33.923969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980561+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:42:37.980561+0000","last_peered":"2024-09-19T05:42:37.980561+0000","last_clean":"2024-09-19T05:42:37.980561+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:42:37.980561+0000","last_undegraded":"2024-09-19T05:42:37.980561+0000","last_fullsized":"2024-09-19T05:42:37.980561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058299+0000","last_change":"2024-09-19T05:42:32.945685+0000","last_active":"2024-09-19T05:42:34.058299+0000","last_peered":"2024-09-19T05:42:34.058299+0000","last_clean":"2024-09-19T05:42:34.058299+0000","last_became_active":"2024-09-19T05:42:32.945337+0000","last_became_peered":"2024-09-19T05:42:32.945337+0000","last_unstale":"2024-09-19T05:42:34.058299+0000","last_undegraded":"2024-09-19T05:42:34.058299+0000","last_fullsized":"2024-09-19T05:42:34.058299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832321+0000","last_change":"2024-09-19T05:42:31.991430+0000","last_active":"2024-09-19T05:42:34.832321+0000","last_peered":"2024-09-19T05:42:34.832321+0000","last_clean":"2024-09-19T05:42:34.832321+0000","last_became_active":"2024-09-19T05:42:31.991057+0000","last_became_peered":"2024-09-19T05:42:31.991057+0000","last_unstale":"2024-09-19T05:42:34.832321+0000","last_undegraded":"2024-09-19T05:42:34.832321+0000","last_fullsized":"2024-09-19T05:42:34.832321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:58.247495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058366+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:42:34.058366+0000","last_peered":"2024-09-19T05:42:34.058366+0000","last_clean":"2024-09-19T05:42:34.058366+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:42:34.058366+0000","last_undegraded":"2024-09-19T05:42:34.058366+0000","last_fullsized":"2024-09-19T05:42:34.058366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058851+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:42:34.058851+0000","last_peered":"2024-09-19T05:42:34.058851+0000","last_clean":"2024-09-19T05:42:34.058851+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:42:34.058851+0000","last_undegraded":"2024-09-19T05:42:34.058851+0000","last_fullsized":"2024-09-19T05:42:34.058851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059399+0000","last_change":"2024-09-19T05:42:32.941889+0000","last_active":"2024-09-19T05:42:34.059399+0000","last_peered":"2024-09-19T05:42:34.059399+0000","last_clean":"2024-09-19T05:42:34.059399+0000","last_became_active":"2024-09-19T05:42:32.941254+0000","last_became_peered":"2024-09-19T05:42:32.941254+0000","last_unstale":"2024-09-19T05:42:34.059399+0000","last_undegraded":"2024-09-19T05:42:34.059399+0000","last_fullsized":"2024-09-19T05:42:34.059399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.771973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057837+0000","last_change":"2024-09-19T05:42:23.458936+0000","last_active":"2024-09-19T05:42:34.057837+0000","last_peered":"2024-09-19T05:42:34.057837+0000","last_clean":"2024-09-19T05:42:34.057837+0000","last_became_active":"2024-09-19T05:42:23.457990+0000","last_became_peered":"2024-09-19T05:42:23.457990+0000","last_unstale":"2024-09-19T05:42:34.057837+0000","last_undegraded":"2024-09-19T05:42:34.057837+0000","last_fullsized":"2024-09-19T05:42:34.057837+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:45.259241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059347+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:42:34.059347+0000","last_peered":"2024-09-19T05:42:34.059347+0000","last_clean":"2024-09-19T05:42:34.059347+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:42:34.059347+0000","last_undegraded":"2024-09-19T05:42:34.059347+0000","last_fullsized":"2024-09-19T05:42:34.059347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925215+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:42:33.925215+0000","last_peered":"2024-09-19T05:42:33.925215+0000","last_clean":"2024-09-19T05:42:33.925215+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:42:33.925215+0000","last_undegraded":"2024-09-19T05:42:33.925215+0000","last_fullsized":"2024-09-19T05:42:33.925215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057735+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:42:34.057735+0000","last_peered":"2024-09-19T05:42:34.057735+0000","last_clean":"2024-09-19T05:42:34.057735+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:42:34.057735+0000","last_undegraded":"2024-09-19T05:42:34.057735+0000","last_fullsized":"2024-09-19T05:42:34.057735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058126+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:42:34.058126+0000","last_peered":"2024-09-19T05:42:34.058126+0000","last_clean":"2024-09-19T05:42:34.058126+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:42:34.058126+0000","last_undegraded":"2024-09-19T05:42:34.058126+0000","last_fullsized":"2024-09-19T05:42:34.058126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924861+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:42:33.924861+0000","last_peered":"2024-09-19T05:42:33.924861+0000","last_clean":"2024-09-19T05:42:33.924861+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:42:33.924861+0000","last_undegraded":"2024-09-19T05:42:33.924861+0000","last_fullsized":"2024-09-19T05:42:33.924861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057949+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:42:34.057949+0000","last_peered":"2024-09-19T05:42:34.057949+0000","last_clean":"2024-09-19T05:42:34.057949+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:42:34.057949+0000","last_undegraded":"2024-09-19T05:42:34.057949+0000","last_fullsized":"2024-09-19T05:42:34.057949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059345+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:42:34.059345+0000","last_peered":"2024-09-19T05:42:34.059345+0000","last_clean":"2024-09-19T05:42:34.059345+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:42:34.059345+0000","last_undegraded":"2024-09-19T05:42:34.059345+0000","last_fullsized":"2024-09-19T05:42:34.059345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057870+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:42:34.057870+0000","last_peered":"2024-09-19T05:42:34.057870+0000","last_clean":"2024-09-19T05:42:34.057870+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:42:34.057870+0000","last_undegraded":"2024-09-19T05:42:34.057870+0000","last_fullsized":"2024-09-19T05:42:34.057870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058164+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:42:34.058164+0000","last_peered":"2024-09-19T05:42:34.058164+0000","last_clean":"2024-09-19T05:42:34.058164+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:42:34.058164+0000","last_undegraded":"2024-09-19T05:42:34.058164+0000","last_fullsized":"2024-09-19T05:42:34.058164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923659+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:42:33.923659+0000","last_peered":"2024-09-19T05:42:33.923659+0000","last_clean":"2024-09-19T05:42:33.923659+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:42:33.923659+0000","last_undegraded":"2024-09-19T05:42:33.923659+0000","last_fullsized":"2024-09-19T05:42:33.923659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059484+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:42:34.059484+0000","last_peered":"2024-09-19T05:42:34.059484+0000","last_clean":"2024-09-19T05:42:34.059484+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:42:34.059484+0000","last_undegraded":"2024-09-19T05:42:34.059484+0000","last_fullsized":"2024-09-19T05:42:34.059484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058726+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:42:34.058726+0000","last_peered":"2024-09-19T05:42:34.058726+0000","last_clean":"2024-09-19T05:42:34.058726+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:42:34.058726+0000","last_undegraded":"2024-09-19T05:42:34.058726+0000","last_fullsized":"2024-09-19T05:42:34.058726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.838616+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:42:34.838616+0000","last_peered":"2024-09-19T05:42:34.838616+0000","last_clean":"2024-09-19T05:42:34.838616+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:42:34.838616+0000","last_undegraded":"2024-09-19T05:42:34.838616+0000","last_fullsized":"2024-09-19T05:42:34.838616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925132+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:42:33.925132+0000","last_peered":"2024-09-19T05:42:33.925132+0000","last_clean":"2024-09-19T05:42:33.925132+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:42:33.925132+0000","last_undegraded":"2024-09-19T05:42:33.925132+0000","last_fullsized":"2024-09-19T05:42:33.925132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977979+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:42:37.977979+0000","last_peered":"2024-09-19T05:42:37.977979+0000","last_clean":"2024-09-19T05:42:37.977979+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:42:37.977979+0000","last_undegraded":"2024-09-19T05:42:37.977979+0000","last_fullsized":"2024-09-19T05:42:37.977979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925887+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:42:33.925887+0000","last_peered":"2024-09-19T05:42:33.925887+0000","last_clean":"2024-09-19T05:42:33.925887+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:42:33.925887+0000","last_undegraded":"2024-09-19T05:42:33.925887+0000","last_fullsized":"2024-09-19T05:42:33.925887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057349+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:42:34.057349+0000","last_peered":"2024-09-19T05:42:34.057349+0000","last_clean":"2024-09-19T05:42:34.057349+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:42:34.057349+0000","last_undegraded":"2024-09-19T05:42:34.057349+0000","last_fullsized":"2024-09-19T05:42:34.057349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059062+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:42:34.059062+0000","last_peered":"2024-09-19T05:42:34.059062+0000","last_clean":"2024-09-19T05:42:34.059062+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:42:34.059062+0000","last_undegraded":"2024-09-19T05:42:34.059062+0000","last_fullsized":"2024-09-19T05:42:34.059062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925905+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:42:33.925905+0000","last_peered":"2024-09-19T05:42:33.925905+0000","last_clean":"2024-09-19T05:42:33.925905+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:42:33.925905+0000","last_undegraded":"2024-09-19T05:42:33.925905+0000","last_fullsized":"2024-09-19T05:42:33.925905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925556+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:42:33.925556+0000","last_peered":"2024-09-19T05:42:33.925556+0000","last_clean":"2024-09-19T05:42:33.925556+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:42:33.925556+0000","last_undegraded":"2024-09-19T05:42:33.925556+0000","last_fullsized":"2024-09-19T05:42:33.925556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058603+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:42:34.058603+0000","last_peered":"2024-09-19T05:42:34.058603+0000","last_clean":"2024-09-19T05:42:34.058603+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:42:34.058603+0000","last_undegraded":"2024-09-19T05:42:34.058603+0000","last_fullsized":"2024-09-19T05:42:34.058603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925165+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:42:33.925165+0000","last_peered":"2024-09-19T05:42:33.925165+0000","last_clean":"2024-09-19T05:42:33.925165+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:42:33.925165+0000","last_undegraded":"2024-09-19T05:42:33.925165+0000","last_fullsized":"2024-09-19T05:42:33.925165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978153+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:42:37.978153+0000","last_peered":"2024-09-19T05:42:37.978153+0000","last_clean":"2024-09-19T05:42:37.978153+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:42:37.978153+0000","last_undegraded":"2024-09-19T05:42:37.978153+0000","last_fullsized":"2024-09-19T05:42:37.978153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059198+0000","last_change":"2024-09-19T05:42:32.941131+0000","last_active":"2024-09-19T05:42:34.059198+0000","last_peered":"2024-09-19T05:42:34.059198+0000","last_clean":"2024-09-19T05:42:34.059198+0000","last_became_active":"2024-09-19T05:42:32.940596+0000","last_became_peered":"2024-09-19T05:42:32.940596+0000","last_unstale":"2024-09-19T05:42:34.059198+0000","last_undegraded":"2024-09-19T05:42:34.059198+0000","last_fullsized":"2024-09-19T05:42:34.059198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:00.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059376+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:42:34.059376+0000","last_peered":"2024-09-19T05:42:34.059376+0000","last_clean":"2024-09-19T05:42:34.059376+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:42:34.059376+0000","last_undegraded":"2024-09-19T05:42:34.059376+0000","last_fullsized":"2024-09-19T05:42:34.059376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978276+0000","last_change":"2024-09-19T05:42:31.988393+0000","last_active":"2024-09-19T05:42:37.978276+0000","last_peered":"2024-09-19T05:42:37.978276+0000","last_clean":"2024-09-19T05:42:37.978276+0000","last_became_active":"2024-09-19T05:42:31.986557+0000","last_became_peered":"2024-09-19T05:42:31.986557+0000","last_unstale":"2024-09-19T05:42:37.978276+0000","last_undegraded":"2024-09-19T05:42:37.978276+0000","last_fullsized":"2024-09-19T05:42:37.978276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058853+0000","last_change":"2024-09-19T05:42:32.929140+0000","last_active":"2024-09-19T05:42:34.058853+0000","last_peered":"2024-09-19T05:42:34.058853+0000","last_clean":"2024-09-19T05:42:34.058853+0000","last_became_active":"2024-09-19T05:42:32.928811+0000","last_became_peered":"2024-09-19T05:42:32.928811+0000","last_unstale":"2024-09-19T05:42:34.058853+0000","last_undegraded":"2024-09-19T05:42:34.058853+0000","last_fullsized":"2024-09-19T05:42:34.058853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:29.306206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977607+0000","last_change":"2024-09-19T05:42:31.988083+0000","last_active":"2024-09-19T05:42:37.977607+0000","last_peered":"2024-09-19T05:42:37.977607+0000","last_clean":"2024-09-19T05:42:37.977607+0000","last_became_active":"2024-09-19T05:42:31.987088+0000","last_became_peered":"2024-09-19T05:42:31.987088+0000","last_unstale":"2024-09-19T05:42:37.977607+0000","last_undegraded":"2024-09-19T05:42:37.977607+0000","last_fullsized":"2024-09-19T05:42:37.977607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.941310+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.940997+0000","last_became_peered":"2024-09-19T05:42:32.940997+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:58.755996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832348+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:42:34.832348+0000","last_peered":"2024-09-19T05:42:34.832348+0000","last_clean":"2024-09-19T05:42:34.832348+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:42:34.832348+0000","last_undegraded":"2024-09-19T05:42:34.832348+0000","last_fullsized":"2024-09-19T05:42:34.832348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058604+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:42:34.058604+0000","last_peered":"2024-09-19T05:42:34.058604+0000","last_clean":"2024-09-19T05:42:34.058604+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:42:34.058604+0000","last_undegraded":"2024-09-19T05:42:34.058604+0000","last_fullsized":"2024-09-19T05:42:34.058604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980788+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:42:37.980788+0000","last_peered":"2024-09-19T05:42:37.980788+0000","last_clean":"2024-09-19T05:42:37.980788+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:42:37.980788+0000","last_undegraded":"2024-09-19T05:42:37.980788+0000","last_fullsized":"2024-09-19T05:42:37.980788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":80728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669576,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":81400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":79698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":294912,"data_stored":669554,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:42:47.622 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:42:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.757+0000 7f19fdad2640 1 Processor -- start 2024-09-19T05:42:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.757+0000 7f19fdad2640 1 -- start start 2024-09-19T05:42:47.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19fdad2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f80c87d0 0x7f19f80c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19fdad2640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19f806c8c0 0x7f19f80c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19fdad2640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f80c7290 con 0x7f19f814f320 2024-09-19T05:42:47.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19fdad2640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f19f80702f0 con 0x7f19f806c8c0 2024-09-19T05:42:47.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19fdad2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f19f80c6d10 con 0x7f19f80c87d0 2024-09-19T05:42:47.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f77fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f19f814f320 0x7f19f81586e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41184/0 (socket says 172.21.15.2:41184) 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f77fe640 1 -- 172.21.15.2:0/1666497586 learned_addr learned my addr 172.21.15.2:0/1666497586 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f6ffd640 1 --2- 172.21.15.2:0/1666497586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f80c87d0 0x7f19f80c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f7fff640 1 --2- 172.21.15.2:0/1666497586 >> v2:172.21.15.123:3300/0 conn(0x7f19f806c8c0 0x7f19f80c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f67fc640 1 -- 172.21.15.2:0/1666497586 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1850617901 0 0) 0x7f19e8001110 con 0x7f19f814f320 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f67fc640 1 -- 172.21.15.2:0/1666497586 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19e0003520 con 0x7f19f814f320 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f67fc640 1 -- 172.21.15.2:0/1666497586 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1150605942 0 0) 0x7f19e0003520 con 0x7f19f814f320 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f6ffd640 1 -- 172.21.15.2:0/1666497586 >> v2:172.21.15.123:3300/0 conn(0x7f19f806c8c0 msgr2=0x7f19f80c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f6ffd640 1 --2- 172.21.15.2:0/1666497586 >> v2:172.21.15.123:3300/0 conn(0x7f19f806c8c0 0x7f19f80c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f6ffd640 1 -- 172.21.15.2:0/1666497586 >> v1:172.21.15.2:6789/0 conn(0x7f19f814f320 legacy=0x7f19f81586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.758+0000 7f19f6ffd640 1 -- 172.21.15.2:0/1666497586 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f80ca1a0 con 0x7f19f80c87d0 2024-09-19T05:42:47.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.759+0000 7f19f6ffd640 1 --2- 172.21.15.2:0/1666497586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f80c87d0 0x7f19f80c5980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0097a0 tx=0x7f19ec02a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa904c8ca4e4c242 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.759+0000 7f19f67fc640 1 -- 172.21.15.2:0/1666497586 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec037070 con 0x7f19f80c87d0 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.759+0000 7f19f67fc640 1 -- 172.21.15.2:0/1666497586 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec02ab10 con 0x7f19f80c87d0 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.759+0000 7f19f67fc640 1 -- 172.21.15.2:0/1666497586 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8001840 con 0x7f19f80c87d0 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 -- 172.21.15.2:0/1666497586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f80c87d0 msgr2=0x7f19f80c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 --2- 172.21.15.2:0/1666497586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f80c87d0 0x7f19f80c5980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0097a0 tx=0x7f19ec02a080 comp rx=0 tx=0).stop 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 -- 172.21.15.2:0/1666497586 shutdown_connections 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 --2- 172.21.15.2:0/1666497586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f80c87d0 0x7f19f80c5980 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 --2- 172.21.15.2:0/1666497586 >> v2:172.21.15.123:3300/0 conn(0x7f19f806c8c0 0x7f19f80c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 -- 172.21.15.2:0/1666497586 >> 172.21.15.2:0/1666497586 conn(0x7f19f80cc270 msgr2=0x7f19f80cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 -- 172.21.15.2:0/1666497586 shutdown_connections 2024-09-19T05:42:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.760+0000 7f19fdad2640 1 -- 172.21.15.2:0/1666497586 wait complete. 2024-09-19T05:42:47.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.761+0000 7f19fdad2640 1 Processor -- start 2024-09-19T05:42:47.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.761+0000 7f19fdad2640 1 -- start start 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19fdad2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f806c8c0 0x7f19f814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19fdad2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f80c87d0 0x7f19f814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19fdad2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19f814f320 0x7f19f81eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19fdad2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f19f8152ca0 con 0x7f19f814f320 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19fdad2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19f80ca480 con 0x7f19f80c87d0 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19fdad2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f19f806fed0 con 0x7f19f806c8c0 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19f814f320 0x7f19f81eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19f7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19f814f320 0x7f19f81eed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47688/0 (socket says 172.21.15.2:47688) 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19f7fff640 1 -- 172.21.15.2:0/3497710740 learned_addr learned my addr 172.21.15.2:0/3497710740 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19f6ffd640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f80c87d0 0x7f19f814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.762+0000 7f19f77fe640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f806c8c0 0x7f19f814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19f7fff640 1 -- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f806c8c0 msgr2=0x7f19f814d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19f7fff640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f806c8c0 0x7f19f814d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19f7fff640 1 -- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f80c87d0 msgr2=0x7f19f814f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19f7fff640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f80c87d0 0x7f19f814f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19f7fff640 1 -- 172.21.15.2:0/3497710740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f8153080 con 0x7f19f814f320 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19f7fff640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19f814f320 0x7f19f81eed90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009dd0 tx=0x7f19f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19dffff640 1 -- 172.21.15.2:0/3497710740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0018020 con 0x7f19f814f320 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19dffff640 1 -- 172.21.15.2:0/3497710740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f0014480 con 0x7f19f814f320 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19dffff640 1 -- 172.21.15.2:0/3497710740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0014870 con 0x7f19f814f320 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f8153360 con 0x7f19f814f320 2024-09-19T05:42:47.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.763+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f8153920 con 0x7f19f814f320 2024-09-19T05:42:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.764+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f806c1b0 con 0x7f19f814f320 2024-09-19T05:42:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.764+0000 7f19dffff640 1 -- 172.21.15.2:0/3497710740 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f19f0005d10 con 0x7f19f814f320 2024-09-19T05:42:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.765+0000 7f19dffff640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f19e406c2b0 0x7f19e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.765+0000 7f19dffff640 1 -- 172.21.15.2:0/3497710740 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f19f008be10 con 0x7f19f814f320 2024-09-19T05:42:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.765+0000 7f19f77fe640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f19e406c2b0 0x7f19e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:47.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.766+0000 7f19f77fe640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f19e406c2b0 0x7f19e406e770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19e8005880 tx=0x7f19e8001230 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:47.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.769+0000 7f19dffff640 1 -- 172.21.15.2:0/3497710740 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f19f0022070 con 0x7f19f814f320 2024-09-19T05:42:47.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.941+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19f8156000 con 0x7f19e406c2b0 2024-09-19T05:42:47.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.947+0000 7f19dffff640 1 -- 172.21.15.2:0/3497710740 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251910 (secure 0 0 0) 0x7f19f8156000 con 0x7f19e406c2b0 2024-09-19T05:42:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:42:47.980 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:42:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f19e406c2b0 msgr2=0x7f19e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f19e406c2b0 0x7f19e406e770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19e8005880 tx=0x7f19e8001230 comp rx=0 tx=0).stop 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19f814f320 msgr2=0x7f19f81eed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19f814f320 0x7f19f81eed90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009dd0 tx=0x7f19f000c490 comp rx=0 tx=0).stop 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 shutdown_connections 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f19e406c2b0 0x7f19e406e770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f80c87d0 0x7f19f814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19f814f320 0x7f19f81eed90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 --2- 172.21.15.2:0/3497710740 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19f806c8c0 0x7f19f814d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.950+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 >> 172.21.15.2:0/3497710740 conn(0x7f19f80cc270 msgr2=0x7f19f8149030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.951+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 shutdown_connections 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:47.951+0000 7f19fdad2640 1 -- 172.21.15.2:0/3497710740 wait complete. 2024-09-19T05:42:47.982 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-19T05:42:47.823038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593034,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234796,"kb_used_data":1952,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739924,"statfs":{"total":773094113280,"available":772853682176,"internally_reserved":0,"allocated":1998848,"data_stored":1818484,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003208"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058235+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:42:34.058235+0000","last_peered":"2024-09-19T05:42:34.058235+0000","last_clean":"2024-09-19T05:42:34.058235+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:42:34.058235+0000","last_undegraded":"2024-09-19T05:42:34.058235+0000","last_fullsized":"2024-09-19T05:42:34.058235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058494+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:42:34.058494+0000","last_peered":"2024-09-19T05:42:34.058494+0000","last_clean":"2024-09-19T05:42:34.058494+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:42:34.058494+0000","last_undegraded":"2024-09-19T05:42:34.058494+0000","last_fullsized":"2024-09-19T05:42:34.058494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058017+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:42:34.058017+0000","last_peered":"2024-09-19T05:42:34.058017+0000","last_clean":"2024-09-19T05:42:34.058017+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:42:34.058017+0000","last_undegraded":"2024-09-19T05:42:34.058017+0000","last_fullsized":"2024-09-19T05:42:34.058017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923708+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:42:33.923708+0000","last_peered":"2024-09-19T05:42:33.923708+0000","last_clean":"2024-09-19T05:42:33.923708+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:42:33.923708+0000","last_undegraded":"2024-09-19T05:42:33.923708+0000","last_fullsized":"2024-09-19T05:42:33.923708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057846+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:42:34.057846+0000","last_peered":"2024-09-19T05:42:34.057846+0000","last_clean":"2024-09-19T05:42:34.057846+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:42:34.057846+0000","last_undegraded":"2024-09-19T05:42:34.057846+0000","last_fullsized":"2024-09-19T05:42:34.057846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923679+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:42:33.923679+0000","last_peered":"2024-09-19T05:42:33.923679+0000","last_clean":"2024-09-19T05:42:33.923679+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:42:33.923679+0000","last_undegraded":"2024-09-19T05:42:33.923679+0000","last_fullsized":"2024-09-19T05:42:33.923679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057764+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:42:34.057764+0000","last_peered":"2024-09-19T05:42:34.057764+0000","last_clean":"2024-09-19T05:42:34.057764+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:42:34.057764+0000","last_undegraded":"2024-09-19T05:42:34.057764+0000","last_fullsized":"2024-09-19T05:42:34.057764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058259+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:42:34.058259+0000","last_peered":"2024-09-19T05:42:34.058259+0000","last_clean":"2024-09-19T05:42:34.058259+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:42:34.058259+0000","last_undegraded":"2024-09-19T05:42:34.058259+0000","last_fullsized":"2024-09-19T05:42:34.058259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058756+0000","last_change":"2024-09-19T05:42:32.945872+0000","last_active":"2024-09-19T05:42:34.058756+0000","last_peered":"2024-09-19T05:42:34.058756+0000","last_clean":"2024-09-19T05:42:34.058756+0000","last_became_active":"2024-09-19T05:42:32.945545+0000","last_became_peered":"2024-09-19T05:42:32.945545+0000","last_unstale":"2024-09-19T05:42:34.058756+0000","last_undegraded":"2024-09-19T05:42:34.058756+0000","last_fullsized":"2024-09-19T05:42:34.058756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058073+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:42:34.058073+0000","last_peered":"2024-09-19T05:42:34.058073+0000","last_clean":"2024-09-19T05:42:34.058073+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:42:34.058073+0000","last_undegraded":"2024-09-19T05:42:34.058073+0000","last_fullsized":"2024-09-19T05:42:34.058073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058096+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:42:34.058096+0000","last_peered":"2024-09-19T05:42:34.058096+0000","last_clean":"2024-09-19T05:42:34.058096+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:42:34.058096+0000","last_undegraded":"2024-09-19T05:42:34.058096+0000","last_fullsized":"2024-09-19T05:42:34.058096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058167+0000","last_change":"2024-09-19T05:42:32.941601+0000","last_active":"2024-09-19T05:42:34.058167+0000","last_peered":"2024-09-19T05:42:34.058167+0000","last_clean":"2024-09-19T05:42:34.058167+0000","last_became_active":"2024-09-19T05:42:32.941288+0000","last_became_peered":"2024-09-19T05:42:32.941288+0000","last_unstale":"2024-09-19T05:42:34.058167+0000","last_undegraded":"2024-09-19T05:42:34.058167+0000","last_fullsized":"2024-09-19T05:42:34.058167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:35.670719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058583+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:42:34.058583+0000","last_peered":"2024-09-19T05:42:34.058583+0000","last_clean":"2024-09-19T05:42:34.058583+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:42:34.058583+0000","last_undegraded":"2024-09-19T05:42:34.058583+0000","last_fullsized":"2024-09-19T05:42:34.058583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058482+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:42:34.058482+0000","last_peered":"2024-09-19T05:42:34.058482+0000","last_clean":"2024-09-19T05:42:34.058482+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:42:34.058482+0000","last_undegraded":"2024-09-19T05:42:34.058482+0000","last_fullsized":"2024-09-19T05:42:34.058482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058261+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:42:34.058261+0000","last_peered":"2024-09-19T05:42:34.058261+0000","last_clean":"2024-09-19T05:42:34.058261+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:42:34.058261+0000","last_undegraded":"2024-09-19T05:42:34.058261+0000","last_fullsized":"2024-09-19T05:42:34.058261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057516+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:42:34.057516+0000","last_peered":"2024-09-19T05:42:34.057516+0000","last_clean":"2024-09-19T05:42:34.057516+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:42:34.057516+0000","last_undegraded":"2024-09-19T05:42:34.057516+0000","last_fullsized":"2024-09-19T05:42:34.057516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059660+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:42:34.059660+0000","last_peered":"2024-09-19T05:42:34.059660+0000","last_clean":"2024-09-19T05:42:34.059660+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:42:34.059660+0000","last_undegraded":"2024-09-19T05:42:34.059660+0000","last_fullsized":"2024-09-19T05:42:34.059660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925327+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:42:33.925327+0000","last_peered":"2024-09-19T05:42:33.925327+0000","last_clean":"2024-09-19T05:42:33.925327+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:42:33.925327+0000","last_undegraded":"2024-09-19T05:42:33.925327+0000","last_fullsized":"2024-09-19T05:42:33.925327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059534+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:42:34.059534+0000","last_peered":"2024-09-19T05:42:34.059534+0000","last_clean":"2024-09-19T05:42:34.059534+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:42:34.059534+0000","last_undegraded":"2024-09-19T05:42:34.059534+0000","last_fullsized":"2024-09-19T05:42:34.059534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058089+0000","last_change":"2024-09-19T05:42:32.940627+0000","last_active":"2024-09-19T05:42:34.058089+0000","last_peered":"2024-09-19T05:42:34.058089+0000","last_clean":"2024-09-19T05:42:34.058089+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058089+0000","last_undegraded":"2024-09-19T05:42:34.058089+0000","last_fullsized":"2024-09-19T05:42:34.058089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:53.616840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925755+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:42:33.925755+0000","last_peered":"2024-09-19T05:42:33.925755+0000","last_clean":"2024-09-19T05:42:33.925755+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:42:33.925755+0000","last_undegraded":"2024-09-19T05:42:33.925755+0000","last_fullsized":"2024-09-19T05:42:33.925755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058013+0000","last_change":"2024-09-19T05:42:32.940425+0000","last_active":"2024-09-19T05:42:34.058013+0000","last_peered":"2024-09-19T05:42:34.058013+0000","last_clean":"2024-09-19T05:42:34.058013+0000","last_became_active":"2024-09-19T05:42:32.940059+0000","last_became_peered":"2024-09-19T05:42:32.940059+0000","last_unstale":"2024-09-19T05:42:34.058013+0000","last_undegraded":"2024-09-19T05:42:34.058013+0000","last_fullsized":"2024-09-19T05:42:34.058013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058206+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:42:34.058206+0000","last_peered":"2024-09-19T05:42:34.058206+0000","last_clean":"2024-09-19T05:42:34.058206+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:42:34.058206+0000","last_undegraded":"2024-09-19T05:42:34.058206+0000","last_fullsized":"2024-09-19T05:42:34.058206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057512+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:42:34.057512+0000","last_peered":"2024-09-19T05:42:34.057512+0000","last_clean":"2024-09-19T05:42:34.057512+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:42:34.057512+0000","last_undegraded":"2024-09-19T05:42:34.057512+0000","last_fullsized":"2024-09-19T05:42:34.057512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058955+0000","last_change":"2024-09-19T05:42:32.941414+0000","last_active":"2024-09-19T05:42:34.058955+0000","last_peered":"2024-09-19T05:42:34.058955+0000","last_clean":"2024-09-19T05:42:34.058955+0000","last_became_active":"2024-09-19T05:42:32.941076+0000","last_became_peered":"2024-09-19T05:42:32.941076+0000","last_unstale":"2024-09-19T05:42:34.058955+0000","last_undegraded":"2024-09-19T05:42:34.058955+0000","last_fullsized":"2024-09-19T05:42:34.058955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:13.371046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923839+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:42:33.923839+0000","last_peered":"2024-09-19T05:42:33.923839+0000","last_clean":"2024-09-19T05:42:33.923839+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:42:33.923839+0000","last_undegraded":"2024-09-19T05:42:33.923839+0000","last_fullsized":"2024-09-19T05:42:33.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058867+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:42:34.058867+0000","last_peered":"2024-09-19T05:42:34.058867+0000","last_clean":"2024-09-19T05:42:34.058867+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:42:34.058867+0000","last_undegraded":"2024-09-19T05:42:34.058867+0000","last_fullsized":"2024-09-19T05:42:34.058867+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058377+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:42:34.058377+0000","last_peered":"2024-09-19T05:42:34.058377+0000","last_clean":"2024-09-19T05:42:34.058377+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:42:34.058377+0000","last_undegraded":"2024-09-19T05:42:34.058377+0000","last_fullsized":"2024-09-19T05:42:34.058377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059114+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:42:34.059114+0000","last_peered":"2024-09-19T05:42:34.059114+0000","last_clean":"2024-09-19T05:42:34.059114+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:42:34.059114+0000","last_undegraded":"2024-09-19T05:42:34.059114+0000","last_fullsized":"2024-09-19T05:42:34.059114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924874+0000","last_change":"2024-09-19T05:42:32.927184+0000","last_active":"2024-09-19T05:42:33.924874+0000","last_peered":"2024-09-19T05:42:33.924874+0000","last_clean":"2024-09-19T05:42:33.924874+0000","last_became_active":"2024-09-19T05:42:32.926952+0000","last_became_peered":"2024-09-19T05:42:32.926952+0000","last_unstale":"2024-09-19T05:42:33.924874+0000","last_undegraded":"2024-09-19T05:42:33.924874+0000","last_fullsized":"2024-09-19T05:42:33.924874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:36.017999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923475+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:42:33.923475+0000","last_peered":"2024-09-19T05:42:33.923475+0000","last_clean":"2024-09-19T05:42:33.923475+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:42:33.923475+0000","last_undegraded":"2024-09-19T05:42:33.923475+0000","last_fullsized":"2024-09-19T05:42:33.923475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058161+0000","last_change":"2024-09-19T05:42:32.942584+0000","last_active":"2024-09-19T05:42:34.058161+0000","last_peered":"2024-09-19T05:42:34.058161+0000","last_clean":"2024-09-19T05:42:34.058161+0000","last_became_active":"2024-09-19T05:42:32.942274+0000","last_became_peered":"2024-09-19T05:42:32.942274+0000","last_unstale":"2024-09-19T05:42:34.058161+0000","last_undegraded":"2024-09-19T05:42:34.058161+0000","last_fullsized":"2024-09-19T05:42:34.058161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832292+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:42:34.832292+0000","last_peered":"2024-09-19T05:42:34.832292+0000","last_clean":"2024-09-19T05:42:34.832292+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:42:34.832292+0000","last_undegraded":"2024-09-19T05:42:34.832292+0000","last_fullsized":"2024-09-19T05:42:34.832292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925748+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:42:33.925748+0000","last_peered":"2024-09-19T05:42:33.925748+0000","last_clean":"2024-09-19T05:42:33.925748+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:42:33.925748+0000","last_undegraded":"2024-09-19T05:42:33.925748+0000","last_fullsized":"2024-09-19T05:42:33.925748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.839402+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:42:34.839402+0000","last_peered":"2024-09-19T05:42:34.839402+0000","last_clean":"2024-09-19T05:42:34.839402+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:42:34.839402+0000","last_undegraded":"2024-09-19T05:42:34.839402+0000","last_fullsized":"2024-09-19T05:42:34.839402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058211+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:42:34.058211+0000","last_peered":"2024-09-19T05:42:34.058211+0000","last_clean":"2024-09-19T05:42:34.058211+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:42:34.058211+0000","last_undegraded":"2024-09-19T05:42:34.058211+0000","last_fullsized":"2024-09-19T05:42:34.058211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058257+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:42:34.058257+0000","last_peered":"2024-09-19T05:42:34.058257+0000","last_clean":"2024-09-19T05:42:34.058257+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:42:34.058257+0000","last_undegraded":"2024-09-19T05:42:34.058257+0000","last_fullsized":"2024-09-19T05:42:34.058257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057643+0000","last_change":"2024-09-19T05:42:31.995072+0000","last_active":"2024-09-19T05:42:34.057643+0000","last_peered":"2024-09-19T05:42:34.057643+0000","last_clean":"2024-09-19T05:42:34.057643+0000","last_became_active":"2024-09-19T05:42:31.994586+0000","last_became_peered":"2024-09-19T05:42:31.994586+0000","last_unstale":"2024-09-19T05:42:34.057643+0000","last_undegraded":"2024-09-19T05:42:34.057643+0000","last_fullsized":"2024-09-19T05:42:34.057643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925971+0000","last_change":"2024-09-19T05:42:32.928695+0000","last_active":"2024-09-19T05:42:33.925971+0000","last_peered":"2024-09-19T05:42:33.925971+0000","last_clean":"2024-09-19T05:42:33.925971+0000","last_became_active":"2024-09-19T05:42:32.928437+0000","last_became_peered":"2024-09-19T05:42:32.928437+0000","last_unstale":"2024-09-19T05:42:33.925971+0000","last_undegraded":"2024-09-19T05:42:33.925971+0000","last_fullsized":"2024-09-19T05:42:33.925971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058846+0000","last_change":"2024-09-19T05:42:31.985393+0000","last_active":"2024-09-19T05:42:34.058846+0000","last_peered":"2024-09-19T05:42:34.058846+0000","last_clean":"2024-09-19T05:42:34.058846+0000","last_became_active":"2024-09-19T05:42:31.985034+0000","last_became_peered":"2024-09-19T05:42:31.985034+0000","last_unstale":"2024-09-19T05:42:34.058846+0000","last_undegraded":"2024-09-19T05:42:34.058846+0000","last_fullsized":"2024-09-19T05:42:34.058846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:11.070227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923861+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:42:33.923861+0000","last_peered":"2024-09-19T05:42:33.923861+0000","last_clean":"2024-09-19T05:42:33.923861+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:42:33.923861+0000","last_undegraded":"2024-09-19T05:42:33.923861+0000","last_fullsized":"2024-09-19T05:42:33.923861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058193+0000","last_change":"2024-09-19T05:42:31.987634+0000","last_active":"2024-09-19T05:42:34.058193+0000","last_peered":"2024-09-19T05:42:34.058193+0000","last_clean":"2024-09-19T05:42:34.058193+0000","last_became_active":"2024-09-19T05:42:31.986452+0000","last_became_peered":"2024-09-19T05:42:31.986452+0000","last_unstale":"2024-09-19T05:42:34.058193+0000","last_undegraded":"2024-09-19T05:42:34.058193+0000","last_fullsized":"2024-09-19T05:42:34.058193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.981030+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:42:37.981030+0000","last_peered":"2024-09-19T05:42:37.981030+0000","last_clean":"2024-09-19T05:42:37.981030+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:42:37.981030+0000","last_undegraded":"2024-09-19T05:42:37.981030+0000","last_fullsized":"2024-09-19T05:42:37.981030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059270+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:42:34.059270+0000","last_peered":"2024-09-19T05:42:34.059270+0000","last_clean":"2024-09-19T05:42:34.059270+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:42:34.059270+0000","last_undegraded":"2024-09-19T05:42:34.059270+0000","last_fullsized":"2024-09-19T05:42:34.059270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.988258+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:42:37.988258+0000","last_peered":"2024-09-19T05:42:37.988258+0000","last_clean":"2024-09-19T05:42:37.988258+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:42:37.988258+0000","last_undegraded":"2024-09-19T05:42:37.988258+0000","last_fullsized":"2024-09-19T05:42:37.988258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1868,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058283+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:42:34.058283+0000","last_peered":"2024-09-19T05:42:34.058283+0000","last_clean":"2024-09-19T05:42:34.058283+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:42:34.058283+0000","last_undegraded":"2024-09-19T05:42:34.058283+0000","last_fullsized":"2024-09-19T05:42:34.058283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978898+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:42:37.978898+0000","last_peered":"2024-09-19T05:42:37.978898+0000","last_clean":"2024-09-19T05:42:37.978898+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:42:37.978898+0000","last_undegraded":"2024-09-19T05:42:37.978898+0000","last_fullsized":"2024-09-19T05:42:37.978898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058589+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:42:34.058589+0000","last_peered":"2024-09-19T05:42:34.058589+0000","last_clean":"2024-09-19T05:42:34.058589+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:42:34.058589+0000","last_undegraded":"2024-09-19T05:42:34.058589+0000","last_fullsized":"2024-09-19T05:42:34.058589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059325+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:42:34.059325+0000","last_peered":"2024-09-19T05:42:34.059325+0000","last_clean":"2024-09-19T05:42:34.059325+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:42:34.059325+0000","last_undegraded":"2024-09-19T05:42:34.059325+0000","last_fullsized":"2024-09-19T05:42:34.059325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059177+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:42:34.059177+0000","last_peered":"2024-09-19T05:42:34.059177+0000","last_clean":"2024-09-19T05:42:34.059177+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:42:34.059177+0000","last_undegraded":"2024-09-19T05:42:34.059177+0000","last_fullsized":"2024-09-19T05:42:34.059177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057760+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:42:34.057760+0000","last_peered":"2024-09-19T05:42:34.057760+0000","last_clean":"2024-09-19T05:42:34.057760+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:42:34.057760+0000","last_undegraded":"2024-09-19T05:42:34.057760+0000","last_fullsized":"2024-09-19T05:42:34.057760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058006+0000","last_change":"2024-09-19T05:42:32.938549+0000","last_active":"2024-09-19T05:42:34.058006+0000","last_peered":"2024-09-19T05:42:34.058006+0000","last_clean":"2024-09-19T05:42:34.058006+0000","last_became_active":"2024-09-19T05:42:32.938113+0000","last_became_peered":"2024-09-19T05:42:32.938113+0000","last_unstale":"2024-09-19T05:42:34.058006+0000","last_undegraded":"2024-09-19T05:42:34.058006+0000","last_fullsized":"2024-09-19T05:42:34.058006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:15.787698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.985533+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:42:37.985533+0000","last_peered":"2024-09-19T05:42:37.985533+0000","last_clean":"2024-09-19T05:42:37.985533+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:42:37.985533+0000","last_undegraded":"2024-09-19T05:42:37.985533+0000","last_fullsized":"2024-09-19T05:42:37.985533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058990+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:42:34.058990+0000","last_peered":"2024-09-19T05:42:34.058990+0000","last_clean":"2024-09-19T05:42:34.058990+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:42:34.058990+0000","last_undegraded":"2024-09-19T05:42:34.058990+0000","last_fullsized":"2024-09-19T05:42:34.058990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.837827+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:42:34.837827+0000","last_peered":"2024-09-19T05:42:34.837827+0000","last_clean":"2024-09-19T05:42:34.837827+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:42:34.837827+0000","last_undegraded":"2024-09-19T05:42:34.837827+0000","last_fullsized":"2024-09-19T05:42:34.837827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923969+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:42:33.923969+0000","last_peered":"2024-09-19T05:42:33.923969+0000","last_clean":"2024-09-19T05:42:33.923969+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:42:33.923969+0000","last_undegraded":"2024-09-19T05:42:33.923969+0000","last_fullsized":"2024-09-19T05:42:33.923969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980561+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:42:37.980561+0000","last_peered":"2024-09-19T05:42:37.980561+0000","last_clean":"2024-09-19T05:42:37.980561+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:42:37.980561+0000","last_undegraded":"2024-09-19T05:42:37.980561+0000","last_fullsized":"2024-09-19T05:42:37.980561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058299+0000","last_change":"2024-09-19T05:42:32.945685+0000","last_active":"2024-09-19T05:42:34.058299+0000","last_peered":"2024-09-19T05:42:34.058299+0000","last_clean":"2024-09-19T05:42:34.058299+0000","last_became_active":"2024-09-19T05:42:32.945337+0000","last_became_peered":"2024-09-19T05:42:32.945337+0000","last_unstale":"2024-09-19T05:42:34.058299+0000","last_undegraded":"2024-09-19T05:42:34.058299+0000","last_fullsized":"2024-09-19T05:42:34.058299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832321+0000","last_change":"2024-09-19T05:42:31.991430+0000","last_active":"2024-09-19T05:42:34.832321+0000","last_peered":"2024-09-19T05:42:34.832321+0000","last_clean":"2024-09-19T05:42:34.832321+0000","last_became_active":"2024-09-19T05:42:31.991057+0000","last_became_peered":"2024-09-19T05:42:31.991057+0000","last_unstale":"2024-09-19T05:42:34.832321+0000","last_undegraded":"2024-09-19T05:42:34.832321+0000","last_fullsized":"2024-09-19T05:42:34.832321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:58.247495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058366+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:42:34.058366+0000","last_peered":"2024-09-19T05:42:34.058366+0000","last_clean":"2024-09-19T05:42:34.058366+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:42:34.058366+0000","last_undegraded":"2024-09-19T05:42:34.058366+0000","last_fullsized":"2024-09-19T05:42:34.058366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058851+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:42:34.058851+0000","last_peered":"2024-09-19T05:42:34.058851+0000","last_clean":"2024-09-19T05:42:34.058851+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:42:34.058851+0000","last_undegraded":"2024-09-19T05:42:34.058851+0000","last_fullsized":"2024-09-19T05:42:34.058851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059399+0000","last_change":"2024-09-19T05:42:32.941889+0000","last_active":"2024-09-19T05:42:34.059399+0000","last_peered":"2024-09-19T05:42:34.059399+0000","last_clean":"2024-09-19T05:42:34.059399+0000","last_became_active":"2024-09-19T05:42:32.941254+0000","last_became_peered":"2024-09-19T05:42:32.941254+0000","last_unstale":"2024-09-19T05:42:34.059399+0000","last_undegraded":"2024-09-19T05:42:34.059399+0000","last_fullsized":"2024-09-19T05:42:34.059399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:44.771973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057837+0000","last_change":"2024-09-19T05:42:23.458936+0000","last_active":"2024-09-19T05:42:34.057837+0000","last_peered":"2024-09-19T05:42:34.057837+0000","last_clean":"2024-09-19T05:42:34.057837+0000","last_became_active":"2024-09-19T05:42:23.457990+0000","last_became_peered":"2024-09-19T05:42:23.457990+0000","last_unstale":"2024-09-19T05:42:34.057837+0000","last_undegraded":"2024-09-19T05:42:34.057837+0000","last_fullsized":"2024-09-19T05:42:34.057837+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:45.259241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059347+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:42:34.059347+0000","last_peered":"2024-09-19T05:42:34.059347+0000","last_clean":"2024-09-19T05:42:34.059347+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:42:34.059347+0000","last_undegraded":"2024-09-19T05:42:34.059347+0000","last_fullsized":"2024-09-19T05:42:34.059347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925215+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:42:33.925215+0000","last_peered":"2024-09-19T05:42:33.925215+0000","last_clean":"2024-09-19T05:42:33.925215+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:42:33.925215+0000","last_undegraded":"2024-09-19T05:42:33.925215+0000","last_fullsized":"2024-09-19T05:42:33.925215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057735+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:42:34.057735+0000","last_peered":"2024-09-19T05:42:34.057735+0000","last_clean":"2024-09-19T05:42:34.057735+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:42:34.057735+0000","last_undegraded":"2024-09-19T05:42:34.057735+0000","last_fullsized":"2024-09-19T05:42:34.057735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058126+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:42:34.058126+0000","last_peered":"2024-09-19T05:42:34.058126+0000","last_clean":"2024-09-19T05:42:34.058126+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:42:34.058126+0000","last_undegraded":"2024-09-19T05:42:34.058126+0000","last_fullsized":"2024-09-19T05:42:34.058126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.924861+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:42:33.924861+0000","last_peered":"2024-09-19T05:42:33.924861+0000","last_clean":"2024-09-19T05:42:33.924861+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:42:33.924861+0000","last_undegraded":"2024-09-19T05:42:33.924861+0000","last_fullsized":"2024-09-19T05:42:33.924861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057949+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:42:34.057949+0000","last_peered":"2024-09-19T05:42:34.057949+0000","last_clean":"2024-09-19T05:42:34.057949+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:42:34.057949+0000","last_undegraded":"2024-09-19T05:42:34.057949+0000","last_fullsized":"2024-09-19T05:42:34.057949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059345+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:42:34.059345+0000","last_peered":"2024-09-19T05:42:34.059345+0000","last_clean":"2024-09-19T05:42:34.059345+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:42:34.059345+0000","last_undegraded":"2024-09-19T05:42:34.059345+0000","last_fullsized":"2024-09-19T05:42:34.059345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057870+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:42:34.057870+0000","last_peered":"2024-09-19T05:42:34.057870+0000","last_clean":"2024-09-19T05:42:34.057870+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:42:34.057870+0000","last_undegraded":"2024-09-19T05:42:34.057870+0000","last_fullsized":"2024-09-19T05:42:34.057870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058164+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:42:34.058164+0000","last_peered":"2024-09-19T05:42:34.058164+0000","last_clean":"2024-09-19T05:42:34.058164+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:42:34.058164+0000","last_undegraded":"2024-09-19T05:42:34.058164+0000","last_fullsized":"2024-09-19T05:42:34.058164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.923659+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:42:33.923659+0000","last_peered":"2024-09-19T05:42:33.923659+0000","last_clean":"2024-09-19T05:42:33.923659+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:42:33.923659+0000","last_undegraded":"2024-09-19T05:42:33.923659+0000","last_fullsized":"2024-09-19T05:42:33.923659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059484+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:42:34.059484+0000","last_peered":"2024-09-19T05:42:34.059484+0000","last_clean":"2024-09-19T05:42:34.059484+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:42:34.059484+0000","last_undegraded":"2024-09-19T05:42:34.059484+0000","last_fullsized":"2024-09-19T05:42:34.059484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058726+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:42:34.058726+0000","last_peered":"2024-09-19T05:42:34.058726+0000","last_clean":"2024-09-19T05:42:34.058726+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:42:34.058726+0000","last_undegraded":"2024-09-19T05:42:34.058726+0000","last_fullsized":"2024-09-19T05:42:34.058726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.838616+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:42:34.838616+0000","last_peered":"2024-09-19T05:42:34.838616+0000","last_clean":"2024-09-19T05:42:34.838616+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:42:34.838616+0000","last_undegraded":"2024-09-19T05:42:34.838616+0000","last_fullsized":"2024-09-19T05:42:34.838616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925132+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:42:33.925132+0000","last_peered":"2024-09-19T05:42:33.925132+0000","last_clean":"2024-09-19T05:42:33.925132+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:42:33.925132+0000","last_undegraded":"2024-09-19T05:42:33.925132+0000","last_fullsized":"2024-09-19T05:42:33.925132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977979+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:42:37.977979+0000","last_peered":"2024-09-19T05:42:37.977979+0000","last_clean":"2024-09-19T05:42:37.977979+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:42:37.977979+0000","last_undegraded":"2024-09-19T05:42:37.977979+0000","last_fullsized":"2024-09-19T05:42:37.977979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925887+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:42:33.925887+0000","last_peered":"2024-09-19T05:42:33.925887+0000","last_clean":"2024-09-19T05:42:33.925887+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:42:33.925887+0000","last_undegraded":"2024-09-19T05:42:33.925887+0000","last_fullsized":"2024-09-19T05:42:33.925887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.057349+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:42:34.057349+0000","last_peered":"2024-09-19T05:42:34.057349+0000","last_clean":"2024-09-19T05:42:34.057349+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:42:34.057349+0000","last_undegraded":"2024-09-19T05:42:34.057349+0000","last_fullsized":"2024-09-19T05:42:34.057349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059062+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:42:34.059062+0000","last_peered":"2024-09-19T05:42:34.059062+0000","last_clean":"2024-09-19T05:42:34.059062+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:42:34.059062+0000","last_undegraded":"2024-09-19T05:42:34.059062+0000","last_fullsized":"2024-09-19T05:42:34.059062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925905+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:42:33.925905+0000","last_peered":"2024-09-19T05:42:33.925905+0000","last_clean":"2024-09-19T05:42:33.925905+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:42:33.925905+0000","last_undegraded":"2024-09-19T05:42:33.925905+0000","last_fullsized":"2024-09-19T05:42:33.925905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925556+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:42:33.925556+0000","last_peered":"2024-09-19T05:42:33.925556+0000","last_clean":"2024-09-19T05:42:33.925556+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:42:33.925556+0000","last_undegraded":"2024-09-19T05:42:33.925556+0000","last_fullsized":"2024-09-19T05:42:33.925556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058603+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:42:34.058603+0000","last_peered":"2024-09-19T05:42:34.058603+0000","last_clean":"2024-09-19T05:42:34.058603+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:42:34.058603+0000","last_undegraded":"2024-09-19T05:42:34.058603+0000","last_fullsized":"2024-09-19T05:42:34.058603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:33.925165+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:42:33.925165+0000","last_peered":"2024-09-19T05:42:33.925165+0000","last_clean":"2024-09-19T05:42:33.925165+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:42:33.925165+0000","last_undegraded":"2024-09-19T05:42:33.925165+0000","last_fullsized":"2024-09-19T05:42:33.925165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978153+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:42:37.978153+0000","last_peered":"2024-09-19T05:42:37.978153+0000","last_clean":"2024-09-19T05:42:37.978153+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:42:37.978153+0000","last_undegraded":"2024-09-19T05:42:37.978153+0000","last_fullsized":"2024-09-19T05:42:37.978153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059198+0000","last_change":"2024-09-19T05:42:32.941131+0000","last_active":"2024-09-19T05:42:34.059198+0000","last_peered":"2024-09-19T05:42:34.059198+0000","last_clean":"2024-09-19T05:42:34.059198+0000","last_became_active":"2024-09-19T05:42:32.940596+0000","last_became_peered":"2024-09-19T05:42:32.940596+0000","last_unstale":"2024-09-19T05:42:34.059198+0000","last_undegraded":"2024-09-19T05:42:34.059198+0000","last_fullsized":"2024-09-19T05:42:34.059198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:00.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.059376+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:42:34.059376+0000","last_peered":"2024-09-19T05:42:34.059376+0000","last_clean":"2024-09-19T05:42:34.059376+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:42:34.059376+0000","last_undegraded":"2024-09-19T05:42:34.059376+0000","last_fullsized":"2024-09-19T05:42:34.059376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.978276+0000","last_change":"2024-09-19T05:42:31.988393+0000","last_active":"2024-09-19T05:42:37.978276+0000","last_peered":"2024-09-19T05:42:37.978276+0000","last_clean":"2024-09-19T05:42:37.978276+0000","last_became_active":"2024-09-19T05:42:31.986557+0000","last_became_peered":"2024-09-19T05:42:31.986557+0000","last_unstale":"2024-09-19T05:42:37.978276+0000","last_undegraded":"2024-09-19T05:42:37.978276+0000","last_fullsized":"2024-09-19T05:42:37.978276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058853+0000","last_change":"2024-09-19T05:42:32.929140+0000","last_active":"2024-09-19T05:42:34.058853+0000","last_peered":"2024-09-19T05:42:34.058853+0000","last_clean":"2024-09-19T05:42:34.058853+0000","last_became_active":"2024-09-19T05:42:32.928811+0000","last_became_peered":"2024-09-19T05:42:32.928811+0000","last_unstale":"2024-09-19T05:42:34.058853+0000","last_undegraded":"2024-09-19T05:42:34.058853+0000","last_fullsized":"2024-09-19T05:42:34.058853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:29.306206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.977607+0000","last_change":"2024-09-19T05:42:31.988083+0000","last_active":"2024-09-19T05:42:37.977607+0000","last_peered":"2024-09-19T05:42:37.977607+0000","last_clean":"2024-09-19T05:42:37.977607+0000","last_became_active":"2024-09-19T05:42:31.987088+0000","last_became_peered":"2024-09-19T05:42:31.987088+0000","last_unstale":"2024-09-19T05:42:37.977607+0000","last_undegraded":"2024-09-19T05:42:37.977607+0000","last_fullsized":"2024-09-19T05:42:37.977607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058885+0000","last_change":"2024-09-19T05:42:32.941310+0000","last_active":"2024-09-19T05:42:34.058885+0000","last_peered":"2024-09-19T05:42:34.058885+0000","last_clean":"2024-09-19T05:42:34.058885+0000","last_became_active":"2024-09-19T05:42:32.940997+0000","last_became_peered":"2024-09-19T05:42:32.940997+0000","last_unstale":"2024-09-19T05:42:34.058885+0000","last_undegraded":"2024-09-19T05:42:34.058885+0000","last_fullsized":"2024-09-19T05:42:34.058885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:19:58.755996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.832348+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:42:34.832348+0000","last_peered":"2024-09-19T05:42:34.832348+0000","last_clean":"2024-09-19T05:42:34.832348+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:42:34.832348+0000","last_undegraded":"2024-09-19T05:42:34.832348+0000","last_fullsized":"2024-09-19T05:42:34.832348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:34.058604+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:42:34.058604+0000","last_peered":"2024-09-19T05:42:34.058604+0000","last_clean":"2024-09-19T05:42:34.058604+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:42:34.058604+0000","last_undegraded":"2024-09-19T05:42:34.058604+0000","last_fullsized":"2024-09-19T05:42:34.058604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-19T05:42:37.980788+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:42:37.980788+0000","last_peered":"2024-09-19T05:42:37.980788+0000","last_clean":"2024-09-19T05:42:37.980788+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:42:37.980788+0000","last_undegraded":"2024-09-19T05:42:37.980788+0000","last_fullsized":"2024-09-19T05:42:37.980788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2666,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65150,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":80728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":79116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669576,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":81400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":229376,"data_stored":79698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":294912,"data_stored":669554,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:42:47.986 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T05:42:47.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-19T05:42:48.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.087+0000 7f5c9bd6d640 1 Processor -- start 2024-09-19T05:42:48.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.088+0000 7f5c9bd6d640 1 -- start start 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.088+0000 7f5c9bd6d640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c9415df60 0x7f5c9415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.088+0000 7f5c9bd6d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c9415b700 0x7f5c94071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.088+0000 7f5c9bd6d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c940cb9f0 con 0x7f5c94154d90 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.088+0000 7f5c9bd6d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f5c94071060 con 0x7f5c9415b700 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.088+0000 7f5c9bd6d640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9415e480 con 0x7f5c9415df60 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.088+0000 7f5c9a30a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c9415b700 0x7f5c94071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.089+0000 7f5c9a30a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c9415b700 0x7f5c94071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47698/0 (socket says 172.21.15.2:47698) 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.089+0000 7f5c9a30a640 1 -- 172.21.15.2:0/331429600 learned_addr learned my addr 172.21.15.2:0/331429600 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:48.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.089+0000 7f5c99308640 1 --2- 172.21.15.2:0/331429600 >> v2:172.21.15.123:3300/0 conn(0x7f5c9415df60 0x7f5c9415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.089+0000 7f5c9a30a640 1 -- 172.21.15.2:0/331429600 >> v2:172.21.15.123:3300/0 conn(0x7f5c9415df60 msgr2=0x7f5c9415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.089+0000 7f5c9a30a640 1 --2- 172.21.15.2:0/331429600 >> v2:172.21.15.123:3300/0 conn(0x7f5c9415df60 0x7f5c9415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.089+0000 7f5c9a30a640 1 -- 172.21.15.2:0/331429600 >> v1:172.21.15.123:6789/0 conn(0x7f5c94154d90 legacy=0x7f5c940cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.089+0000 7f5c9a30a640 1 -- 172.21.15.2:0/331429600 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9415e760 con 0x7f5c9415b700 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9a30a640 1 --2- 172.21.15.2:0/331429600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c94071630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5c9406c6e0 tx=0x7f5c8c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1eaf0b8071b770f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c98b07640 1 -- 172.21.15.2:0/331429600 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c036070 con 0x7f5c9415b700 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c98b07640 1 -- 172.21.15.2:0/331429600 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c031600 con 0x7f5c9415b700 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c98b07640 1 -- 172.21.15.2:0/331429600 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0319f0 con 0x7f5c9415b700 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/331429600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 msgr2=0x7f5c94071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/331429600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c94071630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5c9406c6e0 tx=0x7f5c8c02a7c0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/331429600 shutdown_connections 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/331429600 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c94071630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/331429600 >> v2:172.21.15.123:3300/0 conn(0x7f5c9415df60 0x7f5c9415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/331429600 >> 172.21.15.2:0/331429600 conn(0x7f5c9406bc10 msgr2=0x7f5c9406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:48.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/331429600 shutdown_connections 2024-09-19T05:42:48.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.090+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/331429600 wait complete. 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 Processor -- start 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 -- start start 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c94154d90 0x7f5c941f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c9415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c9415df60 0x7f5c940cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9415aca0 con 0x7f5c9415b700 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9415ed20 con 0x7f5c94154d90 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c9bd6d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5c940c9400 con 0x7f5c9415df60 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c99308640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c9415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c99308640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c9415cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47708/0 (socket says 172.21.15.2:47708) 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c99308640 1 -- 172.21.15.2:0/2537811020 learned_addr learned my addr 172.21.15.2:0/2537811020 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.092+0000 7f5c99b09640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c94154d90 0x7f5c941f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c9a30a640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c9415df60 0x7f5c940cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c99308640 1 -- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c9415df60 msgr2=0x7f5c940cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c99308640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c9415df60 0x7f5c940cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c99308640 1 -- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c94154d90 msgr2=0x7f5c941f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c99308640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c94154d90 0x7f5c941f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c99308640 1 -- 172.21.15.2:0/2537811020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c940cd8c0 con 0x7f5c9415b700 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c99308640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c9415cdf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d620 tx=0x7f5c8400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c8a7fc640 1 -- 172.21.15.2:0/2537811020 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840125f0 con 0x7f5c9415b700 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c8a7fc640 1 -- 172.21.15.2:0/2537811020 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c84019070 con 0x7f5c9415b700 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c940cdba0 con 0x7f5c9415b700 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c8a7fc640 1 -- 172.21.15.2:0/2537811020 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840129d0 con 0x7f5c9415b700 2024-09-19T05:42:48.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.093+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c941f4240 con 0x7f5c9415b700 2024-09-19T05:42:48.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.094+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c940c4660 con 0x7f5c9415b700 2024-09-19T05:42:48.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.095+0000 7f5c8a7fc640 1 -- 172.21.15.2:0/2537811020 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5c84012b70 con 0x7f5c9415b700 2024-09-19T05:42:48.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.095+0000 7f5c8a7fc640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c7806c2e0 0x7f5c7806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.095+0000 7f5c8a7fc640 1 -- 172.21.15.2:0/2537811020 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5c8408b620 con 0x7f5c9415b700 2024-09-19T05:42:48.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.095+0000 7f5c99b09640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c7806c2e0 0x7f5c7806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.096+0000 7f5c99b09640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c7806c2e0 0x7f5c7806e7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c90001780 tx=0x7f5c90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.099+0000 7f5c8a7fc640 1 -- 172.21.15.2:0/2537811020 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5c940c4660 con 0x7f5c9415b700 2024-09-19T05:42:48.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.276+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f5c9415a970 con 0x7f5c9415b700 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.277+0000 7f5c8a7fc640 1 -- 172.21.15.2:0/2537811020 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v22) ==== 56+0+3227 (secure 0 0 0) 0x7f5c84059740 con 0x7f5c9415b700 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 22 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 896e162c-4658-4743-b70e-7ec2b5bce950 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-19T05:42:09.734568+0000 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-09-19T05:42:33.917577+0000 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 6 2024-09-19T05:42:48.278 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client jewel 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release squid 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:pool 3 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 8 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] [v2:172.21.15.2:6810/288510134,v1:172.21.15.2:6811/288510134] exists,up 665c2ac5-a964-4f38-8fa6-f4e95cac9272 2024-09-19T05:42:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] [v2:172.21.15.2:6818/2074594312,v1:172.21.15.2:6819/2074594312] exists,up a4443f63-2c47-4c6f-a2df-6765b78afcc2 2024-09-19T05:42:48.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] [v2:172.21.15.2:6826/2644191985,v1:172.21.15.2:6827/2644191985] exists,up 61465313-1ba1-4f26-b488-6ea73981419b 2024-09-19T05:42:48.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] [v2:172.21.15.2:6802/3985782072,v1:172.21.15.2:6803/3985782072] exists,up fa9d27d5-68d4-4467-a2a5-79f767a071aa 2024-09-19T05:42:48.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] [v2:172.21.15.123:6818/2654957182,v1:172.21.15.123:6819/2654957182] exists,up 85a040ff-ad8b-44ff-b82e-a36b05d83087 2024-09-19T05:42:48.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] [v2:172.21.15.123:6802/3690741328,v1:172.21.15.123:6803/3690741328] exists,up f90f8fa4-4afc-4272-89eb-5e90123155d9 2024-09-19T05:42:48.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] [v2:172.21.15.123:6826/593730548,v1:172.21.15.123:6827/593730548] exists,up 48561580-ba9a-4096-9722-16c308f414ee 2024-09-19T05:42:48.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] [v2:172.21.15.123:6810/1657801827,v1:172.21.15.123:6811/1657801827] exists,up d1ee7357-25a5-432c-b96a-e808139a1cf3 2024-09-19T05:42:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c7806c2e0 msgr2=0x7f5c7806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c7806c2e0 0x7f5c7806e7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5c90001780 tx=0x7f5c90009040 comp rx=0 tx=0).stop 2024-09-19T05:42:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 msgr2=0x7f5c9415cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c9415cdf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d620 tx=0x7f5c8400dae0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 shutdown_connections 2024-09-19T05:42:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c7806c2e0 0x7f5c7806e7a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c9415df60 0x7f5c940cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c9415b700 0x7f5c9415cdf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 --2- 172.21.15.2:0/2537811020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c94154d90 0x7f5c941f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 >> 172.21.15.2:0/2537811020 conn(0x7f5c9406bc10 msgr2=0x7f5c9414adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 shutdown_connections 2024-09-19T05:42:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.279+0000 7f5c9bd6d640 1 -- 172.21.15.2:0/2537811020 wait complete. 2024-09-19T05:42:48.290 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] [v2:172.21.15.2:6810/288510134,v1:172.21.15.2:6811/288510134] exists,up 665c2ac5-a964-4f38-8fa6-f4e95cac9272', 'osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] [v2:172.21.15.2:6818/2074594312,v1:172.21.15.2:6819/2074594312] exists,up a4443f63-2c47-4c6f-a2df-6765b78afcc2', 'osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] [v2:172.21.15.2:6826/2644191985,v1:172.21.15.2:6827/2644191985] exists,up 61465313-1ba1-4f26-b488-6ea73981419b', 'osd.3 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] [v2:172.21.15.2:6802/3985782072,v1:172.21.15.2:6803/3985782072] exists,up fa9d27d5-68d4-4467-a2a5-79f767a071aa', 'osd.4 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] [v2:172.21.15.123:6818/2654957182,v1:172.21.15.123:6819/2654957182] exists,up 85a040ff-ad8b-44ff-b82e-a36b05d83087', 'osd.5 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] [v2:172.21.15.123:6802/3690741328,v1:172.21.15.123:6803/3690741328] exists,up f90f8fa4-4afc-4272-89eb-5e90123155d9', 'osd.6 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] [v2:172.21.15.123:6826/593730548,v1:172.21.15.123:6827/593730548] exists,up 48561580-ba9a-4096-9722-16c308f414ee', 'osd.7 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] [v2:172.21.15.123:6810/1657801827,v1:172.21.15.123:6811/1657801827] exists,up d1ee7357-25a5-432c-b96a-e808139a1cf3'] 2024-09-19T05:42:48.291 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-19T05:42:48.393 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:48.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mon.a", 2024-09-19T05:42:48.393 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster": "ceph", 2024-09-19T05:42:48.393 INFO:teuthology.orchestra.run.smithi002.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-19T05:42:48.393 INFO:teuthology.orchestra.run.smithi002.stdout: "admin_socket_mode": "", 2024-09-19T05:42:48.393 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_client_required": "cephx, none", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_cluster_required": "cephx", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_debug": "true", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_service_required": "cephx", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_supported": "cephx", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio": "true", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_poll_ms": "250", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_reap_max": "16", 2024-09-19T05:42:48.394 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_async_discard": "false", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_block_size": "4096", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio": "false", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_inflight_ios": "false", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_enable_discard": "false", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_flock_retry": "3", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_inject_crash": "0", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring": "false", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring_hipri": "false", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-19T05:42:48.395 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_type": "", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_alloc_size": "1048576", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_allocator": "hybrid", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_buffered_io": "true", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_for_zeros": "false", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_compact_log_sync": "false", 2024-09-19T05:42:48.396 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_max_log_runway": "4194304", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_max_prefetch": "1048576", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_min_flush_size": "524288", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_min_log_runway": "1048576", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_replay_recovery": "false", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_sync_write": "false", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-19T05:42:48.397 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_allocation_from_file": "true", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_allocator": "hybrid", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bdev_label_multi": "true", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-19T05:42:48.398 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_create": "true", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_create": "false", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_path": "", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_size": "0", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_path": "", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_preallocate_file": "false", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_size": "107374182400", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_create": "false", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_path": "", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_size": "100663296", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs": "true", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-19T05:42:48.399 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_autotune": "true", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size": "0", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-19T05:42:48.400 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_type": "2q", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_clone_cow": "true", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_mode": "none", 2024-09-19T05:42:48.401 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_csum_type": "crc32c", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_freelist": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-19T05:42:48.402 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_small_allocations": "0", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_default_buffered_read": "true", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_default_buffered_write": "false", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-19T05:42:48.403 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fail_eio": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mount": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_umount": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-19T05:42:48.404 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_ignore_data_csum": "false", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_op_age": "5.000000", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_alloc_size": "0", 2024-09-19T05:42:48.405 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size": "0", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_deferred_txc": "32", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size": "0", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_nid_prealloc": "1024", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_retry_disk_reads": "3", 2024-09-19T05:42:48.406 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_cf": "true", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_mem": "512", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-19T05:42:48.407 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_tracing": "false", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-19T05:42:48.408 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_zero_block_detection": "false", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-19T05:42:48.409 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_cluster_require_signatures": "false", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_cluster_require_version": "2", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_require_signatures": "false", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_require_version": "2", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_service_require_signatures": "false", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_service_require_version": "2", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_sign_messages": "true", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "chdir": "", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_acl_type": "", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_asio_thread_count": "2", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_cache_mid": "0.750000", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_cache_size": "16384", 2024-09-19T05:42:48.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_caps_release_delay": "5", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_check_pool_perm": "true", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_force_sync_read": "false", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_getattr_caps": "false", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_inject_features": "", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_inject_tick_delay": "0", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_die_on_failed_remount": "false", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_dirsize_rbytes": "true", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_force_lazyio": "false", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_fs": "", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-19T05:42:48.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_inject_release_failure": "false", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_max_inline_size": "4096", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mds_namespace": "", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_metadata": "", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_gid": "-1", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_timeout": "300", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_uid": "-1", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mountpoint": "/", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_notify_timeout": "10", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc": "true", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_dirty": "104857600", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_objects": "1000", 2024-09-19T05:42:48.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_size": "209715200", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_target_dirty": "8388608", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_permissions": "true", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_quota": "true", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_quota_df": "true", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_max_bytes": "0", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_max_periods": "4", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_min": "131072", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_reconnect_stale": "false", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_shutdown_timeout": "30", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_snapdir": ".snap", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_tick_interval": "1", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_trace": "", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_try_dentry_invalidate": "false", 2024-09-19T05:42:48.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_use_faked_inos": "false", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_use_random_mds": "false", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog": "false", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog_port": "12201", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_monitors": "default=true", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog": "false", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog_level": "info", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "-", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_network": "", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_network_interface": "", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_isal": "false", 2024-09-19T05:42:48.414 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_level": "5", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_winsize": "-15", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zstd_level": "1", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_alien_op_num_threads": "6", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_stat_interval": "0", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_seastar_cpu_cores": "", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_seastar_num_threads": "0", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location": "", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location_hook": "", 2024-09-19T05:42:48.415 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location_hook_timeout": "10", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "daemonize": "false", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "daos_pool": "tank", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_alienstore": "0/5", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_allow_any_pool_priority": "false", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asok": "1/5", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asok_assert_abort": "false", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asserts_on_shutdown": "true", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_auth": "20/20", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bdev": "1/3", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bluefs": "1/5", 2024-09-19T05:42:48.416 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bluestore": "1/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_buffer": "0/1", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_ceph_exporter": "1/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cephfs_mirror": "0/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cephsqlite": "0/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_client": "0/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_compressor": "1/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_context": "0/1", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crimson_interrupt": "0/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crush": "1/1", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crypto": "1/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cyanstore": "0/5", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_deliberately_leak_memory": "false", 2024-09-19T05:42:48.417 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_disable_randomized_ping": "false", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_dpdk": "1/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_eventtrace": "1/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_filer": "0/1", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_filestore": "1/3", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_finisher": "1/1", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_fuse": "1/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_heartbeat_testing_span": "0", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_heartbeatmap": "1/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_javaclient": "1/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_journal": "1/3", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_journaler": "0/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_kstore": "1/5", 2024-09-19T05:42:48.418 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_lockdep": "0/1", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mclock": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_balancer": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_locker": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_log": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_log_expire": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_migrator": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_quiesce": "3/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_memstore": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mgr": "2/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mgrc": "1/5", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mon": "20/20", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_monc": "0/10", 2024-09-19T05:42:48.419 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_ms": "1/1", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_none": "0/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objclass": "0/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objectcacher": "0/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objecter": "0/1", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_optracker": "0/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_osd": "1/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_paxos": "20/20", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_perfcounter": "1/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_prioritycache": "1/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rados": "0/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd": "0/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_mirror": "0/5", 2024-09-19T05:42:48.420 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_pwl": "0/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_replay": "0/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_refs": "0/0", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_reserver": "1/1", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw": "1/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_access": "1/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_datacache": "1/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_dbstore": "1/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_flight": "1/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_sync": "1/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rocksdb": "4/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore": "0/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_backref": "0/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_cache": "0/5", 2024-09-19T05:42:48.421 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_cleaner": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_device": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_epm": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_journal": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_lba": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_odata": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_omap": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_onode": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_t": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_tm": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_striper": "0/1", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_test": "0/5", 2024-09-19T05:42:48.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_throttle": "1/1", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_timer": "0/1", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_tp": "0/5", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_trace": "1/5", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "device_failure_prediction_mode": "none", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_graylog": "false", 2024-09-19T05:42:48.423 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_journald": "false", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_stderr": "true", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_syslog": "false", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event_tracing": "false", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_addr": "0.0.0.0", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_http_port": "9926", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_prio_limit": "5", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_sort_metrics": "true", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_stats_period": "5", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "fake_statfs_for_testing": "0", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "fatal_signal_handlers": "true", 2024-09-19T05:42:48.424 INFO:teuthology.orchestra.run.smithi002.stdout: "filer_max_purge_ops": "10", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filer_max_truncate_ops": "128", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_apply_finisher_threads": "1", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_blackhole": "false", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_btrfs_clone_range": "true", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_btrfs_snap": "true", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_caller_concurrency": "10", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_collect_device_partition_information": "true", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_commit_timeout": "600.000000", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_inject_read_err": "false", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_omap_check": "false", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_verify_split": "false", 2024-09-19T05:42:48.425 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_dump_file": "", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fadvise": "true", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fail_eio": "true", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fd_cache_shards": "16", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fd_cache_size": "128", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fiemap": "false", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fiemap_threshold": "4096", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_inject_stall": "0", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_parallel": "false", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_trailing": "false", 2024-09-19T05:42:48.426 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_writeahead": "false", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_kill_at": "0", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs": "0", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size": "0", 2024-09-19T05:42:48.427 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_merge_threshold": "-10", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_odsync_write": "false", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_backend": "rocksdb", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_backend_path": "", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_thread_timeout": "60", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_threads": "2", 2024-09-19T05:42:48.428 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_punch_hole": "false", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_ops": "50", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_seek_data_hole": "false", 2024-09-19T05:42:48.429 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_sloppy_crc": "false", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_splice": "false", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_split_multiple": "2", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_split_rand_factor": "20", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_update_to": "1000", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_enable": "true", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-19T05:42:48.430 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_xfs_extsize": "false", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_zfs_snap": "false", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fio_dir": "/tmp/fio", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "896e162c-4658-4743-b70e-7ec2b5bce950", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_allow_other": "true", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_atomic_o_trunc": "true", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_big_writes": "true", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_debug": "false", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_default_permissions": "false", 2024-09-19T05:42:48.431 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_disable_pagecache": "false", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_max_write": "0", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_multithreaded": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_require_active_mds": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_set_user_groups": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_move": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_read": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_write": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_use_invalidate_cb": "true", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "gss_target_name": "ceph", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_file": "", 2024-09-19T05:42:48.432 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_inject_failure": "0", 2024-09-19T05:42:48.433 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_interval": "5", 2024-09-19T05:42:48.433 INFO:teuthology.orchestra.run.smithi002.stdout: "host": "smithi002", 2024-09-19T05:42:48.433 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-19T05:42:48.433 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-19T05:42:48.433 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-19T05:42:48.433 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-19T05:42:48.433 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-19T05:42:48.436 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-19T05:42:48.436 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-19T05:42:48.436 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "inject_early_sigterm": "false", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "jaeger_agent_port": "6799", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "jaeger_tracing_enable": "false", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_aio": "true", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_align_min_size": "65536", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_block_align": "true", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_block_size": "4096", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_dio": "true", 2024-09-19T05:42:48.437 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_discard": "false", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_force_aio": "false", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_ignore_corruption": "false", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_max_write_bytes": "10485760", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_max_write_entries": "100", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_replay_from": "0", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_write_header_frequency": "0", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_zero_on_create": "false", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_prefetch_periods": "10", 2024-09-19T05:42:48.438 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_prezero_periods": "5", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_write_head_interval": "15", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "keyfile": "", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_backend": "rocksdb", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_default_stripe_size": "65536", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_fsck_on_mount": "false", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_max_bytes": "67108864", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_max_ops": "512", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_nid_prealloc": "1024", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_onode_map_size": "1024", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-19T05:42:48.439 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_sync_submit_transaction": "false", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_sync_transaction": "false", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "librados_thread_count": "2", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "lockdep": "false", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "lockdep_force_backtrace": "false", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_coarse_timestamps": "true", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_flush_on_exit": "false", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_graylog_host": "127.0.0.1", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_graylog_port": "12201", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_max_new": "1000", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_max_recent": "10000", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_stderr_prefix": "", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_stop_at_utilization": "0.970000", 2024-09-19T05:42:48.440 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": "true", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_graylog": "false", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_journald": "false", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_stderr": "false", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_syslog": "false", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max_rotating_auth_attempts": "10", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_action_on_write_error": "1", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_alternate_name_max": "8192", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_asio_thread_count": "2", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_export_pin": "true", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_dirs": "true", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-19T05:42:48.441 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_interval": "5", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_interval": "10", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_max": "-1", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_max_until": "-1", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_merge_size": "50", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_midchunk": "0.300000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_min_start": "0.200000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_minchunk": "0.001000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_mode": "0", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_need_max": "1.200000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_need_min": "0.800000", 2024-09-19T05:42:48.442 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_overload_epochs": "2", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_bits": "3", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_size": "10000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_target_decay": "10.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_grace": "15.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_interval": "4.000000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_mid": "0.700000", 2024-09-19T05:42:48.443 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_delay": "0", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_release_free_interval": "10", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_reservation": "0.050000", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_interval": "1", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_threshold": "262144", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-19T05:42:48.444 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_prealloc_inos": "1000", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_connect_bootstrapping": "false", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_damage_table_max_entries": "10000", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_auth_pins": "false", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_frag": "false", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_scatterstat": "false", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_subtrees": "false", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_decay_halflife": "5.000000", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_default_dir_hash": "2", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_defer_session_stale": "true", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_deny_all_reconnect": "false", 2024-09-19T05:42:48.445 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_keys_per_op": "16384", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_max_commit_size": "10", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_max_entries": "0", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_prefetch": "true", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_on_map": "false", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_early_reply": "true", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_enable_op_tracker": "true", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_enforce_unique_name": "true", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-19T05:42:48.446 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_random": "true", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_health_summarize_threshold": "10", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_health_dummy": "false", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-19T05:42:48.447 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_migrator_session_race": "false", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_join_fs": "", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_journal_format": "1", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_create_at": "0", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_export_at": "0", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_import_at": "0", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_at": "0", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_expire_at": "0", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_replay_at": "0", 2024-09-19T05:42:48.448 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_link_at": "0", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_mdstable_at": "0", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_openc_at": "0", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_rename_at": "0", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_shutdown_at": "0", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_event_large_threshold": "524288", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_events_per_segment": "1024", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_max_events": "-1", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_max_segments": "128", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_pause": "false", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_segment_size": "0", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-19T05:42:48.449 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_trim_threshold": "128", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_warn_factor": "2.000000", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_caps_per_client": "1048576", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_completed_flushes": "100000", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_completed_requests": "100000", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_export_size": "20971520", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_file_recover": "32", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_files": "64", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_ops": "8192", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-19T05:42:48.450 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_snaps_per_dir": "100", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_metrics_update_interval": "2", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_min_caps_per_client": "100", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_min_caps_working_set": "10000", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_numa_node": "-1", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_complaint_time": "30.000000", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_duration": "600", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_size": "20", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_slow_op_size": "20", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_log_threshold": "5", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_ping_grace": "15", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_ping_interval": "5", 2024-09-19T05:42:48.451 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_caps": "30000", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_warning_threshold": "262144", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_replay_interval": "1.000000", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_root_ino_gid": "0", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_root_ino_uid": "0", 2024-09-19T05:42:48.452 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_shutdown_check": "0", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_skip_ino": "0", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_max_uid": "4294967294", 2024-09-19T05:42:48.453 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_min_uid": "0", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_rstat": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_standby_replay_damaged": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_symlink_recovery": "true", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_thrash_exports": "0", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_thrash_fragments": "0", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_tick_interval": "5.000000", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_valgrind_exit": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_verify_backtrace": "1", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_verify_scatter": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_wipe_sessions": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool_debug": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-19T05:42:48.454 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_device_bytes": "1073741824", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_page_set": "false", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_page_size": "65536", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_bytes": "134217728", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_messages": "512", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_disabled_modules": "", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_enable_op_tracker": "true", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_max_pg_num_change": "128", 2024-09-19T05:42:48.455 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mds_bytes": "134217728", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mds_messages": "128", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mon_bytes": "134217728", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mon_messages": "128", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_duration": "600", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_size": "20", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_op_log_threshold": "5", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_osd_bytes": "536870912", 2024-09-19T05:42:48.456 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_osd_messages": "8192", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_pool": "true", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_standby_modules": "true", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_stats_period": "5", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_stats_threshold": "5", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_tick_period": "2", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_allow_pool_delete": "true", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_allow_pool_size_one": "true", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_auth_validate_all_caps": "true", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-19T05:42:48.457 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_bytes": "104857600", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_directed_command_retry": "2", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_parallel": "3", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_log_interval": "1.000000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_ping_interval": "10.000000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_target_rank": "-1", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-19T05:42:48.458 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_file_level": "debug", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_file": "true", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_journald": "false", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_bootstrap": "false", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_start": "false", 2024-09-19T05:42:48.459 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_trim": "true", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cpu_threads": "4", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_crush_min_required_version": "hammer", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_daemon_bytes": "419430400", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_avail_crit": "5", 2024-09-19T05:42:48.460 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_avail_warn": "5", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_size_warn": "16106127360", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_json": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_transactions": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_extra_checks": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_reef": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_squid": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-19T05:42:48.461 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_down_mkfs_grace": "300", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_down_uptime_grace": "60", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_election_default_strategy": "1", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_election_timeout": "5.000000", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ping_divisor": "2", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_enable_op_tracker": "true", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_fake_pool_delete": "false", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_force_quorum_join": "false", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_globalid_prealloc": "10000", 2024-09-19T05:42:48.462 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_detail_to_clog": "false", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_log_update_period": "5", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_max_detail": "50", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog": "true", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog_interval": "600", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_host": "172.21.15.2,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_host_override": "", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_initial_members": "", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-19T05:42:48.463 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease": "5.000000", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_full_interval": "50", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_max": "10000", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_max_summary": "50", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_log_epochs": "500", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_osd": "10000", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_pg_per_osd": "10000", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_pool_pg_num": "65536", 2024-09-19T05:42:48.464 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_force_trim_to": "0", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_skip_sanity": "false", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_memory_autotune": "true", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_memory_target": "2147483648", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_beacon_grace": "30", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_digest_period": "5", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_inactive_grace": "60", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_min_osdmap_epochs": "500", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_complaint_time": "120", 2024-09-19T05:42:48.465 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_duration": "600", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_size": "20", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_slow_op_size": "20", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_log_threshold": "5", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_in": "false", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_cache_size": "500", 2024-09-19T05:42:48.466 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_down_out_interval": "600", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_force_trim_to": "0", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-19T05:42:48.467 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_down_reporters": "2", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_report_timeout": "900", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-19T05:42:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_stuck_threshold": "60", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_probe_timeout": "2.000000", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_max_change": "0.050000", 2024-09-19T05:42:48.469 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_max_osds": "4", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_interval": "86400", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_max_keys": "100", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_timeout": "300", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_session_timeout": "300", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_smart_report_timeout": "5", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stat_smooth_intervals": "6", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-19T05:42:48.470 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_pool_min_size": "2", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_pool_size": "4", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_debug": "false", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_provider_kill_at": "0", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_requester_kill_at": "0", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_timeout": "60.000000", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_target_pg_per_osd": "100", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_tick_interval": "5", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_timecheck_interval": "300.000000", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-19T05:42:48.471 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_use_min_delay_socket": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_older_version_delay": "604800", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_misplaced": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_older_version": "true", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-19T05:42:48.472 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "monmap": "", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-19T05:42:48.473 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_tracing_enabled": "false", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_op_threads": "3", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_cm": "false", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_device_name": "", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_dscp": "96", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_local_gid": "", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_port_num": "1", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-19T05:42:48.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_sl": "3", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_support_srq": "true", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_type": "ib", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_reap_threshold": "5", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_before_connect": "false", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_ipv4": "true", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_ipv6": "false", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_msgr1": "true", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_msgr2": "true", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_port_max": "7568", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_port_min": "6800", 2024-09-19T05:42:48.475 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_retry_count": "3", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_retry_delay": "5", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_client": "false", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mds": "false", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mgr": "false", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mon": "false", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_osd": "false", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_client_mode": "crc secure", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_cluster_mode": "crc secure", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_cluster_type": "", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_compress_secure": "false", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_connection_idle_timeout": "900", 2024-09-19T05:42:48.476 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_connection_ready_timeout": "10", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_crc_data": "true", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_crc_header": "true", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_bad_msg": "false", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_bug": "true", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_old_message": "true", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_skipped_message": "false", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_coremask": "0xF", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_enable_tso": "true", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-19T05:42:48.477 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hugepages": "", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_lro": "true", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_memory_channel": "4", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_pmd": "", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_port_id": "0", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dump_on_send": "false", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_initial_backoff": "0.200000", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_max": "1.000000", 2024-09-19T05:42:48.478 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_type": "", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_network_congestion": "0", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_socket_failures": "0", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_learn_addr_from_peer": "true", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_max_accept_failures": "4", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_max_backoff": "15.000000", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_client_mode": "secure crc", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_service_mode": "secure crc", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compress_min_size": "1024", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compress_mode": "none", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-19T05:42:48.479 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_pq_min_cost": "65536", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_public_type": "", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_service_mode": "crc secure", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_listen_backlog": "512", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_nodelay": "true", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_rcvbuf": "0", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_type": "async+posix", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "no_config_file": "false", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_completion_locks_per_session": "32", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inflight_ops": "1024", 2024-09-19T05:42:48.480 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_tick_interval": "5.000000", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_timeout": "10.000000", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "objectstore_blackhole": "false", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "openssl_engine_opts": "", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_delay_time": "5.000000", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_hist_halflife": "1000", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_max_low_ops": "2", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_max_ops": "4", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_slop": "0.020000", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-19T05:42:48.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_asio_thread_count": "2", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_async_recovery_min_cost": "100", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_scan_max": "512", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_scan_min": "64", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_degraded": "false", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_peering": "false", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_unfound": "true", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_beacon_report_interval": "300", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_duration": "30", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_max_block_size": "67108864", 2024-09-19T05:42:48.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_blkin_trace_all": "false", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_check_for_log_corruption": "false", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_update_on_start": "true", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_message_cap": "256", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_message_size_cap": "524288000", 2024-09-19T05:42:48.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_op_priority": "63", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_watch_timeout": "30", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_command_max_records": "256", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_compact_on_start": "false", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_update_on_start": "true", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_update_weight_set": "true", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-19T05:42:48.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_feed_pullee": "-1", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_misdirected_ops": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_no_acting_change": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_no_purge_strays": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_op_order": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_shutdown": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-19T05:42:48.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_trim_objects": "false", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_snaps": "false", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_keys": "1024", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_stride": "524288", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-19T05:42:48.486 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_default_notify_timeout": "30", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep": "0.000000", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_discard_disconnected_ops": "true", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_enable_op_tracker": "true", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_info": "true", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown": "true", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-19T05:42:48.487 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_function_tracing": "false", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_grace": "20", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_interval": "6", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_peers": "10", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_size": "2000", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_stale": "600", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_max_size": "100000", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_min_size": "1000", 2024-09-19T05:42:48.488 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal_size": "5120", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_kill_backfill_at": "0", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_cache_size": "50", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_dedup": "true", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_message_max": "40", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-19T05:42:48.489 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_share_max_epochs": "40", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_attr_name_len": "100", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_attr_size": "0", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_backfills": "1", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_markdown_count": "5", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_markdown_period": "600", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_name_len": "2048", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_namespace_len": "256", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_size": "134217728", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_blocked_by": "16", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_log_entries": "10000", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pgls": "1024", 2024-09-19T05:42:48.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_push_cost": "8388608", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_push_objects": "10", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_scrubs": "3", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_trimming_pgs": "2", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_write_op_reply_len": "64", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_write_size": "90", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-19T05:42:48.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_profile": "balanced", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-19T05:42:48.492 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_base": "805306368", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_cache_min": "134217728", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target": "4294967296", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target_autotune": "false", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_min_pg_log_entries": "250", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_min_recovery_priority": "0", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_report_interval": "5", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-19T05:42:48.493 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_num_cache_shards": "32", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_num_op_tracker_shard": "32", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_auto_affinity": "true", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_node": "-1", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_prefer_iface": "true", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objecter_finishers": "1", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore": "bluestore", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore_fuse": "false", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore_tracing": "false", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_complaint_time": "30.000000", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_duration": "600", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_size": "20", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_slow_op_size": "20", 2024-09-19T05:42:48.494 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_log_threshold": "5", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards": "0", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards_hdd": "5", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards_ssd": "8", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_pq_min_cost": "65536", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_queue_cut_off": "high", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_thread_timeout": "15", 2024-09-19T05:42:48.495 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_open_classes_on_start": "true", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_os_flags": "0", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_peering_op_priority": "255", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_delete_cost": "1048576", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_delete_priority": "5", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_trim_max": "10000", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_trim_min": "100", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-19T05:42:48.496 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_crimson": "false", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-19T05:42:48.497 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flags": "0", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_min_size": "0", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pg_num": "32", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pgp_num": "0", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_read_ratio": "70", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_size": "2", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_type": "replicated", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-19T05:42:48.498 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_push_per_object_cost": "1000", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recover_clone_overlap": "true", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_cost": "20971520", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active": "0", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_single_start": "1", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_op_priority": "3", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-19T05:42:48.499 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_priority": "5", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep": "0.000000", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_repair_during_recovery": "false", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_requested_scrub_priority": "120", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_auto_repair": "false", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-19T05:42:48.500 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_begin_hour": "0", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_begin_week_day": "0", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_chunk_max": "25", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_chunk_min": "5", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_cost": "52428800", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_during_recovery": "false", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_end_hour": "0", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_end_week_day": "0", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_event_cost": "4096", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_invalid_stats": "true", 2024-09-19T05:42:48.501 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_max_preemptions": "5", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_priority": "5", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_sleep": "0.000000", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_skip_data_digest": "false", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_smart_report_timeout": "5", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_cost": "1048576", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_priority": "5", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-19T05:42:48.502 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_target_transaction_size": "30", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-19T05:42:48.503 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tracing": "false", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_use_stale_snap": "false", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "osdc_blkin_trace_all": "false", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_kill_at": "0", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_max_join_drift": "10", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_min": "500", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_min_wait": "0.050000", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_propose_interval": "1.000000", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_max": "500", 2024-09-19T05:42:48.504 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_min": "250", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_stash_full_interval": "25", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_trim_max": "500", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_trim_min": "250", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "perf": "true", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "-", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrv": "", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "public_bind_addr": "-", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "public_network": "", 2024-09-19T05:42:48.505 INFO:teuthology.orchestra.run.smithi002.stdout: "public_network_interface": "", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_busy_polling": "false", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_enabled": "false", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_session_max_number": "256", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_mon_op_timeout": "0", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_osd_op_timeout": "0", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_tracing": "false", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_atime_update_interval": "60", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_balance_parent_reads": "false", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_balance_snap_reads": "false", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blkin_trace_all": "false", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache": "true", 2024-09-19T05:42:48.506 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_policy": "writearound", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_size": "33554432", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_clone_copy_on_read": "false", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_compression_hint": "none", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_concurrent_management_ops": "10", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_clone_format": "auto", 2024-09-19T05:42:48.507 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_data_pool": "", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_features": "61", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_format": "2", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_map_options": "", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_order": "22", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_pool": "rbd", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_stripe_count": "0", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_stripe_unit": "0", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_enable_alloc_hint": "true", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-19T05:42:48.508 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_io_scheduler": "simple", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_order": "24", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_pool": "", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_splay_width": "4", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_localize_parent_reads": "false", 2024-09-19T05:42:48.509 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_localize_snap_reads": "false", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-19T05:42:48.510 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-19T05:42:48.511 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mtime_update_interval": "60", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_non_blocking_aio": "true", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_op_thread_timeout": "60", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_op_threads": "1", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_parent_cache_enabled": "false", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_plugins": "", 2024-09-19T05:42:48.512 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_burst": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_limit": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_exclude_ops": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_burst": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_limit": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-19T05:42:48.513 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_read_from_replica_policy": "default", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_skip_partial_discard": "true", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-19T05:42:48.514 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_tracing": "false", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_validate_names": "true", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_validate_pool": "true", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "restapi_base_url": "", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "restapi_log_level": "", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_acl_grants_max_num": "100", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_admin_entry": "admin", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_backend_store": "rados", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_barbican_url": "", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_beast_enable_async": "true", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_counters_cache": "false", 2024-09-19T05:42:48.515 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_enabled": "true", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_expiry_interval": "900", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_lru_size": "25000", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_config_store": "rados", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_content_length_compat": "false", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_copy_obj_progress": "true", 2024-09-19T05:42:48.516 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cors_rules_max_num": "100", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cross_domain_policy": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_addr": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_password": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_username": "", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_require_ssl": "true", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-19T05:42:48.517 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-19T05:42:48.518 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_addr": "", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_auth": "token", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_namespace": "", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_prefix": "", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_token_file": "", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_buffersize": "524288", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_low_speed_time": "300", 2024-09-19T05:42:48.519 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d4n_port": "6379", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_changes_size": "1000", 2024-09-19T05:42:48.520 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_num_shards": "128", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_window": "30", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-19T05:42:48.521 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-19T05:42:48.522 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dns_name": "", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dns_s3website_name": "", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dynamic_resharding": "true", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_gc_threads": "true", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_lc_threads": "true", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_ops_log": "false", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_quota_threads": "true", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_static_website": "false", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_usage_log": "false", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enforce_swift_acls": "true", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_exit_timeout_secs": "120", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_expose_bucket": "false", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_extended_http_attrs": "", 2024-09-19T05:42:48.523 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_filter": "none", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_frontends": "beast port=7480", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_deferred": "50", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_objs": "32", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_processor_period": "3600", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-19T05:42:48.524 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_init_timeout": "300", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_kafka_connection_idle": "300", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_domain": "", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_password": "", 2024-09-19T05:42:48.525 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_password_path": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_project": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_tenant": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_token": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_token_path": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_user": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_api_version": "2", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_domain": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_password": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_project": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_user": "", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-19T05:42:48.526 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_url": "", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_debug_interval": "-1", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_lock_max_time": "90", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_objs": "32", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_rules": "1000", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_worker": "3", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_thread_delay": "0", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_dnattr": "uid", 2024-09-19T05:42:48.527 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_searchfilter": "", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_uri": "", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_http_headers": "", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_object_name_utc": "false", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-19T05:42:48.528 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attr_name_len": "0", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attr_size": "0", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_chunk_size": "4194304", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_listing_results": "1000", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_notify_retries": "10", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_put_param_size": "1048576", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_put_size": "5368709120", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_slo_entries": "1000", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_md_log_max_shards": "64", 2024-09-19T05:42:48.529 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_mp_lock_max_time": "600", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_max_gc": "300", 2024-09-19T05:42:48.530 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_num_async_rados_threads": "32", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_num_control_oids": "8", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_numa_node": "-1", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_chunk_size": "100", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_gc_interval": "600", 2024-09-19T05:42:48.531 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_thread_timeout": "600", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_tracing": "false", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_token": "", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_url": "", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_verify_ssl": "true", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_rados": "false", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_socket_path": "", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-19T05:42:48.532 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_push_interval": "2.000000", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_lanes": "3", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_cache_partitions": "3", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_print_continue": "true", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-19T05:42:48.533 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_tracing": "false", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm": "", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm_id": "", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_region": "", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_request_uri": "", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_batch_size": "64", 2024-09-19T05:42:48.534 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_max_aio": "128", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_num_logs": "16", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_thread_interval": "600", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_resolve_cname": "false", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_run_sync_thread": "true", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-19T05:42:48.535 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_scheduler_type": "throttler", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_script_uri": "", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_service_provider_name": "", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_client_id": "", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_client_secret": "", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_entry": "sts", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_key": "sts", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_min_session_duration": "900", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_token_introspection_url": "", 2024-09-19T05:42:48.536 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_account_in_url": "false", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_auth_entry": "auth", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_auth_url": "", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_custom_header": "", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_need_stats": "true", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_tenant_name": "", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_token_expiration": "86400", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_url": "", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_url_prefix": "swift", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_lease_period": "120", 2024-09-19T05:42:48.537 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_thread_pool_size": "512", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_comment": "", 2024-09-19T05:42:48.538 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_createby": "", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_encoding": "", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_flag": "false", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_origin": "", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_tracker": "", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_trust_forwarded_https": "false", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_max_shards": "32", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_max_user_shards": "1", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_use_opa_authz": "false", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_counters_cache": "false", 2024-09-19T05:42:48.539 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_max_buckets": "1000", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_policies_max_num": "100", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_unique_email": "true", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_verify_ssl": "true", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone": "", 2024-09-19T05:42:48.540 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone_id": "", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup": "", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup_id": "", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rgwlc_auto_session_clear": "true", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_block_size": "4096", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_size": "536870912", 2024-09-19T05:42:48.541 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_index_type": "binary_search", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_partition_filters": "false", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_perf": "false", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-19T05:42:48.542 INFO:teuthology.orchestra.run.smithi002.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "run_dir": "/var/run/ceph", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_block_create": "true", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_cache_lru_size": "67108864", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_default_max_object_size": "16777216", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_device_size": "53687091200", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_full_integrity_check": "false", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_capacity": "16", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-19T05:42:48.543 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_main_device_type": "SSD", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_segment_size": "67108864", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "set_keepcaps": "false", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "setgroup": "", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "setuser": "", 2024-09-19T05:42:48.544 INFO:teuthology.orchestra.run.smithi002.stdout: "setuser_match_path": "", 2024-09-19T05:42:48.545 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-19T05:42:48.545 INFO:teuthology.orchestra.run.smithi002.stdout: "thp": "false", 2024-09-19T05:42:48.545 INFO:teuthology.orchestra.run.smithi002.stdout: "threadpool_default_timeout": "60", 2024-09-19T05:42:48.545 INFO:teuthology.orchestra.run.smithi002.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-19T05:42:48.545 INFO:teuthology.orchestra.run.smithi002.stdout: "throttler_perf_counter": "true" 2024-09-19T05:42:48.545 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:48.547 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-19T05:42:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.646+0000 7fdb7a759640 1 Processor -- start 2024-09-19T05:42:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.647+0000 7fdb7a759640 1 -- start start 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.647+0000 7fdb7a759640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb74152980 0x7fdb74158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.647+0000 7fdb7a759640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb740c5550 con 0x7fdb74159350 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.647+0000 7fdb7a759640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb740ca350 con 0x7fdb740c9e90 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.647+0000 7fdb7a759640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb740c3d10 con 0x7fdb74152980 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb6bfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fdb74159350 0x7fdb74070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41186/0 (socket says 172.21.15.2:41186) 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb6bfff640 1 -- 172.21.15.2:0/2525206432 learned_addr learned my addr 172.21.15.2:0/2525206432 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb73fff640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb74152980 0x7fdb74158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760920056 0 0) 0x7fdb740c5550 con 0x7fdb74159350 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb60003520 con 0x7fdb74159350 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762036950 0 0) 0x7fdb740ca350 con 0x7fdb740c9e90 2024-09-19T05:42:48.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb740c5550 con 0x7fdb740c9e90 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1438065327 0 0) 0x7fdb60003520 con 0x7fdb74159350 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb74152980 msgr2=0x7fdb74158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb74152980 0x7fdb74158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 >> v1:172.21.15.123:6789/0 conn(0x7fdb740c9e90 legacy=0x7fdb74162fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb74158630 con 0x7fdb74159350 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7fdb5c001670 con 0x7fdb74159350 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 >> v1:172.21.15.2:6789/0 conn(0x7fdb74159350 legacy=0x7fdb74070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb60003b70 0x7fdb60003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60004800 0x7fdb60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740c3d10 con 0x7fdb600051c0 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6000bf60 con 0x7fdb60004800 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb6000c160 con 0x7fdb60003b70 2024-09-19T05:42:48.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb73fff640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb74152980 0x7fdb74158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.648+0000 7fdb6bfff640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60004800 0x7fdb60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.649+0000 7fdb73fff640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb60003b70 0x7fdb60003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.649+0000 7fdb78cf6640 1 -- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb60003b70 msgr2=0x7fdb60003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.649+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb60003b70 0x7fdb60003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.649+0000 7fdb78cf6640 1 -- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60004800 msgr2=0x7fdb60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.649+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb60004800 0x7fdb60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.649+0000 7fdb78cf6640 1 -- 172.21.15.2:0/2525206432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb60003520 con 0x7fdb600051c0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.649+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb6000b9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001fc0 tx=0x7fdb60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be441e11db2306b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c001be0 con 0x7fdb600051c0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6c003b90 con 0x7fdb600051c0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb737fe640 1 -- 172.21.15.2:0/2525206432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0050d0 con 0x7fdb600051c0 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb7a759640 1 -- 172.21.15.2:0/2525206432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 msgr2=0x7fdb6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb7a759640 1 --2- 172.21.15.2:0/2525206432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb6000b9c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001fc0 tx=0x7fdb60005830 comp rx=0 tx=0).stop 2024-09-19T05:42:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb78cf6640 1 -- 172.21.15.2:0/2525206432 reap_dead start 2024-09-19T05:42:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb7a759640 1 -- 172.21.15.2:0/2525206432 shutdown_connections 2024-09-19T05:42:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb7a759640 1 -- 172.21.15.2:0/2525206432 >> 172.21.15.2:0/2525206432 conn(0x7fdb74067950 msgr2=0x7fdb74067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb7a759640 1 -- 172.21.15.2:0/2525206432 shutdown_connections 2024-09-19T05:42:48.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.650+0000 7fdb7a759640 1 -- 172.21.15.2:0/2525206432 wait complete. 2024-09-19T05:42:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.651+0000 7fdb7a759640 1 Processor -- start 2024-09-19T05:42:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.651+0000 7fdb7a759640 1 -- start start 2024-09-19T05:42:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb7a759640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb74151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb7a759640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7414bc10 0x7fdb740af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb7a759640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af5d0 0x7fdb740b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb7a759640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740c3a30 con 0x7fdb600051c0 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb7a759640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74152d60 con 0x7fdb7414bc10 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb7a759640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb741588f0 con 0x7fdb740af5d0 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb73fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb74151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb73fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb74151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47728/0 (socket says 172.21.15.2:47728) 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb73fff640 1 -- 172.21.15.2:0/748766825 learned_addr learned my addr 172.21.15.2:0/748766825 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb6bfff640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7414bc10 0x7fdb740af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.652+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af5d0 0x7fdb740b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb73fff640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af5d0 msgr2=0x7fdb740b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb73fff640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af5d0 0x7fdb740b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb73fff640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7414bc10 msgr2=0x7fdb740af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb73fff640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7414bc10 0x7fdb740af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb73fff640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb741510b0 con 0x7fdb600051c0 2024-09-19T05:42:48.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb6bfff640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb7414bc10 0x7fdb740af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af5d0 0x7fdb740b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb73fff640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb74151a90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb64011c00 tx=0x7fdb6400f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64011cf0 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb64011e90 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb740c8bd0 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64014640 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.653+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7406eb90 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.654+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdb741516c0 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdb64012040 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdb5406c2b0 0x7fdb5406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdb6408c1e0 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1281 (secure 0 0 0) 0x7fdb6402b020 con 0x7fdb600051c0 2024-09-19T05:42:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb6bfff640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdb5406c2b0 0x7fdb5406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb740af9b0 0x7fdb740b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740c51f0 con 0x7fdb740af9b0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.655+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb740af9b0 0x7fdb740b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7fdb74071330 con 0x7fdb740af9b0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb6bfff640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdb5406c2b0 0x7fdb5406e770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb74157070 tx=0x7fdb5c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb740af9b0 0x7fdb740b2aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001f90 tx=0x7fdb6c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c041070 con 0x7fdb740af9b0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdb6c003e30 con 0x7fdb740af9b0 2024-09-19T05:42:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb740af9b0 msgr2=0x7fdb740b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb740af9b0 0x7fdb740b2aa0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001f90 tx=0x7fdb6c001650 comp rx=0 tx=0).stop 2024-09-19T05:42:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb740af9b0 msgr2=0x7fdb740b2aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:42:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.656+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb740af9b0 0x7fdb740b2aa0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001f90 tx=0x7fdb6c001650 comp rx=0 tx=0).stop 2024-09-19T05:42:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.662+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7414bff0 0x7fdb740b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.662+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74156810 con 0x7fdb7414bff0 2024-09-19T05:42:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.663+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7414bff0 0x7fdb740b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.663+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fdb740708d0 con 0x7fdb7414bff0 2024-09-19T05:42:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.663+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7414bff0 0x7fdb740b2fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001fc0 tx=0x7fdb6c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.663+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c047220 con 0x7fdb7414bff0 2024-09-19T05:42:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.664+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.0 v2:172.21.15.2:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fdb6c03d450 con 0x7fdb7414bff0 2024-09-19T05:42:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.664+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7414bff0 msgr2=0x7fdb740b2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.664+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7414bff0 0x7fdb740b2fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001fc0 tx=0x7fdb6c02f000 comp rx=0 tx=0).stop 2024-09-19T05:42:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.664+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7414bff0 msgr2=0x7fdb740b2fe0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:42:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.664+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb7414bff0 0x7fdb740b2fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.666 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a: {} 2024-09-19T05:42:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T05:42:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.665+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb741f3f50 0x7fdb741f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.665+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb740756a0 con 0x7fdb741f3f50 2024-09-19T05:42:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.665+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb741f3f50 0x7fdb741f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.665+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7fdb740ae890 con 0x7fdb741f3f50 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.665+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb741f3f50 0x7fdb741f4360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c007f40 tx=0x7fdb6c009030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.666+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0472e0 con 0x7fdb741f3f50 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.666+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.1 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdb6c046620 con 0x7fdb741f3f50 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.666+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb741f3f50 msgr2=0x7fdb741f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.666+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb741f3f50 0x7fdb741f4360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c007f40 tx=0x7fdb6c009030 comp rx=0 tx=0).stop 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.666+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb741f3f50 msgr2=0x7fdb741f4360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.666+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb741f3f50 0x7fdb741f4360 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c007f40 tx=0x7fdb6c009030 comp rx=0 tx=0).stop 2024-09-19T05:42:48.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.666+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 reap_dead start 2024-09-19T05:42:48.671 INFO:tasks.ceph.mds.f.smithi123.stderr:2024-09-19T05:42:48.669+0000 7f4f04a0a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T05:42:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.672+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb740b2560 0x7fdb741f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.672+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741f4de0 con 0x7fdb740b2560 2024-09-19T05:42:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.672+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb740b2560 0x7fdb741f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.673+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fdb74151fd0 con 0x7fdb740b2560 2024-09-19T05:42:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.673+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb740b2560 0x7fdb741f48a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001690 tx=0x7fdb6c04e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.673+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0472e0 con 0x7fdb740b2560 2024-09-19T05:42:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.1 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fdb6c0622b0 con 0x7fdb740b2560 2024-09-19T05:42:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb740b2560 msgr2=0x7fdb741f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb740b2560 0x7fdb741f48a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c001690 tx=0x7fdb6c04e000 comp rx=0 tx=0).stop 2024-09-19T05:42:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb740b2560 msgr2=0x7fdb741f48a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:42:48.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb740b2560 0x7fdb741f48a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.676 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b: {} 2024-09-19T05:42:48.677 INFO:teuthology.orchestra.run.smithi002.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af090 0x7fdb740af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb741f4de0 con 0x7fdb740af090 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af090 0x7fdb740af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fdb7406cf30 con 0x7fdb740af090 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.675+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af090 0x7fdb740af470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c047cc0 tx=0x7fdb6c04d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.676+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c062810 con 0x7fdb740af090 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.676+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.2 v2:172.21.15.123:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdb6c003c40 con 0x7fdb740af090 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.676+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af090 msgr2=0x7fdb740af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.676+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af090 0x7fdb740af470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c047cc0 tx=0x7fdb6c04d000 comp rx=0 tx=0).stop 2024-09-19T05:42:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.676+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af090 msgr2=0x7fdb740af470 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:42:48.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.676+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb740af090 0x7fdb740af470 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.682+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb7414a820 0x7fdb7414ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.682+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb741f4de0 con 0x7fdb7414a820 2024-09-19T05:42:48.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.682+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb7414a820 0x7fdb7414ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.683+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7fdb740b0030 con 0x7fdb7414a820 2024-09-19T05:42:48.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.683+0000 7fdb78cf6640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb7414a820 0x7fdb7414ac00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0050a0 tx=0x7fdb6c06d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.683+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c0810a0 con 0x7fdb7414a820 2024-09-19T05:42:48.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.684+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 <== mon.2 v2:172.21.15.123:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fdb6c03e0f0 con 0x7fdb7414a820 2024-09-19T05:42:48.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.684+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb7414a820 msgr2=0x7fdb7414ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.684+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb7414a820 0x7fdb7414ac00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0050a0 tx=0x7fdb6c06d000 comp rx=0 tx=0).stop 2024-09-19T05:42:48.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb717fa640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb7414a820 msgr2=0x7fdb7414ac00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:42:48.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb717fa640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb7414a820 0x7fdb7414ac00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.686 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c: {} 2024-09-19T05:42:48.687 INFO:teuthology.orchestra.run.smithi002.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T05:42:48.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdb5406c2b0 msgr2=0x7fdb5406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdb5406c2b0 0x7fdb5406e770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb74157070 tx=0x7fdb5c009040 comp rx=0 tx=0).stop 2024-09-19T05:42:48.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 msgr2=0x7fdb74151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 --2- 172.21.15.2:0/748766825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb600051c0 0x7fdb74151a90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb64011c00 tx=0x7fdb6400f610 comp rx=0 tx=0).stop 2024-09-19T05:42:48.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb78cf6640 1 -- 172.21.15.2:0/748766825 reap_dead start 2024-09-19T05:42:48.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 shutdown_connections 2024-09-19T05:42:48.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 >> 172.21.15.2:0/748766825 conn(0x7fdb74067950 msgr2=0x7fdb740b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:48.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 shutdown_connections 2024-09-19T05:42:48.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.685+0000 7fdb7a759640 1 -- 172.21.15.2:0/748766825 wait complete. 2024-09-19T05:42:48.697 DEBUG:teuthology.orchestra.run.smithi123:> type ceph-objectstore-tool 2024-09-19T05:42:48.715 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-19T05:42:48.716 DEBUG:teuthology.orchestra.run.smithi002:> type ceph-objectstore-tool 2024-09-19T05:42:48.731 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-19T05:42:48.731 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T05:42:48.744 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:42:48.744 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T05:42:48.745 INFO:tasks.workunit:timeout=3h 2024-09-19T05:42:48.745 INFO:tasks.workunit:cleanup=True 2024-09-19T05:42:48.745 DEBUG:teuthology.orchestra.run.smithi002:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:48.745 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-19T05:42:48.745 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:42:48.746 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:42:48.746 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-19T05:42:48.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-19T05:42:48.746 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-19T05:42:48.746 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-19T05:42:48.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:42:48.747 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-19T05:42:48.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:42:48.747 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-19T05:42:48.748 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:42:48.757 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:48.755+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 29h/41d Inode: 1 Links: 2 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-19 05:42:37.972076082 +0000 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-19 05:42:46.585211840 +0000 2024-09-19T05:42:48.791 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-19 05:42:37.972076082 +0000 2024-09-19T05:42:48.792 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T05:42:48.792 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T05:42:48.874 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:48.873+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:48.877 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T05:42:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.882+0000 7f8def3b7640 1 Processor -- start 2024-09-19T05:42:48.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.882+0000 7f8def3b7640 1 -- start start 2024-09-19T05:42:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f70ca00a640 1 Processor -- start 2024-09-19T05:42:48.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f70ca00a640 1 -- start start 2024-09-19T05:42:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8def3b7640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8de8154b30 0x7f8de806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8def3b7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8de80c9bc0 0x7f8de806dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8def3b7640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de80c95f0 con 0x7f8de80cbc00 2024-09-19T05:42:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8def3b7640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8de80ca220 con 0x7f8de80c9bc0 2024-09-19T05:42:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8def3b7640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8de80ca500 con 0x7f8de8154b30 2024-09-19T05:42:48.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8dec952640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8de80cbc00 0x7f8de815e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41194/0 (socket says 172.21.15.2:41194) 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8dec952640 1 -- 172.21.15.2:0/2136190359 learned_addr learned my addr 172.21.15.2:0/2136190359 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.883+0000 7f8ded954640 1 --2- 172.21.15.2:0/2136190359 >> v2:172.21.15.2:3300/0 conn(0x7f8de80c9bc0 0x7f8de806dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f8ded153640 1 --2- 172.21.15.2:0/2136190359 >> v2:172.21.15.123:3300/0 conn(0x7f8de8154b30 0x7f8de806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f8ddffff640 1 -- 172.21.15.2:0/2136190359 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342266031 0 0) 0x7f8de80c95f0 con 0x7f8de80cbc00 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f8ded954640 1 -- 172.21.15.2:0/2136190359 >> v2:172.21.15.123:3300/0 conn(0x7f8de8154b30 msgr2=0x7f8de806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f8ded954640 1 --2- 172.21.15.2:0/2136190359 >> v2:172.21.15.123:3300/0 conn(0x7f8de8154b30 0x7f8de806d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f8ded954640 1 -- 172.21.15.2:0/2136190359 >> v1:172.21.15.2:6789/0 conn(0x7f8de80cbc00 legacy=0x7f8de815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f8ded954640 1 -- 172.21.15.2:0/2136190359 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de80ca7e0 con 0x7f8de80c9bc0 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70ca00a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 0x7f70c41572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70c2ffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f70c406d370 0x7f70c4156be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41210/0 (socket says 172.21.15.2:41210) 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70c2ffd640 1 -- 172.21.15.2:0/4106914669 learned_addr learned my addr 172.21.15.2:0/4106914669 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c4072db0 con 0x7f70c406d370 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c40739a0 con 0x7f70c406cf90 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70c4073750 con 0x7f70c41586e0 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70c3fff640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 0x7f70c41572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.884+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 480519895 0 0) 0x7f70c4072db0 con 0x7f70c406d370 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70ac003520 con 0x7f70c406d370 2024-09-19T05:42:48.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266761974 0 0) 0x7f70c40739a0 con 0x7f70c406cf90 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70c4072db0 con 0x7f70c406cf90 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3091185466 0 0) 0x7f70ac003520 con 0x7f70c406d370 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 msgr2=0x7f70c41572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 0x7f70c41572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 >> v1:172.21.15.123:6789/0 conn(0x7f70c406cf90 legacy=0x7f70c4157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c4073c80 con 0x7f70c406d370 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f70b8001670 con 0x7f70c406d370 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 >> v1:172.21.15.2:6789/0 conn(0x7f70c406d370 legacy=0x7f70c4156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70ac003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 0x7f70ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70ac004d40 0x7f70ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70c4073750 con 0x7f70ac003b70 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac00dc80 con 0x7f70ac004d40 2024-09-19T05:42:48.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70ac00de80 con 0x7f70c41586e0 2024-09-19T05:42:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c3fff640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70ac004d40 0x7f70ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c37fe640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70ac003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c2ffd640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 0x7f70ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f8ded954640 1 --2- 172.21.15.2:0/2136190359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de80c9bc0 0x7f8de806dc20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8de80cf550 tx=0x7f8de002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9654264103e8b788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c37fe640 1 -- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 msgr2=0x7f70ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c37fe640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c41586e0 0x7f70ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c37fe640 1 -- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70ac004d40 msgr2=0x7f70ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.885+0000 7f70c37fe640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70ac004d40 0x7f70ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f70c37fe640 1 -- 172.21.15.2:0/4106914669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac003520 con 0x7f70ac003b70 2024-09-19T05:42:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f8ddffff640 1 -- 172.21.15.2:0/2136190359 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0007740 con 0x7f8de80c9bc0 2024-09-19T05:42:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f8ddffff640 1 -- 172.21.15.2:0/2136190359 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de00078e0 con 0x7f8de80c9bc0 2024-09-19T05:42:48.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f8ddffff640 1 -- 172.21.15.2:0/2136190359 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0007cb0 con 0x7f8de80c9bc0 2024-09-19T05:42:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f70c37fe640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70ac003f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70b40018d0 tx=0x7f70ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d255873090fd062 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b40014f0 con 0x7f70ac003b70 2024-09-19T05:42:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b40052d0 con 0x7f70ac003b70 2024-09-19T05:42:48.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.886+0000 7f70c27fc640 1 -- 172.21.15.2:0/4106914669 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b402aca0 con 0x7f70ac003b70 2024-09-19T05:42:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 -- 172.21.15.2:0/2136190359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de80c9bc0 msgr2=0x7f8de806dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 --2- 172.21.15.2:0/2136190359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de80c9bc0 0x7f8de806dc20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8de80cf550 tx=0x7f8de002a5f0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 -- 172.21.15.2:0/2136190359 shutdown_connections 2024-09-19T05:42:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 --2- 172.21.15.2:0/2136190359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de80c9bc0 0x7f8de806dc20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 --2- 172.21.15.2:0/2136190359 >> v2:172.21.15.123:3300/0 conn(0x7f8de8154b30 0x7f8de806d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 -- 172.21.15.2:0/2136190359 >> 172.21.15.2:0/2136190359 conn(0x7f8de80707f0 msgr2=0x7f8de8070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 -- 172.21.15.2:0/2136190359 shutdown_connections 2024-09-19T05:42:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f8def3b7640 1 -- 172.21.15.2:0/2136190359 wait complete. 2024-09-19T05:42:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 msgr2=0x7f70ac003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f70ca00a640 1 --2- 172.21.15.2:0/4106914669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70ac003f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70b40018d0 tx=0x7f70ac005830 comp rx=0 tx=0).stop 2024-09-19T05:42:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f70c3fff640 1 -- 172.21.15.2:0/4106914669 reap_dead start 2024-09-19T05:42:48.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 shutdown_connections 2024-09-19T05:42:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.887+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 >> 172.21.15.2:0/4106914669 conn(0x7f70c40cd470 msgr2=0x7f70c40c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.888+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 shutdown_connections 2024-09-19T05:42:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.888+0000 7f70ca00a640 1 -- 172.21.15.2:0/4106914669 wait complete. 2024-09-19T05:42:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.889+0000 7f8def3b7640 1 Processor -- start 2024-09-19T05:42:48.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.889+0000 7f8def3b7640 1 -- start start 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.889+0000 7f70ca00a640 1 Processor -- start 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.889+0000 7f70ca00a640 1 -- start start 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70ca00a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70c40c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70ca00a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c40c2a70 0x7f70c41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70ca00a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c4152620 0x7f70c41ec110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70ca00a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f70c40c9ff0 con 0x7f70ac003b70 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70ca00a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70c40ca700 con 0x7f70c4152620 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70ca00a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70c40ca170 con 0x7f70c40c2a70 2024-09-19T05:42:48.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70c40c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70c40c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47770/0 (socket says 172.21.15.2:47770) 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70c37fe640 1 -- 172.21.15.2:0/3500375527 learned_addr learned my addr 172.21.15.2:0/3500375527 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70c2ffd640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c40c2a70 0x7f70c41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f70c3fff640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c4152620 0x7f70c41ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f8def3b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de80c9bc0 0x7f8de814c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f8def3b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de80cbc00 0x7f8de8155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f8def3b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de8154b30 0x7f8de814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f8def3b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80c9380 con 0x7f8de8154b30 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f8def3b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80c9200 con 0x7f8de80c9bc0 2024-09-19T05:42:48.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f8def3b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8de815ea50 con 0x7f8de80cbc00 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70c3fff640 1 -- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c40c2a70 msgr2=0x7f70c41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70c3fff640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c40c2a70 0x7f70c41520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70c3fff640 1 -- 172.21.15.2:0/3500375527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 msgr2=0x7f70c40c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70c3fff640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70c40c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70c3fff640 1 -- 172.21.15.2:0/3500375527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c40c2eb0 con 0x7f70c4152620 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:48.899 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:48.900 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:48.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70c3fff640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c4152620 0x7f70c41ec110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e050 tx=0x7f70bc00e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.890+0000 7f8ded153640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de80c9bc0 0x7f8de814c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7fdddf59e640 1 Processor -- start 2024-09-19T05:42:48.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de80c9bc0 0x7f8de814c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:48298/0 (socket says 172.21.15.2:48298) 2024-09-19T05:42:48.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70abfff640 1 -- 172.21.15.2:0/3500375527 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc021070 con 0x7f70c4152620 2024-09-19T05:42:48.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70abfff640 1 -- 172.21.15.2:0/3500375527 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70bc01c410 con 0x7f70c4152620 2024-09-19T05:42:48.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70abfff640 1 -- 172.21.15.2:0/3500375527 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00ee60 con 0x7f70c4152620 2024-09-19T05:42:48.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7fdddf59e640 1 -- start start 2024-09-19T05:42:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 -- 172.21.15.2:0/623844844 learned_addr learned my addr 172.21.15.2:0/623844844 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c40c3d30 con 0x7f70c4152620 2024-09-19T05:42:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8dec952640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de80cbc00 0x7f8de8155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.892+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c4159240 con 0x7f70c4152620 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de8154b30 0x7f8de814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de80cbc00 msgr2=0x7f8de8155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de80cbc00 0x7f8de8155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de8154b30 msgr2=0x7f8de814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de8154b30 0x7f8de814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de814d360 con 0x7f8de80c9bc0 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de8154b30 0x7f8de814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:42:48.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.891+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de80c9bc0 0x7f8de814c9e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e9b0 tx=0x7f8de400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.892+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4017070 con 0x7f8de80c9bc0 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.892+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de400c9a0 con 0x7f8de80c9bc0 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.892+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de400cd90 con 0x7f8de80c9bc0 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.892+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de814d620 con 0x7f8de80c9bc0 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de814bed0 con 0x7f8de80c9bc0 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c4072850 con 0x7f70c4152620 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fdddf59e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fdddf59e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdde00c7db0 0x7fdde015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fdddf59e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdde015b1f0 con 0x7fdde00c8860 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fdddf59e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdde0070220 con 0x7fdde00c7db0 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fdddf59e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdde00c6ff0 con 0x7fdde00c8480 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fddde59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fddded9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fdde00c8860 0x7fdde00c4c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41216/0 (socket says 172.21.15.2:41216) 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fddded9d640 1 -- 172.21.15.2:0/4056340800 learned_addr learned my addr 172.21.15.2:0/4056340800 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.893+0000 7fddddd9b640 1 --2- 172.21.15.2:0/4056340800 >> v2:172.21.15.123:3300/0 conn(0x7fdde00c7db0 0x7fdde015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8de8154f10 con 0x7f8de80c9bc0 2024-09-19T05:42:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7fddde59c640 1 -- 172.21.15.2:0/4056340800 >> v2:172.21.15.123:3300/0 conn(0x7fdde00c7db0 msgr2=0x7fdde015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7fddde59c640 1 --2- 172.21.15.2:0/4056340800 >> v2:172.21.15.123:3300/0 conn(0x7fdde00c7db0 0x7fdde015acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7fddde59c640 1 -- 172.21.15.2:0/4056340800 >> v1:172.21.15.2:6789/0 conn(0x7fdde00c8860 legacy=0x7fdde00c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7fdddd59a640 1 -- 172.21.15.2:0/4056340800 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428022517 0 0) 0x7fdde015b1f0 con 0x7fdde00c8860 2024-09-19T05:42:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7fddde59c640 1 -- 172.21.15.2:0/4056340800 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde00c72d0 con 0x7fdde00c8480 2024-09-19T05:42:48.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7fddddd9b640 1 --2- 172.21.15.2:0/4056340800 >> v2:172.21.15.123:3300/0 conn(0x7fdde00c7db0 0x7fdde015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7f70abfff640 1 -- 172.21.15.2:0/3500375527 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f70bc024070 con 0x7f70c4152620 2024-09-19T05:42:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7f70abfff640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f70b006c330 0x7f70b006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.894+0000 7f70abfff640 1 -- 172.21.15.2:0/3500375527 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f70bc08f360 con 0x7f70c4152620 2024-09-19T05:42:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8de401a070 con 0x7f8de80c9bc0 2024-09-19T05:42:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7fddde59c640 1 --2- 172.21.15.2:0/4056340800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde006fb70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fddd00097a0 tx=0x7fddd002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584da0d93fc37b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7f70c37fe640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f70b006c330 0x7f70b006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7fdddd59a640 1 -- 172.21.15.2:0/4056340800 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0037070 con 0x7fdde00c8480 2024-09-19T05:42:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7fdddd59a640 1 -- 172.21.15.2:0/4056340800 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd0032440 con 0x7fdde00c8480 2024-09-19T05:42:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7fdddd59a640 1 -- 172.21.15.2:0/4056340800 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0032810 con 0x7fdde00c8480 2024-09-19T05:42:48.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7f70c37fe640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f70b006c330 0x7f70b006e7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70b4000f60 tx=0x7f70b4009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7f8dddffb640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c300 0x7f8dcc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.895+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8de408b660 con 0x7f8de80c9bc0 2024-09-19T05:42:48.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7f8dec952640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c300 0x7f8dcc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 -- 172.21.15.2:0/4056340800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 msgr2=0x7fdde006fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 --2- 172.21.15.2:0/4056340800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde006fb70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fddd00097a0 tx=0x7fddd002a230 comp rx=0 tx=0).stop 2024-09-19T05:42:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 -- 172.21.15.2:0/4056340800 shutdown_connections 2024-09-19T05:42:48.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7f8dec952640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c300 0x7f8dcc06e7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8007fd0 tx=0x7f8dd8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 --2- 172.21.15.2:0/4056340800 >> v2:172.21.15.123:3300/0 conn(0x7fdde00c7db0 0x7fdde015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 --2- 172.21.15.2:0/4056340800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde006fb70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 -- 172.21.15.2:0/4056340800 >> 172.21.15.2:0/4056340800 conn(0x7fdde00cc450 msgr2=0x7fdde00cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:48.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 -- 172.21.15.2:0/4056340800 shutdown_connections 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.896+0000 7fdddf59e640 1 -- 172.21.15.2:0/4056340800 wait complete. 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.898+0000 7fdddf59e640 1 Processor -- start 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.898+0000 7fdddf59e640 1 -- start start 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fdddf59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde00c7db0 0x7fdde01f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fdddf59e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde014a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fdddf59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdde00c8860 0x7fdde01edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fdddf59e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00cd090 con 0x7fdde00c8860 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fdddf59e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00c7890 con 0x7fdde00c7db0 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fdddf59e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdde01ee510 con 0x7fdde00c8480 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdde00c8860 0x7fdde01edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdde00c8860 0x7fdde01edfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47798/0 (socket says 172.21.15.2:47798) 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 -- 172.21.15.2:0/106504389 learned_addr learned my addr 172.21.15.2:0/106504389 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddde59c640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde00c7db0 0x7fdde01f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddddd9b640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde014a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v22) ==== 54+0+16 (secure 0 0 0) 0x7f8de4059870 con 0x7f8de80c9bc0 2024-09-19T05:42:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 -- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 msgr2=0x7fdde014a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde014a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 -- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde00c7db0 msgr2=0x7fdde01f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:48.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde00c7db0 0x7fdde01f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:48.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.899+0000 7fddded9d640 1 -- 172.21.15.2:0/106504389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde00cbb50 con 0x7fdde00c8860 2024-09-19T05:42:48.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7f70abfff640 1 -- 172.21.15.2:0/3500375527 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f70bc05d4d0 con 0x7f70c4152620 2024-09-19T05:42:48.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8de81f0c90 0x7f8de80ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7fddded9d640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdde00c8860 0x7fdde01edfd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c7b0 tx=0x7fddd800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8de81546e0 con 0x7f8de81f0c90 2024-09-19T05:42:48.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7fddceffd640 1 -- 172.21.15.2:0/106504389 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8018070 con 0x7fdde00c8860 2024-09-19T05:42:48.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8de81f0c90 0x7f8de80ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7fddceffd640 1 -- 172.21.15.2:0/106504389 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd8004d60 con 0x7fdde00c8860 2024-09-19T05:42:48.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde014ace0 con 0x7fdde00c8860 2024-09-19T05:42:48.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7fddceffd640 1 -- 172.21.15.2:0/106504389 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd800a880 con 0x7fdde00c8860 2024-09-19T05:42:48.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.900+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde014b2a0 con 0x7fdde00c8860 2024-09-19T05:42:48.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.901+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8de81f0c90 0x7f8de80ccce0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.901+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde015acb0 con 0x7fdde00c8860 2024-09-19T05:42:48.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.902+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de81546e0 con 0x7f8de81f0c90 2024-09-19T05:42:48.918 INFO:tasks.workunit.client.0.smithi002.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T05:42:48.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.902+0000 7fddceffd640 1 -- 172.21.15.2:0/106504389 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fddd801b070 con 0x7fdde00c8860 2024-09-19T05:42:48.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.903+0000 7fddceffd640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fddc006c2b0 0x7fddc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.903+0000 7fddceffd640 1 -- 172.21.15.2:0/106504389 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fddd808a960 con 0x7fdde00c8860 2024-09-19T05:42:48.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.903+0000 7fddde59c640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fddc006c2b0 0x7fddc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.920 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:42:48.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.904+0000 7fddde59c640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fddc006c2b0 0x7fddc006e770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fddd0034000 tx=0x7fddd002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.912+0000 7fddceffd640 1 -- 172.21.15.2:0/106504389 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fddd8058950 con 0x7fdde00c8860 2024-09-19T05:42:48.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.917+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de814ba80 con 0x7f8de81f0c90 2024-09-19T05:42:48.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.919+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de814ba80 con 0x7f8de81f0c90 2024-09-19T05:42:48.921 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:42:48.922 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:42:48.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.919+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8de8156240 0x7f8de814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.919+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8de814b5e0 con 0x7f8de8156240 2024-09-19T05:42:48.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.919+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8de8156240 0x7f8de814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.920+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8de8156240 0x7f8de814b0a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.920+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de814b5e0 con 0x7f8de8156240 2024-09-19T05:42:48.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.937+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de814cf20 con 0x7f8de8156240 2024-09-19T05:42:48.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.940+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de814cf20 con 0x7f8de8156240 2024-09-19T05:42:48.942 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:42:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:42:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.941+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8de81f3c40 0x7f8de81f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.941+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8de814d130 con 0x7f8de81f3c40 2024-09-19T05:42:48.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.941+0000 7f8dec952640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8de81f3c40 0x7f8de81f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.942+0000 7f8dec952640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8de81f3c40 0x7f8de81f4060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.942+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de814d130 con 0x7f8de81f3c40 2024-09-19T05:42:48.948 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:48.947+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:48.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.958+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de8062900 con 0x7f8de81f3c40 2024-09-19T05:42:48.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.960+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de8062900 con 0x7f8de81f3c40 2024-09-19T05:42:48.961 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:42:48.961 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:42:48.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.960+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8de8158590 0x7f8de81589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.960+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8de81f45a0 con 0x7f8de8158590 2024-09-19T05:42:48.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.961+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8de8158590 0x7f8de81589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.961+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8de8158590 0x7f8de81589b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.962+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de81f45a0 con 0x7f8de8158590 2024-09-19T05:42:48.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.976+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de81f45a0 con 0x7f8de8158590 2024-09-19T05:42:48.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.979+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de81f45a0 con 0x7f8de8158590 2024-09-19T05:42:48.980 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:42:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:42:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.979+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8de8159400 0x7f8de802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.979+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8de82021f0 con 0x7f8de8159400 2024-09-19T05:42:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.979+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8de8159400 0x7f8de802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.980+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8de8159400 0x7f8de802c640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.981+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de82021f0 con 0x7f8de8159400 2024-09-19T05:42:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.994+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de802cb80 con 0x7f8de8159400 2024-09-19T05:42:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.996+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de802cb80 con 0x7f8de8159400 2024-09-19T05:42:48.998 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:42:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:42:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.997+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8de802d2a0 0x7f8de802d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.997+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8de8205380 con 0x7f8de802d2a0 2024-09-19T05:42:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.997+0000 7f8dec952640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8de802d2a0 0x7f8de802d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.997+0000 7f8dec952640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8de802d2a0 0x7f8de802d700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:48.998+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de8205380 con 0x7f8de802d2a0 2024-09-19T05:42:49.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.012+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de8202af0 con 0x7f8de802d2a0 2024-09-19T05:42:49.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.014+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de8202af0 con 0x7f8de802d2a0 2024-09-19T05:42:49.015 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:42:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:42:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.014+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8de8203170 0x7f8de8205560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.014+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8de8208500 con 0x7f8de8203170 2024-09-19T05:42:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.015+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8de8203170 0x7f8de8205560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:49.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.015+0000 7f8ded954640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8de8203170 0x7f8de8205560 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:49.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.016+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de8208500 con 0x7f8de8203170 2024-09-19T05:42:49.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.030+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de8207250 con 0x7f8de8203170 2024-09-19T05:42:49.032 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.031+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de8207250 con 0x7f8de8203170 2024-09-19T05:42:49.033 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:42:49.033 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:42:49.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.032+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8de8207d30 0x7f8de8205b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:49.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.032+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8de8208190 con 0x7f8de8207d30 2024-09-19T05:42:49.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.032+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8de8207d30 0x7f8de8205b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:49.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.032+0000 7f8ded153640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8de8207d30 0x7f8de8205b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:49.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.033+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8de8208190 con 0x7f8de8207d30 2024-09-19T05:42:49.044 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:49.044 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:49.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.047+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8de8208190 con 0x7f8de8207d30 2024-09-19T05:42:49.049 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:49.048+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:49.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.049+0000 7f8dddffb640 1 -- 172.21.15.2:0/623844844 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8de8208190 con 0x7f8de8207d30 2024-09-19T05:42:49.051 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8de81f0c90 msgr2=0x7f8de80ccce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8de81f0c90 0x7f8de80ccce0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8de8156240 msgr2=0x7f8de814b0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8de8156240 0x7f8de814b0a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8de81f3c40 msgr2=0x7f8de81f4060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8de81f3c40 0x7f8de81f4060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8de8158590 msgr2=0x7f8de81589b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8de8158590 0x7f8de81589b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8de8159400 msgr2=0x7f8de802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8de8159400 0x7f8de802c640 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8de802d2a0 msgr2=0x7f8de802d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8de802d2a0 0x7f8de802d700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8de8203170 msgr2=0x7f8de8205560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8de8203170 0x7f8de8205560 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8de8207d30 msgr2=0x7f8de8205b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8de8207d30 0x7f8de8205b10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8ded954640 1 -- 172.21.15.2:0/623844844 reap_dead start 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8ded954640 1 -- 172.21.15.2:0/623844844 reap_dead start 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c300 msgr2=0x7f8dcc06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c300 0x7f8dcc06e7c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8007fd0 tx=0x7f8dd8009040 comp rx=0 tx=0).stop 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de80c9bc0 msgr2=0x7f8de814c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 --2- 172.21.15.2:0/623844844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8de80c9bc0 0x7f8de814c9e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8de400e9b0 tx=0x7f8de400ee70 comp rx=0 tx=0).stop 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8ded954640 1 -- 172.21.15.2:0/623844844 reap_dead start 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8ded954640 1 -- 172.21.15.2:0/623844844 reap_dead start 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8ded954640 1 -- 172.21.15.2:0/623844844 reap_dead start 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 shutdown_connections 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.050+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 >> 172.21.15.2:0/623844844 conn(0x7f8de80707f0 msgr2=0x7f8de8071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.051+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 shutdown_connections 2024-09-19T05:42:49.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.051+0000 7f8def3b7640 1 -- 172.21.15.2:0/623844844 wait complete. 2024-09-19T05:42:49.057 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:42:49.150 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:49.149+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:49.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.172+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set-full-ratio", "ratio": 0.001} v 0) -- 0x7f70c406be20 con 0x7f70c4152620 2024-09-19T05:42:49.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.183+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fdde0150340 con 0x7fdde00c8860 2024-09-19T05:42:49.214 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:49.215 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:49.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:42:49.251 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:49.249+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:49.299 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-19T05:42:49.298+0000 7fe7b90b1640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-19T05:42:49.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.299+0000 7fddceffd640 1 -- 172.21.15.2:0/106504389 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v23) ==== 87+0+0 (secure 0 0 0) 0x7fddd805cc50 con 0x7fdde00c8860 2024-09-19T05:42:49.300 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:42:49.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fddc006c2b0 msgr2=0x7fddc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fddc006c2b0 0x7fddc006e770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fddd0034000 tx=0x7fddd002e000 comp rx=0 tx=0).stop 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdde00c8860 msgr2=0x7fdde01edfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdde00c8860 0x7fdde01edfd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c7b0 tx=0x7fddd800cc70 comp rx=0 tx=0).stop 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 shutdown_connections 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fddc006c2b0 0x7fddc006e770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdde00c8860 0x7fdde01edfd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdde00c8480 0x7fdde014a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 --2- 172.21.15.2:0/106504389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdde00c7db0 0x7fdde01f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 >> 172.21.15.2:0/106504389 conn(0x7fdde00cc450 msgr2=0x7fdde00c4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 shutdown_connections 2024-09-19T05:42:49.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.302+0000 7fdddf59e640 1 -- 172.21.15.2:0/106504389 wait complete. 2024-09-19T05:42:49.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.303+0000 7f70abfff640 1 -- 172.21.15.2:0/3500375527 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-full-ratio", "ratio": 0.001}]=0 osd set-full-ratio 0.001 v23) ==== 106+0+0 (secure 0 0 0) 0x7f70bc0617d0 con 0x7f70c4152620 2024-09-19T05:42:49.305 INFO:teuthology.orchestra.run.smithi002.stderr:osd set-full-ratio 0.001 2024-09-19T05:42:49.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f70b006c330 msgr2=0x7f70b006e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f70b006c330 0x7f70b006e7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70b4000f60 tx=0x7f70b4009030 comp rx=0 tx=0).stop 2024-09-19T05:42:49.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c4152620 msgr2=0x7f70c41ec110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:49.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c4152620 0x7f70c41ec110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e050 tx=0x7f70bc00e630 comp rx=0 tx=0).stop 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 shutdown_connections 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f70b006c330 0x7f70b006e7f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c4152620 0x7f70c41ec110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70c40c2a70 0x7f70c41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 --2- 172.21.15.2:0/3500375527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70ac003b70 0x7f70c40c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 >> 172.21.15.2:0/3500375527 conn(0x7f70c40cd470 msgr2=0x7f70c40cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 shutdown_connections 2024-09-19T05:42:49.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:49.307+0000 7f70ca00a640 1 -- 172.21.15.2:0/3500375527 wait complete. 2024-09-19T05:42:49.340 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:49.340 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:49.349 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:42:49.352 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:49.350+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:49.453 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:49.452+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:49.498 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:49.498 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:49.507 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:42:49.591 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:49.590+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:49.655 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:49.654+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:49.661 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:49.661 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:49.667 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:42:49.756 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:49.755+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:49.816 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:49.816 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:49.825 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:42:49.870 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:49.869+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:49.933 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:49.933 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:49.941 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:42:49.958 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:49.956+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:50.059 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:50.058+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:50.089 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:50.089 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:50.098 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:42:50.160 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:50.158+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:50.246 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:50.246 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:50.254 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:42:50.298 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:50.297+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:50.323 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-19T05:42:50.323 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-19T05:42:50.362 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:50.360+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:50.362 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:50.362 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:50.371 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:42:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.427+0000 7f5c4ef86640 1 Processor -- start 2024-09-19T05:42:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.427+0000 7f5c4ef86640 1 -- start start 2024-09-19T05:42:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c4ef86640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c4815b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c4ef86640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c480cc930 con 0x7f5c480d08b0 2024-09-19T05:42:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c4ef86640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c480cbd40 con 0x7f5c480d0c90 2024-09-19T05:42:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c4ef86640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5c480cc010 con 0x7f5c480c55d0 2024-09-19T05:42:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c4cd22640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5c480d08b0 0x7f5c480cd990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41226/0 (socket says 172.21.15.2:41226) 2024-09-19T05:42:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c4cd22640 1 -- 172.21.15.2:0/1365345733 learned_addr learned my addr 172.21.15.2:0/1365345733 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c4d523640 1 --2- 172.21.15.2:0/1365345733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c4815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4114974601 0 0) 0x7f5c480cc930 con 0x7f5c480d08b0 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c38003520 con 0x7f5c480d08b0 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1018638524 0 0) 0x7f5c480cbd40 con 0x7f5c480d0c90 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.428+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c480cc930 con 0x7f5c480d0c90 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.429+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1467391709 0 0) 0x7f5c38003520 con 0x7f5c480d08b0 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.429+0000 7f5c4d523640 1 -- 172.21.15.2:0/1365345733 >> v1:172.21.15.123:6789/0 conn(0x7f5c480d0c90 legacy=0x7f5c480cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.429+0000 7f5c4d523640 1 -- 172.21.15.2:0/1365345733 >> v1:172.21.15.2:6789/0 conn(0x7f5c480d08b0 legacy=0x7f5c480cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.429+0000 7f5c4d523640 1 -- 172.21.15.2:0/1365345733 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c480ccbd0 con 0x7f5c480c55d0 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.429+0000 7f5c4d523640 1 --2- 172.21.15.2:0/1365345733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c4815b660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009dc0 tx=0x7f5c4002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff00c6db1c0549f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c001690 con 0x7f5c480c55d0 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4002ae50 con 0x7f5c480c55d0 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c477fe640 1 -- 172.21.15.2:0/1365345733 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40007760 con 0x7f5c480c55d0 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c4ef86640 1 -- 172.21.15.2:0/1365345733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 msgr2=0x7f5c4815b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/1365345733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c4815b660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009dc0 tx=0x7f5c4002a6f0 comp rx=0 tx=0).stop 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c4ef86640 1 -- 172.21.15.2:0/1365345733 shutdown_connections 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/1365345733 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c4815b660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c4ef86640 1 -- 172.21.15.2:0/1365345733 >> 172.21.15.2:0/1365345733 conn(0x7f5c4806d940 msgr2=0x7f5c48073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c4ef86640 1 -- 172.21.15.2:0/1365345733 shutdown_connections 2024-09-19T05:42:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.430+0000 7f5c4ef86640 1 -- 172.21.15.2:0/1365345733 wait complete. 2024-09-19T05:42:50.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.431+0000 7f5c4ef86640 1 Processor -- start 2024-09-19T05:42:50.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.432+0000 7f5c4ef86640 1 -- start start 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.432+0000 7f5c4ef86640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c48148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.432+0000 7f5c4ef86640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c480d08b0 0x7f5c481f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.432+0000 7f5c4ef86640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c480d0c90 0x7f5c4814a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.432+0000 7f5c4ef86640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c480cd470 con 0x7f5c480d0c90 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.432+0000 7f5c4ef86640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c480cd190 con 0x7f5c480d08b0 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.432+0000 7f5c4ef86640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5c48154770 con 0x7f5c480c55d0 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c4d523640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c480d0c90 0x7f5c4814a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c4cd22640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c48148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c4d523640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c480d0c90 0x7f5c4814a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47800/0 (socket says 172.21.15.2:47800) 2024-09-19T05:42:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c4d523640 1 -- 172.21.15.2:0/2007952164 learned_addr learned my addr 172.21.15.2:0/2007952164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c47fff640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c480d08b0 0x7f5c481f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c47fff640 1 -- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 msgr2=0x7f5c48148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c47fff640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c48148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c47fff640 1 -- 172.21.15.2:0/2007952164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c480d0c90 msgr2=0x7f5c4814a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c47fff640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c480d0c90 0x7f5c4814a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c47fff640 1 -- 172.21.15.2:0/2007952164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4814ac20 con 0x7f5c480d08b0 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c4cd22640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c48148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c47fff640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c480d08b0 0x7f5c481f1040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c390 tx=0x7f5c34001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c457fa640 1 -- 172.21.15.2:0/2007952164 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34017070 con 0x7f5c480d08b0 2024-09-19T05:42:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.433+0000 7f5c457fa640 1 -- 172.21.15.2:0/2007952164 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c340052d0 con 0x7f5c480d08b0 2024-09-19T05:42:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.434+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c48147e50 con 0x7f5c480d08b0 2024-09-19T05:42:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.434+0000 7f5c457fa640 1 -- 172.21.15.2:0/2007952164 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c34003830 con 0x7f5c480d08b0 2024-09-19T05:42:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.434+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c48148110 con 0x7f5c480d08b0 2024-09-19T05:42:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.435+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c480cbc70 con 0x7f5c480d08b0 2024-09-19T05:42:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.435+0000 7f5c457fa640 1 -- 172.21.15.2:0/2007952164 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5c3401a070 con 0x7f5c480d08b0 2024-09-19T05:42:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.436+0000 7f5c457fa640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c2c06c2b0 0x7f5c2c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.436+0000 7f5c457fa640 1 -- 172.21.15.2:0/2007952164 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5c3408a5e0 con 0x7f5c480d08b0 2024-09-19T05:42:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.436+0000 7f5c4cd22640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c2c06c2b0 0x7f5c2c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.436+0000 7f5c4cd22640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c2c06c2b0 0x7f5c2c06e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c001800 tx=0x7f5c3c0016d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.441+0000 7f5c457fa640 1 -- 172.21.15.2:0/2007952164 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5c480cbc70 con 0x7f5c480d08b0 2024-09-19T05:42:50.463 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:50.461+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:50.522 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:50.522 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:50.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:42:50.572 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:50.571+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:50.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:50.624+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set-full-ratio", "ratio": 0.95} v 0) -- 0x7f5c4815b660 con 0x7f5c480d08b0 2024-09-19T05:42:50.644 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:50.644 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:50.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:42:50.698 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:50.696+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:50.756 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:50.756 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:50.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:42:50.766 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:50.765+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:50.867 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:50.866+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:50.908 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:50.908 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:50.916 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:42:50.968 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:50.966+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:51.019 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:51.019 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:51.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:42:51.066 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:42:51.069 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:51.067+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:51.169 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:51.169+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:51.170 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:51.170 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.170+0000 7f2c9972c640 1 Processor -- start 2024-09-19T05:42:51.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.170+0000 7f2c9972c640 1 -- start start 2024-09-19T05:42:51.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c9972c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2c9415aa90 0x7f2c9415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c9972c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c9415ebe0 con 0x7f2c940cd8c0 2024-09-19T05:42:51.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c9972c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c940b2e00 con 0x7f2c940cd4e0 2024-09-19T05:42:51.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c9972c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f2c9415eec0 con 0x7f2c9415aa90 2024-09-19T05:42:51.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c927fc640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2c9415aa90 0x7f2c9415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c937fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f2c940cd8c0 0x7f2c940c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:41240/0 (socket says 172.21.15.2:41240) 2024-09-19T05:42:51.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c937fe640 1 -- 172.21.15.2:0/666063229 learned_addr learned my addr 172.21.15.2:0/666063229 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:51.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c927fc640 1 -- 172.21.15.2:0/666063229 >> v1:172.21.15.123:6789/0 conn(0x7f2c940cd4e0 legacy=0x7f2c940b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c927fc640 1 -- 172.21.15.2:0/666063229 >> v1:172.21.15.2:6789/0 conn(0x7f2c940cd8c0 legacy=0x7f2c940c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c91ffb640 1 -- 172.21.15.2:0/666063229 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1895375881 0 0) 0x7f2c940b2e00 con 0x7f2c940cd4e0 2024-09-19T05:42:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.171+0000 7f2c927fc640 1 -- 172.21.15.2:0/666063229 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c940b30e0 con 0x7f2c9415aa90 2024-09-19T05:42:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.172+0000 7f2c927fc640 1 --2- 172.21.15.2:0/666063229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9415df30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c880097a0 tx=0x7f2c8802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85dd39dee751bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.172+0000 7f2c91ffb640 1 -- 172.21.15.2:0/666063229 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88037070 con 0x7f2c9415aa90 2024-09-19T05:42:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.172+0000 7f2c91ffb640 1 -- 172.21.15.2:0/666063229 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8802adc0 con 0x7f2c9415aa90 2024-09-19T05:42:51.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.172+0000 7f2c91ffb640 1 -- 172.21.15.2:0/666063229 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88032660 con 0x7f2c9415aa90 2024-09-19T05:42:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.173+0000 7f2c9972c640 1 -- 172.21.15.2:0/666063229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 msgr2=0x7f2c9415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.173+0000 7f2c9972c640 1 --2- 172.21.15.2:0/666063229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9415df30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2c880097a0 tx=0x7f2c8802a130 comp rx=0 tx=0).stop 2024-09-19T05:42:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.173+0000 7f2c9972c640 1 -- 172.21.15.2:0/666063229 shutdown_connections 2024-09-19T05:42:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.173+0000 7f2c9972c640 1 --2- 172.21.15.2:0/666063229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9415df30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.173+0000 7f2c9972c640 1 -- 172.21.15.2:0/666063229 >> 172.21.15.2:0/666063229 conn(0x7f2c94067b30 msgr2=0x7f2c94067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.173+0000 7f2c9972c640 1 -- 172.21.15.2:0/666063229 shutdown_connections 2024-09-19T05:42:51.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.173+0000 7f2c9972c640 1 -- 172.21.15.2:0/666063229 wait complete. 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.174+0000 7f2c9972c640 1 Processor -- start 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.174+0000 7f2c9972c640 1 -- start start 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c9972c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c940cd4e0 0x7f2c94153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c9972c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c940cd8c0 0x7f2c94154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c9972c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9414e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c9972c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940b42d0 con 0x7f2c9415aa90 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c9972c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c940b36a0 con 0x7f2c940cd4e0 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c9972c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2c940b4450 con 0x7f2c940cd8c0 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9414e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9414e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:47828/0 (socket says 172.21.15.2:47828) 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 -- 172.21.15.2:0/2944858017 learned_addr learned my addr 172.21.15.2:0/2944858017 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c940cd4e0 0x7f2c94153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c927fc640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c940cd8c0 0x7f2c94154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c940cd8c0 msgr2=0x7f2c94154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c940cd8c0 0x7f2c94154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c940cd4e0 msgr2=0x7f2c94153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c940cd4e0 0x7f2c94153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.175+0000 7f2c937fe640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c94152b70 con 0x7f2c9415aa90 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.176+0000 7f2c937fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9414e8c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00caf0 tx=0x7f2c8c002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.176+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c018070 con 0x7f2c9415aa90 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.176+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8c00a6a0 con 0x7f2c9415aa90 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.176+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00aad0 con 0x7f2c9415aa90 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.176+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c94159820 con 0x7f2c9415aa90 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.176+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c94152d60 con 0x7f2c9415aa90 2024-09-19T05:42:51.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.177+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f2c940b3bc0 con 0x7f2c9415aa90 2024-09-19T05:42:51.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.177+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2c8c01b070 con 0x7f2c9415aa90 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.178+0000 7f2c7b7fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c8006c2b0 0x7f2c8006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.178+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2c8c08ab50 con 0x7f2c9415aa90 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.178+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v23) ==== 54+0+16 (secure 0 0 0) 0x7f2c8c08f240 con 0x7f2c9415aa90 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.178+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c8006c2b0 0x7f2c8006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.178+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2c9415db20 0x7f2c94154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.178+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c94068290 con 0x7f2c9415db20 2024-09-19T05:42:51.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.178+0000 7f2c927fc640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2c9415db20 0x7f2c94154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.179+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c8006c2b0 0x7f2c8006e770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000f60 tx=0x7f2c84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.179+0000 7f2c927fc640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2c9415db20 0x7f2c94154c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.179+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c94068290 con 0x7f2c9415db20 2024-09-19T05:42:51.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.194+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c94067fb0 con 0x7f2c9415db20 2024-09-19T05:42:51.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.195+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c94067fb0 con 0x7f2c9415db20 2024-09-19T05:42:51.197 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:42:51.197 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:42:51.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.195+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2c941534b0 0x7f2c941f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.195+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2c940d0100 con 0x7f2c941534b0 2024-09-19T05:42:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.196+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2c941534b0 0x7f2c941f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.196+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2c941534b0 0x7f2c941f3ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.196+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c940d0100 con 0x7f2c941534b0 2024-09-19T05:42:51.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.211+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c941f4270 con 0x7f2c941534b0 2024-09-19T05:42:51.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.213+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c941f4270 con 0x7f2c941534b0 2024-09-19T05:42:51.214 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:42:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:42:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.213+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2c941f0810 0x7f2c941f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.213+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2c941f1130 con 0x7f2c941f0810 2024-09-19T05:42:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.213+0000 7f2c927fc640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2c941f0810 0x7f2c941f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.213+0000 7f2c927fc640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2c941f0810 0x7f2c941f0bf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.214+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c941f1130 con 0x7f2c941f0810 2024-09-19T05:42:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.227+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c94062880 con 0x7f2c941f0810 2024-09-19T05:42:51.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.229+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c94062880 con 0x7f2c941f0810 2024-09-19T05:42:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:42:51.230 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:42:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.229+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2c94063360 0x7f2c9406c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.229+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f2c94062880 con 0x7f2c94063360 2024-09-19T05:42:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.229+0000 7f2c937fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2c94063360 0x7f2c9406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.229+0000 7f2c937fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2c94063360 0x7f2c9406c970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.230+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c94062880 con 0x7f2c94063360 2024-09-19T05:42:51.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.244+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c94062880 con 0x7f2c94063360 2024-09-19T05:42:51.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.245+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c94062880 con 0x7f2c94063360 2024-09-19T05:42:51.246 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:42:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:42:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.245+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2c9402c640 0x7f2c9402ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.245+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f2c94062880 con 0x7f2c9402c640 2024-09-19T05:42:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.246+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2c9402c640 0x7f2c9402ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.246+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2c9402c640 0x7f2c9402ca80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.247+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c94062880 con 0x7f2c9402c640 2024-09-19T05:42:51.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.260+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c94062880 con 0x7f2c9402c640 2024-09-19T05:42:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.262+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c94062880 con 0x7f2c9402c640 2024-09-19T05:42:51.263 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:42:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:42:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.262+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2c9402d4d0 0x7f2c94206c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.262+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f2c9406d850 con 0x7f2c9402d4d0 2024-09-19T05:42:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.262+0000 7f2c927fc640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2c9402d4d0 0x7f2c94206c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.263+0000 7f2c927fc640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2c9402d4d0 0x7f2c94206c00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.263+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c9406d850 con 0x7f2c9402d4d0 2024-09-19T05:42:51.271 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:51.269+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:51.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.277+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c94000950 con 0x7f2c9402d4d0 2024-09-19T05:42:51.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.279+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c94000950 con 0x7f2c9402d4d0 2024-09-19T05:42:51.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:42:51.281 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:42:51.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.279+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2c94207650 0x7f2c94209a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.279+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f2c9420c810 con 0x7f2c94207650 2024-09-19T05:42:51.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.279+0000 7f2c937fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2c94207650 0x7f2c94209a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.280+0000 7f2c937fe640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2c94207650 0x7f2c94209a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.282 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.280+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c9420c810 con 0x7f2c94207650 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.767881146000001, 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:42:51.289 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:42:51.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.767523178000001, 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:42:51.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:42:51.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:42:51.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:42:51.298 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.767022786999998, 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:42:51.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:42:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:42:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:42:51.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.303 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.760036553999999, 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:42:51.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.759578745999999, 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.308 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:42:51.312 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.736865126000001, 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.316 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.736370181000002, 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:42:51.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.321 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.174567225000001, 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:42:51.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:42:51.325 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.173809449, 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:42:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:42:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.173228401999999, 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:42:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:42:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:42:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.172593356, 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:42:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.166786850000001, 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.339 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:42:51.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:42:51.343 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.166130677999998, 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:42:51.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.689582004000002, 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.348 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:42:51.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:42:51.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.689213689999999, 2024-09-19T05:42:51.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:42:51.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:42:51.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.688834780000001, 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:42:51.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.688137927, 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.361 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:42:51.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 26.682353313, 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.370 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.311238298999999, 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:42:51.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.311195216, 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.378 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:51.379 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:42:51.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.294+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c94209f80 con 0x7f2c94207650 2024-09-19T05:42:51.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.296+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c94209f80 con 0x7f2c94207650 2024-09-19T05:42:51.381 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:42:51.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:42:51.382 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:42:51.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.296+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2c9420a600 0x7f2c9420c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.296+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f2c9420f980 con 0x7f2c9420a600 2024-09-19T05:42:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.297+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2c9420a600 0x7f2c9420c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.297+0000 7f2c92ffd640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2c9420a600 0x7f2c9420c9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.298+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c9420f980 con 0x7f2c9420a600 2024-09-19T05:42:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.312+0000 7f5c457fa640 1 -- 172.21.15.2:0/2007952164 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set-full-ratio", "ratio": 0.95}]=0 osd set-full-ratio 0.95 v24) ==== 104+0+0 (secure 0 0 0) 0x7f5c34058790 con 0x7f5c480d08b0 2024-09-19T05:42:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:osd set-full-ratio 0.95 2024-09-19T05:42:51.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.312+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2c9420f950 con 0x7f2c9420a600 2024-09-19T05:42:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c2c06c2b0 msgr2=0x7f5c2c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c2c06c2b0 0x7f5c2c06e770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c001800 tx=0x7f5c3c0016d0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c480d08b0 msgr2=0x7f5c481f1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c480d08b0 0x7f5c481f1040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c390 tx=0x7f5c34001e70 comp rx=0 tx=0).stop 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 shutdown_connections 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c2c06c2b0 0x7f5c2c06e770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c480d08b0 0x7f5c481f1040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c480d0c90 0x7f5c4814a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 --2- 172.21.15.2:0/2007952164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c480c55d0 0x7f5c48148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 >> 172.21.15.2:0/2007952164 conn(0x7f5c4806d940 msgr2=0x7f5c480c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 shutdown_connections 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f5c4ef86640 1 -- 172.21.15.2:0/2007952164 wait complete. 2024-09-19T05:42:51.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.314+0000 7f2c7b7fe640 1 -- 172.21.15.2:0/2944858017 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c9420f950 con 0x7f2c9420a600 2024-09-19T05:42:51.386 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2c9415db20 msgr2=0x7f2c94154c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2c9415db20 0x7f2c94154c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2c941534b0 msgr2=0x7f2c941f3ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2c941534b0 0x7f2c941f3ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2c941f0810 msgr2=0x7f2c941f0bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2c941f0810 0x7f2c941f0bf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2c94063360 msgr2=0x7f2c9406c970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2c94063360 0x7f2c9406c970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c937fe640 1 -- 172.21.15.2:0/2944858017 reap_dead start 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2c9402c640 msgr2=0x7f2c9402ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2c9402c640 0x7f2c9402ca80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c937fe640 1 -- 172.21.15.2:0/2944858017 reap_dead start 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2c9402d4d0 msgr2=0x7f2c94206c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2c9402d4d0 0x7f2c94206c00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2c94207650 msgr2=0x7f2c94209a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2c94207650 0x7f2c94209a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2c9420a600 msgr2=0x7f2c9420c9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2c9420a600 0x7f2c9420c9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c8006c2b0 msgr2=0x7f2c8006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c8006c2b0 0x7f2c8006e770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000f60 tx=0x7f2c84009040 comp rx=0 tx=0).stop 2024-09-19T05:42:51.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 msgr2=0x7f2c9414e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 --2- 172.21.15.2:0/2944858017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c9415aa90 0x7f2c9414e8c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00caf0 tx=0x7f2c8c002080 comp rx=0 tx=0).stop 2024-09-19T05:42:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c937fe640 1 -- 172.21.15.2:0/2944858017 reap_dead start 2024-09-19T05:42:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 shutdown_connections 2024-09-19T05:42:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 >> 172.21.15.2:0/2944858017 conn(0x7f2c94067b30 msgr2=0x7f2c940c8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 shutdown_connections 2024-09-19T05:42:51.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.315+0000 7f2c9972c640 1 -- 172.21.15.2:0/2944858017 wait complete. 2024-09-19T05:42:51.391 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:51.379+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:51.481 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:51.480+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:51.526 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:51.526 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:42:51.526 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:42:51.526 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:51.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.529+0000 7efeb1e3c640 1 Processor -- start 2024-09-19T05:42:51.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.529+0000 7efeb1e3c640 1 -- start start 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.530+0000 7efeb1e3c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efeac0c6de0 0x7efeac0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.530+0000 7efeb1e3c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeac0c85a0 con 0x7efeac152f00 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.530+0000 7efeb1e3c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeac0737c0 con 0x7efeac0c71c0 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.530+0000 7efeb1e3c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efeac073aa0 con 0x7efeac0c6de0 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.530+0000 7efeabfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7efeac152f00 0x7efeac15c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:38980/0 (socket says 172.21.15.2:38980) 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.530+0000 7efeabfff640 1 -- 172.21.15.2:0/2307341927 learned_addr learned my addr 172.21.15.2:0/2307341927 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.530+0000 7efeab7fe640 1 --2- 172.21.15.2:0/2307341927 >> v2:172.21.15.123:3300/0 conn(0x7efeac0c6de0 0x7efeac0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.531+0000 7efeaaffd640 1 -- 172.21.15.2:0/2307341927 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995997524 0 0) 0x7efeac0c85a0 con 0x7efeac152f00 2024-09-19T05:42:51.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.531+0000 7efeaaffd640 1 -- 172.21.15.2:0/2307341927 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe98003520 con 0x7efeac152f00 2024-09-19T05:42:51.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.531+0000 7efeab7fe640 1 -- 172.21.15.2:0/2307341927 >> v1:172.21.15.123:6789/0 conn(0x7efeac0c71c0 legacy=0x7efeac1527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.531+0000 7efeab7fe640 1 -- 172.21.15.2:0/2307341927 >> v1:172.21.15.2:6789/0 conn(0x7efeac152f00 legacy=0x7efeac15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.531+0000 7efeab7fe640 1 -- 172.21.15.2:0/2307341927 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac073d80 con 0x7efeac0c6de0 2024-09-19T05:42:51.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeab7fe640 1 --2- 172.21.15.2:0/2307341927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac0c7e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0097a0 tx=0x7efe9c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=890367e3592788b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeaaffd640 1 -- 172.21.15.2:0/2307341927 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c037070 con 0x7efeac0c6de0 2024-09-19T05:42:51.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeaaffd640 1 -- 172.21.15.2:0/2307341927 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c032440 con 0x7efeac0c6de0 2024-09-19T05:42:51.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeaaffd640 1 -- 172.21.15.2:0/2307341927 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0327f0 con 0x7efeac0c6de0 2024-09-19T05:42:51.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeb1e3c640 1 -- 172.21.15.2:0/2307341927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 msgr2=0x7efeac0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/2307341927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac0c7e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0097a0 tx=0x7efe9c02a230 comp rx=0 tx=0).stop 2024-09-19T05:42:51.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeb1e3c640 1 -- 172.21.15.2:0/2307341927 shutdown_connections 2024-09-19T05:42:51.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/2307341927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac0c7e60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeb1e3c640 1 -- 172.21.15.2:0/2307341927 >> 172.21.15.2:0/2307341927 conn(0x7efeac06f3d0 msgr2=0x7efeac0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:51.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeb1e3c640 1 -- 172.21.15.2:0/2307341927 shutdown_connections 2024-09-19T05:42:51.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.532+0000 7efeb1e3c640 1 -- 172.21.15.2:0/2307341927 wait complete. 2024-09-19T05:42:51.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 Processor -- start 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 -- start start 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac1f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeac0c71c0 0x7efeac1eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeac152f00 0x7efeac15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efeac06dc50 con 0x7efeac0c71c0 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efeac15bf50 con 0x7efeac0c6de0 2024-09-19T05:42:51.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.534+0000 7efeb1e3c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efeac0c8890 con 0x7efeac152f00 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efea2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeac0c71c0 0x7efeac1eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac1f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac1f2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:54216/0 (socket says 172.21.15.2:54216) 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efea2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeac0c71c0 0x7efeac1eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33520/0 (socket says 172.21.15.2:33520) 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 -- 172.21.15.2:0/3979899869 learned_addr learned my addr 172.21.15.2:0/3979899869 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeabfff640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeac152f00 0x7efeac15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 -- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeac152f00 msgr2=0x7efeac15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeac152f00 0x7efeac15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 -- 172.21.15.2:0/3979899869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeac0c71c0 msgr2=0x7efeac1eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeac0c71c0 0x7efeac1eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 -- 172.21.15.2:0/3979899869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac1ef420 con 0x7efeac0c6de0 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efea2ffd640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeac0c71c0 0x7efeac1eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:51.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeab7fe640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac1f2450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe9c035040 tx=0x7efe9c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efea8ff9640 1 -- 172.21.15.2:0/3979899869 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c003e50 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efea8ff9640 1 -- 172.21.15.2:0/3979899869 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c0044c0 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efea8ff9640 1 -- 172.21.15.2:0/3979899869 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c032830 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac145bd0 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.535+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac146190 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.536+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac1463d0 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.537+0000 7efea8ff9640 1 -- 172.21.15.2:0/3979899869 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe9c04c060 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.537+0000 7efea8ff9640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe8c06c450 0x7efe8c06e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.537+0000 7efea8ff9640 1 -- 172.21.15.2:0/3979899869 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5709+0+0 (secure 0 0 0) 0x7efe9c0af390 con 0x7efeac0c6de0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.537+0000 7efea2ffd640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe8c06c450 0x7efe8c06e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:51.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.538+0000 7efea2ffd640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe8c06c450 0x7efe8c06e910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ccad0 tx=0x7efe94009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:51.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.541+0000 7efea8ff9640 1 -- 172.21.15.2:0/3979899869 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7efeac1463d0 con 0x7efeac0c6de0 2024-09-19T05:42:51.582 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:51.581+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:51.642 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:51.642 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:42:51.643 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:42:51.643 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:42:51.643 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:51.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:42:51.683 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:51.682+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:51.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:51.725+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7efeac0c9480 con 0x7efeac0c6de0 2024-09-19T05:42:51.754 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:51.754 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.777452519000001, 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:42:51.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:42:51.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:51.759 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:51.763 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:42:51.807 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:51.805+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:51.867 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:51.867 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:42:51.868 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:42:51.868 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:51.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:42:51.918 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:51.916+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:51.987 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:51.986+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:52.020 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:52.020 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:42:52.020 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:42:52.020 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:42:52.020 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:52.028 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:42:52.088 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:52.086+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.155965120999999, 2024-09-19T05:42:52.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062186530000000002, 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974705+0000", 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1270000000000002e-06 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974716+0000", 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1306e-05 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974736+0000", 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9293e-05 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974749+0000", 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3059e-05 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022932900000000001 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975163+0000", 2024-09-19T05:42:52.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018497500000000001 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975314+0000", 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151328 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979256+0000", 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039412179999999998 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571296 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980877+0000", 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0387999999999999e-05 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980919+0000", 2024-09-19T05:42:52.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1334000000000001e-05 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.155842313000001, 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062599709999999996, 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974824+0000", 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2500000000000004e-07 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974829+0000", 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.66e-06 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974844+0000", 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5160000000000001e-05 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974852+0000", 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9570000000000002e-06 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975748+0000", 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089547100000000001 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975835+0000", 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7190999999999999e-05 2024-09-19T05:42:52.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975997+0000", 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016230800000000001 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979261+0000", 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032635310000000001 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980973+0000", 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017117199999999999 2024-09-19T05:42:52.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981027+0000", 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4732999999999999e-05 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981083+0000", 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5414999999999997e-05 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:30 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.155781782, 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040710980000000004, 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:42:52.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974884+0000", 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6400000000000001e-07 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974899+0000", 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0584000000000001e-05 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974917+0000", 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9799999999999998e-06 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974994+0000", 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6612000000000004e-05 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975176+0000", 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018201400000000001 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975321+0000", 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014440899999999999 2024-09-19T05:42:52.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977845+0000", 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002524409 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978841+0000", 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099560399999999998 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978896+0000", 2024-09-19T05:42:52.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5075999999999999e-05 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978954+0000", 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8743999999999999e-05 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.155713837, 2024-09-19T05:42:52.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040869110000000004, 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:42:52.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974952+0000", 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974957+0000", 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7260000000000002e-06 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974966+0000", 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974972+0000", 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0970000000000002e-06 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975780+0000", 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080748199999999999 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975894+0000", 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113867 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976037+0000", 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014333 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978068+0000", 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020306439999999999 2024-09-19T05:42:52.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978965+0000", 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089710699999999996 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979003+0000", 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8797999999999997e-05 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979038+0000", 2024-09-19T05:42:52.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4668000000000001e-05 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.155682119, 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062404959999999999, 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3900000000000005e-07 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974999+0000", 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5817000000000001e-05 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975006+0000", 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5509999999999996e-06 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975012+0000", 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8189999999999997e-06 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976240+0000", 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012283870000000001 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976367+0000", 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126465 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976487+0000", 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120548 2024-09-19T05:42:52.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980396+0000", 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003908702 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981127+0000", 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073081199999999995 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981182+0000", 2024-09-19T05:42:52.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5921999999999998e-05 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981223+0000", 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0933999999999998e-05 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1e epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:7bf7e7b5:::200.00000006:head by mds.0.4:37 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.154556664999999, 2024-09-19T05:42:52.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028842130000000001, 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:42:52.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976110+0000", 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2869999999999999e-06 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976123+0000", 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3224000000000001e-05 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976140+0000", 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.755e-05 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976157+0000", 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6464e-05 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976275+0000", 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117602 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978993+0000", 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027180860000000002 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.153349787, 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001691115, 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.165 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977316+0000", 2024-09-19T05:42:52.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209999999999999e-06 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977337+0000", 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0890000000000002e-05 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977356+0000", 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032999999999999e-05 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977365+0000", 2024-09-19T05:42:52.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7649999999999999e-06 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977425+0000", 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0044999999999999e-05 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977449+0000", 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4006999999999999e-05 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978959+0000", 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015099379999999999 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979006+0000", 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7315999999999999e-05 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:30 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:42:52.169 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.15292968, 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118336, 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:42:52.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977736+0000", 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.096e-06 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0979999999999999e-06 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977756+0000", 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3891e-05 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977765+0000", 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7579999999999998e-06 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977830+0000", 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5184999999999993e-05 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977839+0000", 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4770000000000003e-06 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977854+0000", 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4831000000000001e-05 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:27 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.152692129, 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101439, 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977974+0000", 2024-09-19T05:42:52.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6899999999999996e-07 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977979+0000", 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1399999999999999e-06 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977993+0000", 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4215e-05 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978001+0000", 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978055+0000", 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3711000000000001e-05 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978062+0000", 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978074+0000", 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2238999999999999e-05 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.15144808, 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024064609999999999, 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979218+0000", 2024-09-19T05:42:52.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979227+0000", 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979239+0000", 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979247+0000", 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8369999999999997e-06 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979307+0000", 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0438e-05 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979326+0000", 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8938999999999999e-05 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981594+0000", 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022679710000000001 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981623+0000", 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9329999999999999e-05 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:42:52.182 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.151147399999999, 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021274990000000001, 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7800000000000001e-07 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979524+0000", 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8019999999999997e-06 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979534+0000", 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8859999999999999e-06 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8189999999999992e-06 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979610+0000", 2024-09-19T05:42:52.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8433e-05 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979641+0000", 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1233999999999999e-05 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981625+0000", 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001983964 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981645+0000", 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9783e-05 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.151086427999999, 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020817100000000001, 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.187 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:42:52.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979598+0000", 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9072e-05 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979609+0000", 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0360000000000001e-05 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979615+0000", 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8700000000000003e-06 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979866+0000", 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000250454 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979884+0000", 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7725000000000001e-05 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981647+0000", 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017632629999999999 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981660+0000", 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3502999999999999e-05 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-19T05:42:52.191 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.150600009, 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016104190000000001, 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980066+0000", 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8400000000000005e-07 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980070+0000", 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8010000000000003e-06 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980077+0000", 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.08e-06 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980082+0000", 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8450000000000002e-06 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980146+0000", 2024-09-19T05:42:52.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3589000000000004e-05 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980166+0000", 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0514e-05 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981661+0000", 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014949539999999999 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981675+0000", 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.150549795, 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018022209999999999, 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.196 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980116+0000", 2024-09-19T05:42:52.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980126+0000", 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980143+0000", 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6407999999999999e-05 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980150+0000", 2024-09-19T05:42:52.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980385+0000", 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023545400000000001 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980403+0000", 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7652999999999999e-05 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981869+0000", 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014661679999999999 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981917+0000", 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319000000000003e-05 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.149950842999999, 2024-09-19T05:42:52.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000222196, 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980715+0000", 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980721+0000", 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2620000000000004e-06 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980735+0000", 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3756e-05 2024-09-19T05:42:52.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980759+0000", 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980813+0000", 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4249999999999997e-05 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980821+0000", 2024-09-19T05:42:52.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7540000000000006e-06 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980936+0000", 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115574 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.149882781000001, 2024-09-19T05:42:52.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030787300000000001, 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980783+0000", 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980786+0000", 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5279999999999999e-06 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980795+0000", 2024-09-19T05:42:52.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0790000000000007e-06 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980802+0000", 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4910000000000002e-06 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980962+0000", 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015999100000000001 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980968+0000", 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.874e-06 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981090+0000", 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012244099999999999 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:29 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.149852875000001, 2024-09-19T05:42:52.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041785999999999999, 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1600000000000002e-07 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980816+0000", 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6660000000000001e-06 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980823+0000", 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6789999999999999e-06 2024-09-19T05:42:52.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6480000000000002e-06 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981115+0000", 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028752699999999999 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:42:52.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981230+0000", 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108546 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.149830244, 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011264039999999999, 2024-09-19T05:42:52.212 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980840+0000", 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1540000000000001e-06 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980848+0000", 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980853+0000", 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9030000000000003e-06 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980907+0000", 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4224999999999997e-05 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980937+0000", 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9852000000000001e-05 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981931+0000", 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099441099999999995 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981961+0000", 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0144000000000001e-05 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.216 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.149312001, 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013984010000000001, 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:42:52.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981354+0000", 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0160000000000001e-06 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:52.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981361+0000", 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981373+0000", 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1439e-05 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981385+0000", 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2041999999999999e-05 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:52.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981446+0000", 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1446999999999996e-05 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981469+0000", 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982708+0000", 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012396359999999999 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:52.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982751+0000", 2024-09-19T05:42:52.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3198000000000002e-05 2024-09-19T05:42:52.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.221 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:52.221 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:52.221 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:52.222 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:52.221+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:52.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:42:52.323 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:52.322+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:52.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.322+0000 7efea8ff9640 1 -- 172.21.15.2:0/3979899869 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v25) ==== 97+0+0 (secure 0 0 0) 0x7efe9c040070 con 0x7efeac0c6de0 2024-09-19T05:42:52.324 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:42:52.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe8c06c450 msgr2=0x7efe8c06e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:52.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe8c06c450 0x7efe8c06e910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ccad0 tx=0x7efe94009040 comp rx=0 tx=0).stop 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 msgr2=0x7efeac1f2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac1f2450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe9c035040 tx=0x7efe9c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 shutdown_connections 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe8c06c450 0x7efe8c06e910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeac152f00 0x7efeac15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeac0c71c0 0x7efeac1eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 --2- 172.21.15.2:0/3979899869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeac0c6de0 0x7efeac1f2450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 >> 172.21.15.2:0/3979899869 conn(0x7efeac06f3d0 msgr2=0x7efeac0c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 shutdown_connections 2024-09-19T05:42:52.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:52.325+0000 7efeb1e3c640 1 -- 172.21.15.2:0/3979899869 wait complete. 2024-09-19T05:42:52.345 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:52.346 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:52.346 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:52.346 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:52.355 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:42:52.424 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:52.422+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:42:52.502 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:42:52.502 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:52.511 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:42:52.525 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:52.523+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:52.620 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:52.620 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:42:52.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:42:52.620 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:42:52.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.642863638, 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029411070000000001, 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974719+0000", 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1449e-05 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974732+0000", 2024-09-19T05:42:52.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3788000000000001e-05 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974743+0000", 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0831e-05 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974751+0000", 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1659999999999994e-06 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974913+0000", 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016199899999999999 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975022+0000", 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108857 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011757200000000001 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976884+0000", 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017442969999999999 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977555+0000", 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067127799999999998 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977604+0000", 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8068999999999997e-05 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977648+0000", 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4801e-05 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.642797120999999, 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034736569999999998, 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:52.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974776+0000", 2024-09-19T05:42:52.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:42:52.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974783+0000", 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2940000000000002e-06 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974786+0000", 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6959999999999998e-06 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975441+0000", 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065484099999999997 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975514+0000", 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2608999999999995e-05 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975640+0000", 2024-09-19T05:42:52.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125869 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977680+0000", 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020402739999999999 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978154+0000", 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047386900000000003 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978206+0000", 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2250000000000003e-05 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978247+0000", 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369000000000003e-05 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.642762858999999, 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034986209999999999, 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8999999999999998e-07 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974818+0000", 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6870000000000006e-06 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974826+0000", 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9639999999999999e-06 2024-09-19T05:42:52.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974869+0000", 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3380999999999997e-05 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975009+0000", 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013971099999999999 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:42:52.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131235 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976896+0000", 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001756482 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978210+0000", 2024-09-19T05:42:52.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013140090000000001 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978275+0000", 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000004e-05 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978307+0000", 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899000000000001e-05 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.642729263, 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035825869999999999, 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974846+0000", 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974856+0000", 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0701e-05 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974862+0000", 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0669999999999997e-06 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976006+0000", 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001143205 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976080+0000", 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4290999999999997e-05 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976213+0000", 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013278500000000001 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977693+0000", 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014800810000000001 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978354+0000", 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000660971 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978385+0000", 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1423000000000001e-05 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978424+0000", 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9131000000000002e-05 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.645 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.642698827, 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037252449999999999, 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974873+0000", 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2200000000000004e-07 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974876+0000", 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.965e-06 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974882+0000", 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3729999999999998e-06 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039999999999995e-06 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974947+0000", 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6505000000000002e-05 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975044+0000", 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6933000000000002e-05 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975162+0000", 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117879 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977622+0000", 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024597519999999999 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978546+0000", 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092433599999999999 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978574+0000", 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7923e-05 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978597+0000", 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3553e-05 2024-09-19T05:42:52.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:30 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.641990445999999, 2024-09-19T05:42:52.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002124711, 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975581+0000", 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0470000000000001e-06 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975592+0000", 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.097e-05 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975603+0000", 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0891e-05 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975611+0000", 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4569999999999999e-06 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975662+0000", 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1424000000000001e-05 2024-09-19T05:42:52.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975687+0000", 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4913999999999999e-05 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977661+0000", 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019738020000000002 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977705+0000", 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4206000000000003e-05 2024-09-19T05:42:52.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:24 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.641208184, 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012912539999999999, 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2799999999999995e-07 2024-09-19T05:42:52.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976370+0000", 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8369999999999999e-06 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976381+0000", 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0744000000000001e-05 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976388+0000", 2024-09-19T05:42:52.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2320000000000004e-06 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976435+0000", 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6726e-05 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976452+0000", 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6629999999999998e-05 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977623+0000", 2024-09-19T05:42:52.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001170894 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977654+0000", 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1362999999999998e-05 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.641018817999999, 2024-09-19T05:42:52.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001404389, 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976553+0000", 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-07 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976558+0000", 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976568+0000", 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976575+0000", 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976621+0000", 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6136e-05 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976640+0000", 2024-09-19T05:42:52.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8604e-05 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977917+0000", 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00127725 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977956+0000", 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9249000000000003e-05 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.640968537999999, 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099816800000000002, 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:42:52.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976603+0000", 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976610+0000", 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0210000000000002e-06 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5128e-05 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976638+0000", 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2995e-05 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976709+0000", 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0659000000000002e-05 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977601+0000", 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089194000000000003 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:25 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.640744271999999, 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001150365, 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:42:52.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976836+0000", 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3049999999999998e-06 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976845+0000", 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4020000000000002e-06 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976850+0000", 2024-09-19T05:42:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6890000000000001e-06 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5437000000000004e-05 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976933+0000", 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640999999999999e-05 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977941+0000", 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001008175 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6264999999999999e-05 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:16 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:42:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.640180068999999, 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000264987, 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:42:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977392+0000", 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3600000000000002e-07 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977396+0000", 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6190000000000002e-06 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977412+0000", 2024-09-19T05:42:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5919000000000001e-05 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977420+0000", 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977543+0000", 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012284899999999999 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977550+0000", 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977656+0000", 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010567499999999999 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:17 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.639499363000001, 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183666, 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978076+0000", 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.019e-06 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978089+0000", 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2828e-05 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4389999999999997e-06 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978141+0000", 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5649999999999998e-05 2024-09-19T05:42:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978149+0000", 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5820000000000004e-06 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978255+0000", 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010655299999999999 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:20 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.639435709000001, 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176971, 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978136+0000", 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2300000000000001e-07 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978139+0000", 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-19T05:42:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978148+0000", 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978155+0000", 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0169999999999999e-06 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978198+0000", 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3251999999999999e-05 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978205+0000", 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2250000000000003e-06 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978312+0000", 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010694800000000001 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:21 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.639401391, 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000262632, 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978173+0000", 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8169999999999999e-06 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978180+0000", 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4839999999999998e-06 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978185+0000", 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978347+0000", 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016165400000000001 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978351+0000", 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1389999999999997e-06 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978432+0000", 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260999999999993e-05 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.689 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.639118437, 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000148286, 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978453+0000", 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978458+0000", 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978472+0000", 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3733e-05 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978479+0000", 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8040000000000004e-06 2024-09-19T05:42:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978539+0000", 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0038999999999999e-05 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978543+0000", 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2699999999999998e-06 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978601+0000", 2024-09-19T05:42:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7881999999999999e-05 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.622742101, 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001029049, 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994831+0000", 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2299999999999998e-06 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994844+0000", 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.291e-05 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994875+0000", 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0898999999999997e-05 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994899+0000", 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3972999999999999e-05 2024-09-19T05:42:52.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994994+0000", 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5068000000000005e-05 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995015+0000", 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1166e-05 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995800+0000", 2024-09-19T05:42:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078543299999999999 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995858+0000", 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000001e-05 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:52.698 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:52.700 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:42:52.700 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:52.699+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:52.800 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:52.800+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:52.848 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:52.848 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:52.848 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:52.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:42:52.941 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:52.940+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:53.002 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:53.001+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:53.004 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:42:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:42:53.004 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:53.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:42:53.103 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:53.102+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:42:53.163 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.280657882, 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:42:53.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:42:53.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:42:53.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.641573074, 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.170 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:42:53.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:42:53.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:42:53.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.641530544999998, 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:42:53.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:42:53.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.181 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.641412985999999, 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:42:53.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.612155650999998, 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:42:53.187 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:42:53.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:42:53.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:42:53.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.612098713000002, 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:42:53.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:42:53.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.458038981000001, 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:42:53.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:42:53.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.457966649999999, 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:42:53.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:42:53.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.457953101000001, 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:42:53.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:42:53.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.0527862, 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:42:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:42:53.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:42:53.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:53.222 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.052707603000002, 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:53.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:42:53.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:53.228 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.052695665000002, 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:53.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:42:53.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:42:53.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.052684585000002, 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:42:53.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:42:53.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.052610019999999, 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:53.245 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.041183480000001, 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:53.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:42:53.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.041121338, 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:42:53.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:42:53.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:42:53.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.563897915999998, 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.257 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:53.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.262 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.563876552, 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:42:53.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:42:53.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:53.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.563858849999999, 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:42:53.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:42:53.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.563805465000002, 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:53.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:53.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:42:53.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:53.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.279 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:53.280 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:53.280 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:53.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:42:53.282 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:53.280+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:53.381 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:53.380+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:53.391 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:42:53.431 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:53.432 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:53.432 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:53.432 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:53.441 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:42:53.482 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:53.481+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:53.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.491+0000 7f03c87c6640 1 Processor -- start 2024-09-19T05:42:53.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.491+0000 7f03c87c6640 1 -- start start 2024-09-19T05:42:53.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c87c6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03c0151250 0x7f03c00c3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c87c6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 0x7f03c00c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c87c6640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c006db70 con 0x7f03c00b2c60 2024-09-19T05:42:53.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c87c6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f03c006c9f0 con 0x7f03c0151250 2024-09-19T05:42:53.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c87c6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03c015b4d0 con 0x7f03c00c25e0 2024-09-19T05:42:53.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c5d61640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f03c00b2c60 0x7f03c015adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:38992/0 (socket says 172.21.15.2:38992) 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c5d61640 1 -- 172.21.15.2:0/300522719 learned_addr learned my addr 172.21.15.2:0/300522719 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c6d63640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 0x7f03c00c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c6562640 1 --2- 172.21.15.2:0/300522719 >> v2:172.21.15.123:3300/0 conn(0x7f03c0151250 0x7f03c00c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3561348158 0 0) 0x7f03c006db70 con 0x7f03c00b2c60 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03ac003520 con 0x7f03c00b2c60 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1372175823 0 0) 0x7f03ac003520 con 0x7f03c00b2c60 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.492+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 msgr2=0x7f03c00c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 0x7f03c00c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 >> v2:172.21.15.123:3300/0 conn(0x7f03c0151250 msgr2=0x7f03c00c3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 --2- 172.21.15.2:0/300522719 >> v2:172.21.15.123:3300/0 conn(0x7f03c0151250 0x7f03c00c3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c006d2d0 con 0x7f03c00b2c60 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f03b0001670 con 0x7f03c00b2c60 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6d63640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 0x7f03c00c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 >> v1:172.21.15.2:6789/0 conn(0x7f03c00b2c60 legacy=0x7f03c015adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ac0039b0 0x7f03ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 0x7f03ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac004d40 0x7f03ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03c015b4d0 con 0x7f03ac0039b0 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac00b9c0 con 0x7f03ac004d40 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03ac00bb40 con 0x7f03c00c25e0 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6562640 1 --2- 172.21.15.2:0/300522719 >> v2:172.21.15.123:3300/0 conn(0x7f03c0151250 0x7f03c00c3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6562640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ac0039b0 0x7f03ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c5d61640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 0x7f03ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6d63640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac004d40 0x7f03ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6562640 1 -- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 msgr2=0x7f03ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6562640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c00c25e0 0x7f03ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6562640 1 -- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac004d40 msgr2=0x7f03ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6562640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac004d40 0x7f03ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.493+0000 7f03c6562640 1 -- 172.21.15.2:0/300522719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac003520 con 0x7f03ac0039b0 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.494+0000 7f03c6562640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ac0039b0 0x7f03ac003dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009c70 tx=0x7f03ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1132f09889d7e8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.494+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc009840 con 0x7f03ac0039b0 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.494+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03bc004020 con 0x7f03ac0039b0 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.494+0000 7f03c5560640 1 -- 172.21.15.2:0/300522719 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0043d0 con 0x7f03ac0039b0 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.495+0000 7f03c87c6640 1 -- 172.21.15.2:0/300522719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ac0039b0 msgr2=0x7f03ac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.495+0000 7f03c87c6640 1 --2- 172.21.15.2:0/300522719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03ac0039b0 0x7f03ac003dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009c70 tx=0x7f03ac005830 comp rx=0 tx=0).stop 2024-09-19T05:42:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.495+0000 7f03c6d63640 1 -- 172.21.15.2:0/300522719 reap_dead start 2024-09-19T05:42:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.495+0000 7f03c87c6640 1 -- 172.21.15.2:0/300522719 shutdown_connections 2024-09-19T05:42:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.495+0000 7f03c87c6640 1 -- 172.21.15.2:0/300522719 >> 172.21.15.2:0/300522719 conn(0x7f03c00c9630 msgr2=0x7f03c00c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.495+0000 7f03c87c6640 1 -- 172.21.15.2:0/300522719 shutdown_connections 2024-09-19T05:42:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.495+0000 7f03c87c6640 1 -- 172.21.15.2:0/300522719 wait complete. 2024-09-19T05:42:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.496+0000 7f03c87c6640 1 Processor -- start 2024-09-19T05:42:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.496+0000 7f03c87c6640 1 -- start start 2024-09-19T05:42:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c87c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac0039b0 0x7f03c0152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c87c6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c014a420 0x7f03c0145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c87c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c014a8a0 0x7f03c0145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c87c6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03c006d890 con 0x7f03c014a8a0 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c87c6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03c006d080 con 0x7f03ac0039b0 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c87c6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03c015a5c0 con 0x7f03c014a420 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c014a8a0 0x7f03c0145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6562640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac0039b0 0x7f03c0152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c014a8a0 0x7f03c0145fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33540/0 (socket says 172.21.15.2:33540) 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 -- 172.21.15.2:0/774960741 learned_addr learned my addr 172.21.15.2:0/774960741 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c5d61640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c014a420 0x7f03c0145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c014a420 msgr2=0x7f03c0145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03c014a420 0x7f03c0145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac0039b0 msgr2=0x7f03c0152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03ac0039b0 0x7f03c0152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c01483d0 con 0x7f03c014a8a0 2024-09-19T05:42:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.497+0000 7f03c6d63640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c014a8a0 0x7f03c0145fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f03c0068290 tx=0x7f03b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.498+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b800fda0 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.498+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b8003990 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.498+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c01480f0 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.498+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8003d80 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.498+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c0151d10 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.498+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f03c0151630 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.499+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f03b8004020 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.500+0000 7f03b6ffd640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f03a406c2e0 0x7f03a406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.500+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5709+0+0 (secure 0 0 0) 0x7f03b808c5f0 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.500+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v25) ==== 54+0+16 (secure 0 0 0) 0x7f03b8019030 con 0x7f03c014a8a0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.500+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f03a406c2e0 0x7f03a406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.500+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03c0151ee0 0x7f03c0153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.500+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03c006c5c0 con 0x7f03c0151ee0 2024-09-19T05:42:53.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.500+0000 7f03c5d61640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03c0151ee0 0x7f03c0153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.501+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f03a406c2e0 0x7f03a406e7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03bc032000 tx=0x7f03bc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.501+0000 7f03c5d61640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03c0151ee0 0x7f03c0153b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.501+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03c006c5c0 con 0x7f03c0151ee0 2024-09-19T05:42:53.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.516+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c00c3120 con 0x7f03c0151ee0 2024-09-19T05:42:53.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.517+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c00c3120 con 0x7f03c0151ee0 2024-09-19T05:42:53.519 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:42:53.519 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:42:53.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.518+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03c01522c0 0x7f03c01f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.518+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f03c00c8270 con 0x7f03c01522c0 2024-09-19T05:42:53.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.518+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03c01522c0 0x7f03c01f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.518+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03c01522c0 0x7f03c01f19f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.519+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03c00c8270 con 0x7f03c01522c0 2024-09-19T05:42:53.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.533+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c00c7f90 con 0x7f03c01522c0 2024-09-19T05:42:53.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.535+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c00c7f90 con 0x7f03c01522c0 2024-09-19T05:42:53.537 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:42:53.537 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:42:53.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.535+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03c01f2440 0x7f03c0062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.535+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f03c01f2840 con 0x7f03c01f2440 2024-09-19T05:42:53.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.536+0000 7f03c5d61640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03c01f2440 0x7f03c0062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.536+0000 7f03c5d61640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03c01f2440 0x7f03c0062630 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.537+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03b0004c20 con 0x7f03c01f2440 2024-09-19T05:42:53.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.550+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c0062dc0 con 0x7f03c01f2440 2024-09-19T05:42:53.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.552+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c0062dc0 con 0x7f03c01f2440 2024-09-19T05:42:53.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:42:53.554 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:42:53.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.552+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f03c0063390 0x7f03c014ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.552+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f03c0154750 con 0x7f03c0063390 2024-09-19T05:42:53.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.553+0000 7f03c6d63640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f03c0063390 0x7f03c014ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.553+0000 7f03c6d63640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f03c0063390 0x7f03c014ef80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.553+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03c0154750 con 0x7f03c0063390 2024-09-19T05:42:53.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.568+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c014f790 con 0x7f03c0063390 2024-09-19T05:42:53.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.570+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c014f790 con 0x7f03c0063390 2024-09-19T05:42:53.571 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:42:53.572 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:42:53.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.570+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f03c002cb50 0x7f03c002cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.570+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f03c002d4d0 con 0x7f03c002cb50 2024-09-19T05:42:53.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.570+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f03c002cb50 0x7f03c002cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.571+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f03c002cb50 0x7f03c002cf90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.572+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03c002d4d0 con 0x7f03c002cb50 2024-09-19T05:42:53.583 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:53.582+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:53.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.586+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c002d4d0 con 0x7f03c002cb50 2024-09-19T05:42:53.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.588+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c002d4d0 con 0x7f03c002cb50 2024-09-19T05:42:53.589 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:42:53.590 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:42:53.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.588+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f03c0202d70 0x7f03c0205160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.588+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f03c002d9e0 con 0x7f03c0202d70 2024-09-19T05:42:53.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.589+0000 7f03c5d61640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f03c0202d70 0x7f03c0205160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.589+0000 7f03c5d61640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f03c0202d70 0x7f03c0205160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.590+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03c002d9e0 con 0x7f03c0202d70 2024-09-19T05:42:53.593 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:53.593 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:53.602 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:42:53.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.604+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c002d9e0 con 0x7f03c0202d70 2024-09-19T05:42:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.606+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c002d9e0 con 0x7f03c0202d70 2024-09-19T05:42:53.608 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:42:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:42:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.607+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f03c0205bb0 0x7f03c0207fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.607+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f03c020ad70 con 0x7f03c0205bb0 2024-09-19T05:42:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.607+0000 7f03c6d63640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f03c0205bb0 0x7f03c0207fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.608+0000 7f03c6d63640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f03c0205bb0 0x7f03c0207fa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.608+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03c020ad70 con 0x7f03c0205bb0 2024-09-19T05:42:53.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.624+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c02087b0 con 0x7f03c0205bb0 2024-09-19T05:42:53.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.626+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c02087b0 con 0x7f03c0205bb0 2024-09-19T05:42:53.627 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:42:53.628 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:42:53.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.626+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f03c0208ed0 0x7f03c0209310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:53.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.626+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f03c020e1c0 con 0x7f03c0208ed0 2024-09-19T05:42:53.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.627+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f03c0208ed0 0x7f03c0209310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:53.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.627+0000 7f03c6562640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f03c0208ed0 0x7f03c0209310 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:53.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.628+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03c020e1c0 con 0x7f03c0208ed0 2024-09-19T05:42:53.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.642+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f03c020d040 con 0x7f03c0208ed0 2024-09-19T05:42:53.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03b6ffd640 1 -- 172.21.15.2:0/774960741 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03c020d040 con 0x7f03c0208ed0 2024-09-19T05:42:53.645 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03c0151ee0 msgr2=0x7f03c0153b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03c0151ee0 0x7f03c0153b90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03c01522c0 msgr2=0x7f03c01f19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03c01522c0 0x7f03c01f19f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03c01f2440 msgr2=0x7f03c0062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03c01f2440 0x7f03c0062630 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f03c0063390 msgr2=0x7f03c014ef80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f03c0063390 0x7f03c014ef80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f03c002cb50 msgr2=0x7f03c002cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f03c002cb50 0x7f03c002cf90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c6d63640 1 -- 172.21.15.2:0/774960741 reap_dead start 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c6d63640 1 -- 172.21.15.2:0/774960741 reap_dead start 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.644+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f03c0202d70 msgr2=0x7f03c0205160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f03c0202d70 0x7f03c0205160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f03c0205bb0 msgr2=0x7f03c0207fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f03c0205bb0 0x7f03c0207fa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f03c0208ed0 msgr2=0x7f03c0209310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f03c0208ed0 0x7f03c0209310 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f03a406c2e0 msgr2=0x7f03a406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f03a406c2e0 0x7f03a406e7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03bc032000 tx=0x7f03bc008030 comp rx=0 tx=0).stop 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c014a8a0 msgr2=0x7f03c0145fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 --2- 172.21.15.2:0/774960741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03c014a8a0 0x7f03c0145fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f03c0068290 tx=0x7f03b8009f90 comp rx=0 tx=0).stop 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c6d63640 1 -- 172.21.15.2:0/774960741 reap_dead start 2024-09-19T05:42:53.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 shutdown_connections 2024-09-19T05:42:53.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 >> 172.21.15.2:0/774960741 conn(0x7f03c00c9630 msgr2=0x7f03c00c3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:53.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 shutdown_connections 2024-09-19T05:42:53.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:53.645+0000 7f03c87c6640 1 -- 172.21.15.2:0/774960741 wait complete. 2024-09-19T05:42:53.698 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:53.697+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:53.714 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:53.714 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:53.723 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:42:53.785 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:53.783+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:53.871 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:53.871 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:53.880 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:42:53.923 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:53.922+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:53.987 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:53.987 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:53.987 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:53.986+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:53.995 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:42:54.088 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:54.086+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:54.142 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:54.143 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:54.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:42:54.196 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:54.194+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:54.267 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:54.267 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:54.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:42:54.289 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:54.288+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:54.341 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:42:54.389 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:54.388+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:54.421 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:54.421 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:54.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:42:54.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.443+0000 7f427e28b640 1 Processor -- start 2024-09-19T05:42:54.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.443+0000 7f427e28b640 1 -- start start 2024-09-19T05:42:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.444+0000 7f427e28b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f427806c040 0x7f427815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.444+0000 7f427e28b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42780c8ee0 0x7f427815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.444+0000 7f427e28b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42780739e0 con 0x7f4278154d60 2024-09-19T05:42:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.444+0000 7f427e28b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f427815ed70 con 0x7f427806c040 2024-09-19T05:42:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.444+0000 7f427e28b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f427815e160 con 0x7f42780c8ee0 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f42777fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f427806c040 0x7f427815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f4277fff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4278154d60 0x7f42780c8060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:46026/0 (socket says 172.21.15.2:46026) 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f4277fff640 1 -- 172.21.15.2:0/1018506006 learned_addr learned my addr 172.21.15.2:0/1018506006 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f427c828640 1 --2- 172.21.15.2:0/1018506006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42780c8ee0 0x7f427815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f4276ffd640 1 -- 172.21.15.2:0/1018506006 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170886787 0 0) 0x7f42780739e0 con 0x7f4278154d60 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f4276ffd640 1 -- 172.21.15.2:0/1018506006 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4260003520 con 0x7f4278154d60 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f427c828640 1 -- 172.21.15.2:0/1018506006 >> v2:172.21.15.123:3300/0 conn(0x7f427806c040 msgr2=0x7f427815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f427c828640 1 --2- 172.21.15.2:0/1018506006 >> v2:172.21.15.123:3300/0 conn(0x7f427806c040 0x7f427815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f427c828640 1 -- 172.21.15.2:0/1018506006 >> v1:172.21.15.123:6789/0 conn(0x7f4278154d60 legacy=0x7f42780c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f427c828640 1 -- 172.21.15.2:0/1018506006 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427815e440 con 0x7f42780c8ee0 2024-09-19T05:42:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.445+0000 7f42777fe640 1 --2- 172.21.15.2:0/1018506006 >> v2:172.21.15.123:3300/0 conn(0x7f427806c040 0x7f427815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427c828640 1 --2- 172.21.15.2:0/1018506006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42780c8ee0 0x7f427815b0b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f427002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7586cda7f6ce9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f4276ffd640 1 -- 172.21.15.2:0/1018506006 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270007740 con 0x7f42780c8ee0 2024-09-19T05:42:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f4276ffd640 1 -- 172.21.15.2:0/1018506006 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42700078e0 con 0x7f42780c8ee0 2024-09-19T05:42:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f4276ffd640 1 -- 172.21.15.2:0/1018506006 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270007c90 con 0x7f42780c8ee0 2024-09-19T05:42:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 -- 172.21.15.2:0/1018506006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42780c8ee0 msgr2=0x7f427815b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:54.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 --2- 172.21.15.2:0/1018506006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42780c8ee0 0x7f427815b0b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4270009d90 tx=0x7f427002a660 comp rx=0 tx=0).stop 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 -- 172.21.15.2:0/1018506006 shutdown_connections 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 --2- 172.21.15.2:0/1018506006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42780c8ee0 0x7f427815b0b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 --2- 172.21.15.2:0/1018506006 >> v2:172.21.15.123:3300/0 conn(0x7f427806c040 0x7f427815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 -- 172.21.15.2:0/1018506006 >> 172.21.15.2:0/1018506006 conn(0x7f427806f3d0 msgr2=0x7f42780707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 -- 172.21.15.2:0/1018506006 shutdown_connections 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.446+0000 7f427e28b640 1 -- 172.21.15.2:0/1018506006 wait complete. 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.448+0000 7f427e28b640 1 Processor -- start 2024-09-19T05:42:54.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.448+0000 7f427e28b640 1 -- start start 2024-09-19T05:42:54.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427e28b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f427806c040 0x7f427814e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:54.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427e28b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42780c8ee0 0x7f4278152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:54.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427e28b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4278154d60 0x7f427814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:54.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427e28b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42780c8b80 con 0x7f4278154d60 2024-09-19T05:42:54.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427e28b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f427815ea00 con 0x7f42780c8ee0 2024-09-19T05:42:54.452 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427e28b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f42780c8d00 con 0x7f427806c040 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f427806c040 0x7f427814e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427c828640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4278154d60 0x7f427814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f42777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42780c8ee0 0x7f4278152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f427806c040 0x7f427814e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:55462/0 (socket says 172.21.15.2:55462) 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 -- 172.21.15.2:0/569412221 learned_addr learned my addr 172.21.15.2:0/569412221 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 -- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42780c8ee0 msgr2=0x7f4278152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42780c8ee0 0x7f4278152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 -- 172.21.15.2:0/569412221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4278154d60 msgr2=0x7f427814eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4278154d60 0x7f427814eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 -- 172.21.15.2:0/569412221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42781f4020 con 0x7f427806c040 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4277fff640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f427806c040 0x7f427814e520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f426800c3c0 tx=0x7f426800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4274ff9640 1 -- 172.21.15.2:0/569412221 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268017070 con 0x7f427806c040 2024-09-19T05:42:54.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4274ff9640 1 -- 172.21.15.2:0/569412221 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4268005ce0 con 0x7f427806c040 2024-09-19T05:42:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f4274ff9640 1 -- 172.21.15.2:0/569412221 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680072f0 con 0x7f427806c040 2024-09-19T05:42:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.449+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42781f4300 con 0x7f427806c040 2024-09-19T05:42:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.450+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781f48c0 con 0x7f427806c040 2024-09-19T05:42:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.450+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42780735c0 con 0x7f427806c040 2024-09-19T05:42:54.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.451+0000 7f4274ff9640 1 -- 172.21.15.2:0/569412221 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f426801a070 con 0x7f427806c040 2024-09-19T05:42:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.452+0000 7f4274ff9640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f426406c2b0 0x7f426406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.452+0000 7f4274ff9640 1 -- 172.21.15.2:0/569412221 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 5709+0+0 (secure 0 0 0) 0x7f426808a910 con 0x7f427806c040 2024-09-19T05:42:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.452+0000 7f42777fe640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f426406c2b0 0x7f426406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:54.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.452+0000 7f42777fe640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f426406c2b0 0x7f426406e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f426c009560 tx=0x7f426c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:54.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.456+0000 7f4274ff9640 1 -- 172.21.15.2:0/569412221 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f4268058900 con 0x7f427806c040 2024-09-19T05:42:54.490 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:54.489+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:54.533 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:54.533 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:54.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:42:54.590 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:54.589+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:54.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:54.640+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f42780c4610 con 0x7f427806c040 2024-09-19T05:42:54.645 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:54.645 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:54.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:42:54.697 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:54.696+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:54.756 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:54.757 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:54.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:42:54.793 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:54.792+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:54.893 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:54.892+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:54.910 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:54.910 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:54.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:42:54.994 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:54.993+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:55.021 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:55.021 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:55.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:42:55.095 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:55.094+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:55.173 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:55.173 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:55.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:42:55.230 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:55.229+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:55.289 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:55.290 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:55.297 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:55.296+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:55.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:42:55.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.329+0000 7f4274ff9640 1 -- 172.21.15.2:0/569412221 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v26) ==== 91+0+0 (secure 0 0 0) 0x7f426805cc00 con 0x7f427806c040 2024-09-19T05:42:55.330 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f426406c2b0 msgr2=0x7f426406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f426406c2b0 0x7f426406e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f426c009560 tx=0x7f426c009190 comp rx=0 tx=0).stop 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f427806c040 msgr2=0x7f427814e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f427806c040 0x7f427814e520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f426800c3c0 tx=0x7f426800c880 comp rx=0 tx=0).stop 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 shutdown_connections 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f426406c2b0 0x7f426406e770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42780c8ee0 0x7f4278152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4278154d60 0x7f427814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 --2- 172.21.15.2:0/569412221 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f427806c040 0x7f427814e520 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 >> 172.21.15.2:0/569412221 conn(0x7f427806f3d0 msgr2=0x7f427815d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 shutdown_connections 2024-09-19T05:42:55.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.332+0000 7f427e28b640 1 -- 172.21.15.2:0/569412221 wait complete. 2024-09-19T05:42:55.397 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:55.396+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:55.441 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:55.441 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:55.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:42:55.498 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:55.497+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:55.593 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:55.593 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:55.598 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:55.597+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:55.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:42:55.657 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:42:55.702 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:55.701+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:55.745 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:55.745 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:55.754 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:42:55.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.762+0000 7f03458b7640 1 Processor -- start 2024-09-19T05:42:55.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.762+0000 7f03458b7640 1 -- start start 2024-09-19T05:42:55.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f03458b7640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03400711e0 0x7f03401526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f03458b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340152be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f03458b7640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03400c7fa0 con 0x7f0340070e00 2024-09-19T05:42:55.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f03458b7640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f034006bcb0 con 0x7f03400711e0 2024-09-19T05:42:55.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f03458b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f034006bf90 con 0x7f03400702e0 2024-09-19T05:42:55.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f033effd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f0340070e00 0x7f034006fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:38994/0 (socket says 172.21.15.2:38994) 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f033effd640 1 -- 172.21.15.2:0/4105720866 learned_addr learned my addr 172.21.15.2:0/4105720866 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f033e7fc640 1 --2- 172.21.15.2:0/4105720866 >> v2:172.21.15.123:3300/0 conn(0x7f03400711e0 0x7f03401526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.763+0000 7f033f7fe640 1 --2- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.764+0000 7f033dffb640 1 -- 172.21.15.2:0/4105720866 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982602022 0 0) 0x7f03400c7fa0 con 0x7f0340070e00 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.764+0000 7f033dffb640 1 -- 172.21.15.2:0/4105720866 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0328003520 con 0x7f0340070e00 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.764+0000 7f033e7fc640 1 -- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 msgr2=0x7f0340152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.764+0000 7f033e7fc640 1 --2- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.764+0000 7f033e7fc640 1 -- 172.21.15.2:0/4105720866 >> v1:172.21.15.2:6789/0 conn(0x7f0340070e00 legacy=0x7f034006fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.764+0000 7f033e7fc640 1 -- 172.21.15.2:0/4105720866 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034006c270 con 0x7f03400711e0 2024-09-19T05:42:55.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.764+0000 7f033f7fe640 1 --2- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:55.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f033e7fc640 1 --2- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400711e0 0x7f03401526a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f03340097a0 tx=0x7f033402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85ad5601d71ff598 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f033dffb640 1 -- 172.21.15.2:0/4105720866 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334037070 con 0x7f03400711e0 2024-09-19T05:42:55.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f033dffb640 1 -- 172.21.15.2:0/4105720866 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0334032440 con 0x7f03400711e0 2024-09-19T05:42:55.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f033dffb640 1 -- 172.21.15.2:0/4105720866 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334032810 con 0x7f03400711e0 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 -- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400711e0 msgr2=0x7f03401526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 --2- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400711e0 0x7f03401526a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f03340097a0 tx=0x7f033402a180 comp rx=0 tx=0).stop 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 -- 172.21.15.2:0/4105720866 shutdown_connections 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 --2- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 --2- 172.21.15.2:0/4105720866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03400711e0 0x7f03401526a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 -- 172.21.15.2:0/4105720866 >> 172.21.15.2:0/4105720866 conn(0x7f0340072a40 msgr2=0x7f0340072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 -- 172.21.15.2:0/4105720866 shutdown_connections 2024-09-19T05:42:55.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.765+0000 7f03458b7640 1 -- 172.21.15.2:0/4105720866 wait complete. 2024-09-19T05:42:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 Processor -- start 2024-09-19T05:42:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 -- start start 2024-09-19T05:42:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340145040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0340070e00 0x7f03401f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03400711e0 0x7f03401f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f03400c9950 con 0x7f03400711e0 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03400c8280 con 0x7f0340070e00 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.767+0000 7f03458b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03400c9ad0 con 0x7f03400702e0 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03400711e0 0x7f03401f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03400711e0 0x7f03401f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33566/0 (socket says 172.21.15.2:33566) 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 -- 172.21.15.2:0/1363541203 learned_addr learned my addr 172.21.15.2:0/1363541203 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340145040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0340070e00 0x7f03401f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 msgr2=0x7f0340145040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03400702e0 0x7f0340145040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0340070e00 msgr2=0x7f03401f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0340070e00 0x7f03401f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03400cd850 con 0x7f03400711e0 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0340070e00 0x7f03401f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f033f7fe640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03400711e0 0x7f03401f3d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f033800efc0 tx=0x7f0338009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338018030 con 0x7f03400711e0 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033800cde0 con 0x7f03400711e0 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03400c5b30 con 0x7f03400711e0 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.768+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338003bb0 con 0x7f03400711e0 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.769+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03400c5180 con 0x7f03400711e0 2024-09-19T05:42:55.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.769+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f0340157a80 con 0x7f03400711e0 2024-09-19T05:42:55.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.771+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f033801b070 con 0x7f03400711e0 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.771+0000 7f0327fff640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f032c06c2e0 0x7f032c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.771+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5709+0+0 (secure 0 0 0) 0x7f033808bcd0 con 0x7f03400711e0 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.771+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v26) ==== 54+0+16 (secure 0 0 0) 0x7f0340157a80 con 0x7f03400711e0 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.771+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f032c06c2e0 0x7f032c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.771+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03400c6560 0x7f034014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.771+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03400c7b80 con 0x7f03400c6560 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.772+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03400c6560 0x7f034014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.772+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f032c06c2e0 0x7f032c06e7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0330005880 tx=0x7f0330009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.772+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03400c6560 0x7f034014e980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.773+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03400c7b80 con 0x7f03400c6560 2024-09-19T05:42:55.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.786+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f03400c5d00 con 0x7f03400c6560 2024-09-19T05:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.788+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03400c5d00 con 0x7f03400c6560 2024-09-19T05:42:55.790 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.788+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03401e5620 0x7f03401f0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.788+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f0340150130 con 0x7f03401e5620 2024-09-19T05:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.788+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03401e5620 0x7f03401f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.789+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03401e5620 0x7f03401f0250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.789+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0340150130 con 0x7f03401e5620 2024-09-19T05:42:55.801 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:55.799+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.804+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0340156740 con 0x7f03401e5620 2024-09-19T05:42:55.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.806+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0340156740 con 0x7f03401e5620 2024-09-19T05:42:55.808 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:42:55.808 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:42:55.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.806+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03401f0e10 0x7f0340062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.806+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f0340062b70 con 0x7f03401f0e10 2024-09-19T05:42:55.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.806+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03401f0e10 0x7f0340062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.806+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03401f0e10 0x7f0340062630 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.807+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0340062b70 con 0x7f03401f0e10 2024-09-19T05:42:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.821+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0340063000 con 0x7f03401f0e10 2024-09-19T05:42:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.823+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0340063000 con 0x7f03401f0e10 2024-09-19T05:42:55.826 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:42:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:42:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.824+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f0340150e40 0x7f0340151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.824+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f0340063000 con 0x7f0340150e40 2024-09-19T05:42:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.824+0000 7f033f7fe640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f0340150e40 0x7f0340151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.824+0000 7f033f7fe640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f0340150e40 0x7f0340151280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.825+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0340063000 con 0x7f0340150e40 2024-09-19T05:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.838+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0340063000 con 0x7f0340150e40 2024-09-19T05:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.840+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0340063000 con 0x7f0340150e40 2024-09-19T05:42:55.842 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:42:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:42:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.840+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f034002cb50 0x7f034002cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.840+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f03400635d0 con 0x7f034002cb50 2024-09-19T05:42:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.841+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f034002cb50 0x7f034002cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.841+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f034002cb50 0x7f034002cf50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.842+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03400635d0 con 0x7f034002cb50 2024-09-19T05:42:55.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.855+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f03400635d0 con 0x7f034002cb50 2024-09-19T05:42:55.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.857+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f03400635d0 con 0x7f034002cb50 2024-09-19T05:42:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:42:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:42:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.857+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f0340204b40 0x7f0340206f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.857+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f034002d9a0 con 0x7f0340204b40 2024-09-19T05:42:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.858+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f0340204b40 0x7f0340206f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.858+0000 7f033e7fc640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f0340204b40 0x7f0340206f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.859+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f034002d9a0 con 0x7f0340204b40 2024-09-19T05:42:55.867 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:55.867 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:55.867 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:55.867 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:55.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.873+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f034002d9a0 con 0x7f0340204b40 2024-09-19T05:42:55.877 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:42:55.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.875+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f034002d9a0 con 0x7f0340204b40 2024-09-19T05:42:55.878 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:42:55.878 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:42:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.875+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f0340207980 0x7f0340209d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.875+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f034002db10 con 0x7f0340207980 2024-09-19T05:42:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.875+0000 7f033f7fe640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f0340207980 0x7f0340209d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.876+0000 7f033f7fe640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f0340207980 0x7f0340209d70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.877+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f034002db10 con 0x7f0340207980 2024-09-19T05:42:55.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.890+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f034002db10 con 0x7f0340207980 2024-09-19T05:42:55.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.892+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f034002db10 con 0x7f0340207980 2024-09-19T05:42:55.894 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:42:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:42:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.892+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f034020a7c0 0x7f034020cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.892+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f034020f980 con 0x7f034020a7c0 2024-09-19T05:42:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.893+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f034020a7c0 0x7f034020cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:55.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.893+0000 7f033effd640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f034020a7c0 0x7f034020cbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:55.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.894+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f034020f980 con 0x7f034020a7c0 2024-09-19T05:42:55.902 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:55.901+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:55.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.908+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f034020d0f0 con 0x7f034020a7c0 2024-09-19T05:42:55.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f0327fff640 1 -- 172.21.15.2:0/1363541203 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f034020d0f0 con 0x7f034020a7c0 2024-09-19T05:42:55.912 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:42:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:42:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03400c6560 msgr2=0x7f034014e980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f03400c6560 0x7f034014e980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03401e5620 msgr2=0x7f03401f0250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f03401e5620 0x7f03401f0250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03401f0e10 msgr2=0x7f0340062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f03401f0e10 0x7f0340062630 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f0340150e40 msgr2=0x7f0340151280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f0340150e40 0x7f0340151280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f034002cb50 msgr2=0x7f034002cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f033f7fe640 1 -- 172.21.15.2:0/1363541203 reap_dead start 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f034002cb50 0x7f034002cf50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f033f7fe640 1 -- 172.21.15.2:0/1363541203 reap_dead start 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f0340204b40 msgr2=0x7f0340206f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f0340204b40 0x7f0340206f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f0340207980 msgr2=0x7f0340209d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f0340207980 0x7f0340209d70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f034020a7c0 msgr2=0x7f034020cbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f034020a7c0 0x7f034020cbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f032c06c2e0 msgr2=0x7f032c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f032c06c2e0 0x7f032c06e7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0330005880 tx=0x7f0330009040 comp rx=0 tx=0).stop 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03400711e0 msgr2=0x7f03401f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03400711e0 0x7f03401f3d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f033800efc0 tx=0x7f0338009f90 comp rx=0 tx=0).stop 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f033f7fe640 1 -- 172.21.15.2:0/1363541203 reap_dead start 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 shutdown_connections 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 --2- 172.21.15.2:0/1363541203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03400711e0 0x7f03401f3d00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.910+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 >> 172.21.15.2:0/1363541203 conn(0x7f0340072a40 msgr2=0x7f0340156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.911+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 shutdown_connections 2024-09-19T05:42:55.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:55.911+0000 7f03458b7640 1 -- 172.21.15.2:0/1363541203 wait complete. 2024-09-19T05:42:55.985 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:55.985 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:55.985 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:42:55.986 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:42:55.986 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:55.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:42:56.038 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:56.036+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:56.102 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.125988213999999, 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029411070000000001, 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.103 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974719+0000", 2024-09-19T05:42:56.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1449e-05 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974732+0000", 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3788000000000001e-05 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974743+0000", 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0831e-05 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974751+0000", 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1659999999999994e-06 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974913+0000", 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016199899999999999 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975022+0000", 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108857 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011757200000000001 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976884+0000", 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017442969999999999 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977555+0000", 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067127799999999998 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977604+0000", 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8068999999999997e-05 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977648+0000", 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4801e-05 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.125921696999999, 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034736569999999998, 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974776+0000", 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974783+0000", 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2940000000000002e-06 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974786+0000", 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6959999999999998e-06 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975441+0000", 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065484099999999997 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975514+0000", 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2608999999999995e-05 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975640+0000", 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125869 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977680+0000", 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020402739999999999 2024-09-19T05:42:56.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978154+0000", 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047386900000000003 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978206+0000", 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2250000000000003e-05 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978247+0000", 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369000000000003e-05 2024-09-19T05:42:56.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.125887434999999, 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034986209999999999, 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8999999999999998e-07 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974818+0000", 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6870000000000006e-06 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974826+0000", 2024-09-19T05:42:56.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9639999999999999e-06 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974869+0000", 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3380999999999997e-05 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975009+0000", 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013971099999999999 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:56.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131235 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976896+0000", 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001756482 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978210+0000", 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013140090000000001 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978275+0000", 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000004e-05 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978307+0000", 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899000000000001e-05 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:56.119 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.125853839000001, 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035825869999999999, 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:56.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974846+0000", 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974856+0000", 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0701e-05 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974862+0000", 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0669999999999997e-06 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976006+0000", 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001143205 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976080+0000", 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4290999999999997e-05 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976213+0000", 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013278500000000001 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977693+0000", 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014800810000000001 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978354+0000", 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000660971 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978385+0000", 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1423000000000001e-05 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978424+0000", 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9131000000000002e-05 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.125823402999998, 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037252449999999999, 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:42:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974873+0000", 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2200000000000004e-07 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974876+0000", 2024-09-19T05:42:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.965e-06 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974882+0000", 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3729999999999998e-06 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:42:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039999999999995e-06 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974947+0000", 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6505000000000002e-05 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975044+0000", 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6933000000000002e-05 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975162+0000", 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117879 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977622+0000", 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024597519999999999 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978546+0000", 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092433599999999999 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978574+0000", 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7923e-05 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978597+0000", 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3553e-05 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:30 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.125115021999999, 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002124711, 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975581+0000", 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0470000000000001e-06 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975592+0000", 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.097e-05 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975603+0000", 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0891e-05 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975611+0000", 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4569999999999999e-06 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975662+0000", 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1424000000000001e-05 2024-09-19T05:42:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975687+0000", 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4913999999999999e-05 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977661+0000", 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019738020000000002 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977705+0000", 2024-09-19T05:42:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4206000000000003e-05 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:24 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.124332760000001, 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012912539999999999, 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2799999999999995e-07 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976370+0000", 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8369999999999999e-06 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976381+0000", 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0744000000000001e-05 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976388+0000", 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2320000000000004e-06 2024-09-19T05:42:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976435+0000", 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6726e-05 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976452+0000", 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6629999999999998e-05 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977623+0000", 2024-09-19T05:42:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001170894 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977654+0000", 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1362999999999998e-05 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.124143394000001, 2024-09-19T05:42:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001404389, 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976553+0000", 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-07 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976558+0000", 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976568+0000", 2024-09-19T05:42:56.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976575+0000", 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976621+0000", 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6136e-05 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976640+0000", 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8604e-05 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977917+0000", 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00127725 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977956+0000", 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9249000000000003e-05 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.124093114000001, 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099816800000000002, 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976603+0000", 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976610+0000", 2024-09-19T05:42:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0210000000000002e-06 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5128e-05 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976638+0000", 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2995e-05 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976709+0000", 2024-09-19T05:42:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0659000000000002e-05 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977601+0000", 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089194000000000003 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:25 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.123868848000001, 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001150365, 2024-09-19T05:42:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976836+0000", 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3049999999999998e-06 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976845+0000", 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4020000000000002e-06 2024-09-19T05:42:56.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976850+0000", 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6890000000000001e-06 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5437000000000004e-05 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976933+0000", 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640999999999999e-05 2024-09-19T05:42:56.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977941+0000", 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001008175 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6264999999999999e-05 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:16 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.123304645000001, 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000264987, 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977392+0000", 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3600000000000002e-07 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977396+0000", 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6190000000000002e-06 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977412+0000", 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5919000000000001e-05 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977420+0000", 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977543+0000", 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012284899999999999 2024-09-19T05:42:56.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977550+0000", 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977656+0000", 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010567499999999999 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:17 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.122623939, 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183666, 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978076+0000", 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.019e-06 2024-09-19T05:42:56.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978089+0000", 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2828e-05 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4389999999999997e-06 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978141+0000", 2024-09-19T05:42:56.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5649999999999998e-05 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978149+0000", 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5820000000000004e-06 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978255+0000", 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010655299999999999 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:20 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.122560284999999, 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176971, 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978136+0000", 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2300000000000001e-07 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978139+0000", 2024-09-19T05:42:56.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978148+0000", 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978155+0000", 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0169999999999999e-06 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978198+0000", 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3251999999999999e-05 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978205+0000", 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2250000000000003e-06 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978312+0000", 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010694800000000001 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:21 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.122525967000001, 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000262632, 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:56.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978173+0000", 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8169999999999999e-06 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978180+0000", 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4839999999999998e-06 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978185+0000", 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978347+0000", 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016165400000000001 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978351+0000", 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1389999999999997e-06 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978432+0000", 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260999999999993e-05 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.167 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.122243012999999, 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000148286, 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978453+0000", 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978458+0000", 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978472+0000", 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3733e-05 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978479+0000", 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8040000000000004e-06 2024-09-19T05:42:56.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978539+0000", 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0038999999999999e-05 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978543+0000", 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2699999999999998e-06 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978601+0000", 2024-09-19T05:42:56.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7881999999999999e-05 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.105866677, 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001029049, 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.172 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994831+0000", 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2299999999999998e-06 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994844+0000", 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.291e-05 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994875+0000", 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0898999999999997e-05 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994899+0000", 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3972999999999999e-05 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994994+0000", 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5068000000000005e-05 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995015+0000", 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1166e-05 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995800+0000", 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078543299999999999 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995858+0000", 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000001e-05 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.176 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:56.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:42:56.178 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:56.176+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:56.279 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:56.277+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:56.325 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:56.326 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:56.335 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:42:56.380 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:56.378+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:56.391 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:42:56.391 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:42:56.391 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-19T05:42:56.391 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T05:42:56.392 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T05:42:56.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-19T05:42:56.480 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:56.479+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:56.483 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:56.483 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:56.483 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:42:56.483 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:42:56.483 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:56.493 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:42:56.526 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:56.526 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T05:42:56.526 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:56.581 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:56.579+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:42:56.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.753100299000003, 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.114015491000004, 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:42:56.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:42:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:42:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:42:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.113972961999998, 2024-09-19T05:42:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:42:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.113855403000002, 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:42:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.084598067999998, 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:42:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.084541129999998, 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:42:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.930481397999998, 2024-09-19T05:42:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:42:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.930409066999999, 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:42:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:42:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.930395517999997, 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:42:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:42:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.525228617000003, 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:42:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.525150019999998, 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:42:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.525138081999998, 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:42:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:42:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:42:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.525127001999998, 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:42:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:42:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.525052436999999, 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:42:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:42:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.513625896999997, 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:42:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:42:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:42:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:42:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.513563755, 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:42:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:42:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:42:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.036340332999998, 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:42:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:42:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.036318969, 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:42:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:42:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.036301266999999, 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:42:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:42:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:42:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:42:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.036247881999998, 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:42:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:42:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:42:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:56.752 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:56.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:42:56.753 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:56.751+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:56.854 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:56.852+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:56.901 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:56.902 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:56.910 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:42:56.955 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:56.953+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:57.019 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:42:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:42:57.019 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:57.028 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:42:57.073 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:57.071+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.159557422999999, 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:42:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:42:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:42:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.15949986, 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:42:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:42:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:42:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:42:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.159427384000001, 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:42:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.159390126000002, 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.159350323999998, 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.158521163, 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:42:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:42:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.158096214, 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:42:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:42:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.153934212999999, 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013154900000000001, 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3099999999999997e-07 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980460+0000", 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6990000000000001e-06 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980470+0000", 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9420000000000007e-06 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980476+0000", 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0419999999999999e-06 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980510+0000", 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4498000000000001e-05 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980515+0000", 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6539999999999998e-06 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980587+0000", 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2082999999999994e-05 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.153903226000001, 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:42:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:42:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:1 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.153629854999998, 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103356, 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980763+0000", 2024-09-19T05:42:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6249999999999999e-06 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980770+0000", 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2880000000000004e-06 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980774+0000", 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6780000000000002e-06 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980803+0000", 2024-09-19T05:42:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9512000000000001e-05 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980807+0000", 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3610000000000001e-06 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980863+0000", 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6535e-05 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.153323093000001, 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010034, 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981070+0000", 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3730000000000001e-06 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981077+0000", 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6370000000000002e-06 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981080+0000", 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3419999999999999e-06 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981107+0000", 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6806000000000001e-05 2024-09-19T05:42:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981111+0000", 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981167+0000", 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6246999999999999e-05 2024-09-19T05:42:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.151321314, 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:42:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.150909553999998, 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:42:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:42:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:38 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.146256729000001, 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186071, 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988134+0000", 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0999999999999998e-07 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988137+0000", 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3919999999999999e-06 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988147+0000", 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659999999999996e-06 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988151+0000", 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.797e-06 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988194+0000", 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2911000000000003e-05 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988202+0000", 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988319+0000", 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117355 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.580526143, 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:42:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:42:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:40 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.578787825999999, 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178677, 2024-09-19T05:42:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555603+0000", 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0699999999999996e-07 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555608+0000", 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2240000000000003e-06 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555620+0000", 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.151e-05 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555626+0000", 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8699999999999997e-06 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555665+0000", 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9356000000000001e-05 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555673+0000", 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4230000000000005e-06 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555781+0000", 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010738699999999999 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.2638591889999997, 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:42:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.2617737790000003, 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.219 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:57.221 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:42:57.221 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:57.219+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:57.322 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:57.321+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:57.347 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:42:57.371 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:42:57.372 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:57.381 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:42:57.422 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:57.421+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.451+0000 7f71f7a5e640 1 Processor -- start 2024-09-19T05:42:57.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.451+0000 7f71f7a5e640 1 -- start start 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f7a5e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71f00cb530 0x7f71f01548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f7a5e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f71f00cb910 0x7f71f0154df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f7a5e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f015af60 con 0x7f71f015aab0 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f7a5e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f71f015b240 con 0x7f71f00cb910 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f7a5e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f71f015b520 con 0x7f71f00cb530 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f5ffb640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f71f00cb910 0x7f71f0154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f5ffb640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f71f00cb910 0x7f71f0154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33568/0 (socket says 172.21.15.2:33568) 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f5ffb640 1 -- 172.21.15.2:0/332774921 learned_addr learned my addr 172.21.15.2:0/332774921 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f57fa640 1 --2- 172.21.15.2:0/332774921 >> v2:172.21.15.123:3300/0 conn(0x7f71f00cb530 0x7f71f01548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f5ffb640 1 -- 172.21.15.2:0/332774921 >> v2:172.21.15.123:3300/0 conn(0x7f71f00cb530 msgr2=0x7f71f01548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f5ffb640 1 --2- 172.21.15.2:0/332774921 >> v2:172.21.15.123:3300/0 conn(0x7f71f00cb530 0x7f71f01548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f5ffb640 1 -- 172.21.15.2:0/332774921 >> v1:172.21.15.123:6789/0 conn(0x7f71f015aab0 legacy=0x7f71f00ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71f5ffb640 1 -- 172.21.15.2:0/332774921 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f015b800 con 0x7f71f00cb910 2024-09-19T05:42:57.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.452+0000 7f71e7fff640 1 -- 172.21.15.2:0/332774921 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099551627 0 0) 0x7f71f015af60 con 0x7f71f015aab0 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.454+0000 7f71f5ffb640 1 --2- 172.21.15.2:0/332774921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0154df0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009e50 tx=0x7f71e802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f2c9505f32bbf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.454+0000 7f71e7fff640 1 -- 172.21.15.2:0/332774921 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8036070 con 0x7f71f00cb910 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.454+0000 7f71e7fff640 1 -- 172.21.15.2:0/332774921 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e8031600 con 0x7f71f00cb910 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.454+0000 7f71e7fff640 1 -- 172.21.15.2:0/332774921 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80319d0 con 0x7f71f00cb910 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.454+0000 7f71f7a5e640 1 -- 172.21.15.2:0/332774921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 msgr2=0x7f71f0154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.454+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/332774921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0154df0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009e50 tx=0x7f71e802a760 comp rx=0 tx=0).stop 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.455+0000 7f71f7a5e640 1 -- 172.21.15.2:0/332774921 shutdown_connections 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.455+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/332774921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0154df0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:57.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.455+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/332774921 >> v2:172.21.15.123:3300/0 conn(0x7f71f00cb530 0x7f71f01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.455+0000 7f71f7a5e640 1 -- 172.21.15.2:0/332774921 >> 172.21.15.2:0/332774921 conn(0x7f71f0067950 msgr2=0x7f71f0067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.455+0000 7f71f7a5e640 1 -- 172.21.15.2:0/332774921 shutdown_connections 2024-09-19T05:42:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.455+0000 7f71f7a5e640 1 -- 172.21.15.2:0/332774921 wait complete. 2024-09-19T05:42:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.456+0000 7f71f7a5e640 1 Processor -- start 2024-09-19T05:42:57.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.457+0000 7f71f7a5e640 1 -- start start 2024-09-19T05:42:57.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f7a5e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71f00cb530 0x7f71f015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:57.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f7a5e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:57.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f7a5e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f015aab0 0x7f71f01f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:57.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f7a5e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f71f00cd850 con 0x7f71f00cb910 2024-09-19T05:42:57.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f7a5e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71f00cd6d0 con 0x7f71f015aab0 2024-09-19T05:42:57.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f7a5e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f71f006cfe0 con 0x7f71f00cb530 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33574/0 (socket says 172.21.15.2:33574) 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 -- 172.21.15.2:0/1895114675 learned_addr learned my addr 172.21.15.2:0/1895114675 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f57fa640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71f00cb530 0x7f71f015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f5ffb640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f015aab0 0x7f71f01f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 -- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71f00cb530 msgr2=0x7f71f015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71f00cb530 0x7f71f015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 -- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f015aab0 msgr2=0x7f71f01f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f015aab0 0x7f71f01f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 -- 172.21.15.2:0/1895114675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f015eea0 con 0x7f71f00cb910 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.458+0000 7f71f4ff9640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0156430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c410 tx=0x7f71e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.459+0000 7f71e5ffb640 1 -- 172.21.15.2:0/1895114675 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0017070 con 0x7f71f00cb910 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.459+0000 7f71e5ffb640 1 -- 172.21.15.2:0/1895114675 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e00035f0 con 0x7f71f00cb910 2024-09-19T05:42:57.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.459+0000 7f71e5ffb640 1 -- 172.21.15.2:0/1895114675 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e00039e0 con 0x7f71f00cb910 2024-09-19T05:42:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.459+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f015df20 con 0x7f71f00cb910 2024-09-19T05:42:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.459+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f015ebc0 con 0x7f71f00cb910 2024-09-19T05:42:57.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.460+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f00cbe20 con 0x7f71f00cb910 2024-09-19T05:42:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.460+0000 7f71e5ffb640 1 -- 172.21.15.2:0/1895114675 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f71e001a070 con 0x7f71f00cb910 2024-09-19T05:42:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.461+0000 7f71e5ffb640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f71d406c2b0 0x7f71d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.461+0000 7f71e5ffb640 1 -- 172.21.15.2:0/1895114675 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5709+0+0 (secure 0 0 0) 0x7f71e008a770 con 0x7f71f00cb910 2024-09-19T05:42:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.461+0000 7f71f57fa640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f71d406c2b0 0x7f71d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:57.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.462+0000 7f71f57fa640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f71d406c2b0 0x7f71d406e770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009b40 tx=0x7f71ec009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:57.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.465+0000 7f71e5ffb640 1 -- 172.21.15.2:0/1895114675 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f71e0058920 con 0x7f71f00cb910 2024-09-19T05:42:57.523 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:57.522+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:57.532 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:57.532 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:42:57.532 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:42:57.532 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:42:57.533 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:57.542 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:42:57.626 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:57.624+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:57.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:57.653+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f71f0152ca0 con 0x7f71f00cb910 2024-09-19T05:42:57.691 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:42:57.691 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:42:57.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.714493764, 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002950785, 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975069+0000", 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975082+0000", 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3139000000000001e-05 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975096+0000", 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4250000000000001e-05 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975106+0000", 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4949999999999997e-06 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975244+0000", 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013790100000000001 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122292 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975481+0000", 2024-09-19T05:42:57.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011483499999999999 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977502+0000", 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020215849999999998 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977932+0000", 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000429877 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5309000000000003e-05 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978013+0000", 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5796000000000001e-05 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:42:57.697 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.714415412000001, 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033652420000000001, 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:42:57.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-07 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975145+0000", 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975154+0000", 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0219999999999993e-06 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975161+0000", 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9870000000000002e-06 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975720+0000", 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055893000000000002 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975804+0000", 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4025999999999998e-05 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975919+0000", 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115009 2024-09-19T05:42:57.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978071+0000", 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002151563 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978433+0000", 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036205599999999998 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7867999999999997e-05 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978506+0000", 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5397e-05 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:28 2.b 2:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.714384589000002, 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030203629999999999, 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2099999999999997e-07 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975175+0000", 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1109999999999999e-06 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975182+0000", 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975188+0000", 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4090000000000002e-06 2024-09-19T05:42:57.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5392000000000001e-05 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112596 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:42:57.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975488+0000", 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013297499999999999 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977490+0000", 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020015580000000001 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978100+0000", 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060999199999999996 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978150+0000", 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0287000000000002e-05 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978192+0000", 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1635999999999999e-05 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:34 2.b 2:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.714123954000002, 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028916179999999999, 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975433+0000", 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9599999999999999e-07 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975438+0000", 2024-09-19T05:42:57.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975448+0000", 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0145e-05 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975455+0000", 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9149999999999999e-06 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975979+0000", 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052412100000000001 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976090+0000", 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111307 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978324+0000", 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022336130000000002 2024-09-19T05:42:57.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.714063314000001, 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026148299999999998, 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:42:57.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975496+0000", 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975504+0000", 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975510+0000", 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975620+0000", 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109602 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975717+0000", 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7676999999999993e-05 2024-09-19T05:42:57.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978108+0000", 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023904070000000002 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:4 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.713766565, 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022744829999999999, 2024-09-19T05:42:57.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:42:57.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6600000000000005e-07 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975797+0000", 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3790000000000003e-06 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975805+0000", 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975812+0000", 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.029e-06 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975861+0000", 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8266999999999997e-05 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975882+0000", 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896000000000001e-05 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978029+0000", 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021471490000000001 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978064+0000", 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:5 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.712495018999999, 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001037481, 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:42:57.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977066+0000", 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2250000000000002e-06 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977087+0000", 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977095+0000", 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7889999999999995e-06 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977159+0000", 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977190+0000", 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1526999999999998e-05 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000881224 2024-09-19T05:42:57.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978099+0000", 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6962999999999999e-05 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.711913442, 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046917399999999999, 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2399999999999997e-07 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977647+0000", 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977671+0000", 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3926000000000001e-05 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977679+0000", 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.593e-06 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977789+0000", 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109969 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978112+0000", 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032325799999999999 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.711835092000001, 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029909400000000002, 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977722+0000", 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977725+0000", 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3830000000000001e-06 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977737+0000", 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2194999999999999e-05 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977920+0000", 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017706699999999999 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977927+0000", 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978020+0000", 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3358999999999998e-05 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.734 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:28 2.b e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.711601681000001, 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024615999999999999, 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977960+0000", 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3630000000000004e-06 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977970+0000", 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977978+0000", 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0209999999999999e-06 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978088+0000", 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109578 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:42:57.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9630000000000001e-06 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978201+0000", 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106018 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.b epoch 22/19 rep_tid 4 lcod 22'2)", 2024-09-19T05:42:57.739 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.711565276999998, 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033492, 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977999+0000", 2024-09-19T05:42:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8690000000000004e-06 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978007+0000", 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3929999999999999e-06 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978223+0000", 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000211051 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978326+0000", 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102874 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:25 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:42:57.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.711374281000001, 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033417400000000002, 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:42:57.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978186+0000", 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6349999999999999e-06 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978195+0000", 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7870000000000007e-06 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978367+0000", 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017261800000000001 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978420+0000", 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2339999999999997e-05 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978427+0000", 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.25e-06 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978516+0000", 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9038999999999999e-05 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:26 2.1f e22/19 2:fe07bba1:::100.00000000.inode:head v 22'3, mlcod=0'0)", 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.710812547, 2024-09-19T05:42:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047866199999999998, 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978750+0000", 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4809999999999997e-06 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978759+0000", 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1300000000000007e-06 2024-09-19T05:42:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978763+0000", 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3290000000000004e-06 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978801+0000", 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8127999999999998e-05 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978815+0000", 2024-09-19T05:42:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3523000000000001e-05 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979197+0000", 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038223899999999998 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979222+0000", 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5219999999999999e-05 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1f epoch 22/19 rep_tid 9 entries 22'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.710370123000001, 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033723899999999997, 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:42:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979187+0000", 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4270000000000002e-06 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979206+0000", 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313000000000001e-05 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979214+0000", 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420000000000001e-06 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979253+0000", 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.892e-05 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979523+0000", 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000270547 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:42:57.755 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:42:57.760 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:42:57.761 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:57.755+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:57.858 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:57.856+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:57.883 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:57.883 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:42:57.883 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:42:57.883 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:57.893 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:42:57.924 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:42:57.969 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:57.967+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:58.000 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:58.000 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:42:58.000 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:42:58.000 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:42:58.000 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:58.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:42:58.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.023+0000 7fe45725e640 1 Processor -- start 2024-09-19T05:42:58.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.023+0000 7fe45725e640 1 -- start start 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe45725e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4500cb570 0x7fe4500b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe45725e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe4500cb950 0x7fe45015ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe45725e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4500cc6b0 con 0x7fe45015ecd0 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe45725e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe4500b3430 con 0x7fe4500cb950 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe45725e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe4500ccc70 con 0x7fe4500cb570 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe44ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe4500cb950 0x7fe45015ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe44ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe4500cb950 0x7fe45015ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33578/0 (socket says 172.21.15.2:33578) 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe44ffff640 1 -- 172.21.15.2:0/2557536884 learned_addr learned my addr 172.21.15.2:0/2557536884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe454ffa640 1 --2- 172.21.15.2:0/2557536884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4500cb570 0x7fe4500b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe44ffff640 1 -- 172.21.15.2:0/2557536884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4500cb570 msgr2=0x7fe4500b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:42:58.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.024+0000 7fe44ffff640 1 --2- 172.21.15.2:0/2557536884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4500cb570 0x7fe4500b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe44ffff640 1 -- 172.21.15.2:0/2557536884 >> v1:172.21.15.123:6789/0 conn(0x7fe45015ecd0 legacy=0x7fe4500cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe44f7fe640 1 -- 172.21.15.2:0/2557536884 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106832113 0 0) 0x7fe4500cc6b0 con 0x7fe45015ecd0 2024-09-19T05:42:58.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe44ffff640 1 -- 172.21.15.2:0/2557536884 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500cc990 con 0x7fe4500cb950 2024-09-19T05:42:58.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe454ffa640 1 --2- 172.21.15.2:0/2557536884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4500cb570 0x7fe4500b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:42:58.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe44ffff640 1 --2- 172.21.15.2:0/2557536884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb950 0x7fe45015ab50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4440098a0 tx=0x7fe44402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7bf57914739886 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe44f7fe640 1 -- 172.21.15.2:0/2557536884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444037070 con 0x7fe4500cb950 2024-09-19T05:42:58.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe44f7fe640 1 -- 172.21.15.2:0/2557536884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe444032440 con 0x7fe4500cb950 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.025+0000 7fe44f7fe640 1 -- 172.21.15.2:0/2557536884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444032810 con 0x7fe4500cb950 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 -- 172.21.15.2:0/2557536884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb950 msgr2=0x7fe45015ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 --2- 172.21.15.2:0/2557536884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb950 0x7fe45015ab50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4440098a0 tx=0x7fe44402a290 comp rx=0 tx=0).stop 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 -- 172.21.15.2:0/2557536884 shutdown_connections 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 --2- 172.21.15.2:0/2557536884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb950 0x7fe45015ab50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 --2- 172.21.15.2:0/2557536884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4500cb570 0x7fe4500b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 -- 172.21.15.2:0/2557536884 >> 172.21.15.2:0/2557536884 conn(0x7fe450067b30 msgr2=0x7fe450067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 -- 172.21.15.2:0/2557536884 shutdown_connections 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.026+0000 7fe45725e640 1 -- 172.21.15.2:0/2557536884 wait complete. 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.027+0000 7fe45725e640 1 Processor -- start 2024-09-19T05:42:58.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.027+0000 7fe45725e640 1 -- start start 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe45725e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb570 0x7fe4500ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe45725e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4500cb950 0x7fe450151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe45725e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe45015ecd0 0x7fe4501e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe45725e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe45015e7b0 con 0x7fe4500cb570 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe45725e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe45015e4d0 con 0x7fe4500cb950 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe45725e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe45015b090 con 0x7fe45015ecd0 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe454ffa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb570 0x7fe4500ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe454ffa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb570 0x7fe4500ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33586/0 (socket says 172.21.15.2:33586) 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe454ffa640 1 -- 172.21.15.2:0/4233257346 learned_addr learned my addr 172.21.15.2:0/4233257346 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe44ffff640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4500cb950 0x7fe450151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe4557fb640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe45015ecd0 0x7fe4501e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe454ffa640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe45015ecd0 msgr2=0x7fe4501e3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe45015ecd0 0x7fe4501e3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe454ffa640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4500cb950 msgr2=0x7fe450151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.028+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4500cb950 0x7fe450151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.029+0000 7fe454ffa640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500c3fd0 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.029+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb570 0x7fe4500ca140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe44000ec10 tx=0x7fe44000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.029+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440017070 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.029+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44000cc30 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.029+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501f0300 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.029+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440013470 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.029+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501f08a0 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.030+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe4500cc6b0 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.030+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe44001a070 con 0x7fe4500cb570 2024-09-19T05:42:58.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe44d7fa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe43c06c2b0 0x7fe43c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5709+0+0 (secure 0 0 0) 0x7fe44008b440 con 0x7fe4500cb570 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe44ffff640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe43c06c2b0 0x7fe43c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v26) ==== 54+0+16 (secure 0 0 0) 0x7fe4500cc6b0 con 0x7fe4500cb570 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4501545a0 0x7fe4501f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4500b2ef0 con 0x7fe4501545a0 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe4557fb640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4501545a0 0x7fe4501f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.031+0000 7fe44ffff640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe43c06c2b0 0x7fe43c06e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe444034000 tx=0x7fe44402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.032+0000 7fe4557fb640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4501545a0 0x7fe4501f0b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.032+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4500b2ef0 con 0x7fe4501545a0 2024-09-19T05:42:58.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.046+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4501520d0 con 0x7fe4501545a0 2024-09-19T05:42:58.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.048+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4501520d0 con 0x7fe4501545a0 2024-09-19T05:42:58.050 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:42:58.051 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:42:58.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.048+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe450154980 0x7fe4500c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.048+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe4501f10c0 con 0x7fe450154980 2024-09-19T05:42:58.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.048+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe450154980 0x7fe4500c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.049+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe450154980 0x7fe4500c49c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.049+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4501f10c0 con 0x7fe450154980 2024-09-19T05:42:58.059 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:42:58.057+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:42:58.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.064+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4501517b0 con 0x7fe450154980 2024-09-19T05:42:58.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.065+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4501517b0 con 0x7fe450154980 2024-09-19T05:42:58.068 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:42:58.069 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:42:58.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.065+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4501f4140 0x7fe450062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.065+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe4501519c0 con 0x7fe4501f4140 2024-09-19T05:42:58.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.066+0000 7fe44ffff640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4501f4140 0x7fe450062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.066+0000 7fe44ffff640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4501f4140 0x7fe450062630 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.066+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe444032670 con 0x7fe4501f4140 2024-09-19T05:42:58.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.080+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe450062b70 con 0x7fe4501f4140 2024-09-19T05:42:58.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.081+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe450062b70 con 0x7fe4501f4140 2024-09-19T05:42:58.084 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:42:58.085 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:42:58.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.082+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe4500ce280 0x7fe4500ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.082+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe4500cec00 con 0x7fe4500ce280 2024-09-19T05:42:58.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.082+0000 7fe4557fb640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe4500ce280 0x7fe4500ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.082+0000 7fe4557fb640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe4500ce280 0x7fe4500ce6c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.083+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4500cec00 con 0x7fe4500ce280 2024-09-19T05:42:58.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.096+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4500cf0e0 con 0x7fe4500ce280 2024-09-19T05:42:58.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.098+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4500cf0e0 con 0x7fe4500ce280 2024-09-19T05:42:58.100 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:42:58.100 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:42:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.098+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe45002cb50 0x7fe45002cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.098+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe45002d4b0 con 0x7fe45002cb50 2024-09-19T05:42:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.098+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe45002cb50 0x7fe45002cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.098+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe45002cb50 0x7fe45002cf70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.099+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe45002d4b0 con 0x7fe45002cb50 2024-09-19T05:42:58.114 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:42:58.114 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:42:58.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:42:58.114 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:42:58.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.114 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.136325326000001, 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:42:58.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:42:58.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.135449669, 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:42:58.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:42:58.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:42:58.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134825489000001, 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:42:58.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:42:58.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:42:58.132 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134704751000001, 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:42:58.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:42:58.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:42:58.137 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134665462000001, 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.138 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:42:58.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.143 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134600962, 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:42:58.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:42:58.149 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134537544000001, 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.150 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:42:58.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:42:58.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134487888999999, 2024-09-19T05:42:58.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:42:58.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134431719999998, 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:42:58.163 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:42:58.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.170 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134404729, 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:42:58.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:42:58.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134326282, 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:42:58.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:42:58.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134292600999999, 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:42:58.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:42:58.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:42:58.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.188 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134269589999999, 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:42:58.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:42:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:42:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:42:58.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134236195, 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:58.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:42:58.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134193516, 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:42:58.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:42:58.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:42:58.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134154350999999, 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:42:58.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:42:58.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:42:58.214 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.134122324, 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:42:58.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:42:58.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.133923791000001, 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:42:58.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:42:58.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:42:58.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:42:58.225 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.133852612999998, 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.127636455000001, 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:42:58.230 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:42:58.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:42:58.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:42:58.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:42:58.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:42:58.235 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:42:58.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.113+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe45002d4b0 con 0x7fe45002cb50 2024-09-19T05:42:58.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.114+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe45002d4b0 con 0x7fe45002cb50 2024-09-19T05:42:58.237 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:42:58.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:42:58.238 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:58.235+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:58.239 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:42:58.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.115+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe450204bf0 0x7fe450206fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.115+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe45002db30 con 0x7fe450204bf0 2024-09-19T05:42:58.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.115+0000 7fe44ffff640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe450204bf0 0x7fe450206fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.115+0000 7fe44ffff640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe450204bf0 0x7fe450206fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.116+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe45002db30 con 0x7fe450204bf0 2024-09-19T05:42:58.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.129+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe45002db30 con 0x7fe450204bf0 2024-09-19T05:42:58.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.131+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe45002db30 con 0x7fe450204bf0 2024-09-19T05:42:58.240 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.131+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe450207a30 0x7fe450209e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.131+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe45020ccc0 con 0x7fe450207a30 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.132+0000 7fe4557fb640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe450207a30 0x7fe450209e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.132+0000 7fe4557fb640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe450207a30 0x7fe450209e20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.133+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe45020ccc0 con 0x7fe450207a30 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.146+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe45020a360 con 0x7fe450207a30 2024-09-19T05:42:58.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.148+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe45020a360 con 0x7fe450207a30 2024-09-19T05:42:58.242 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:42:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:42:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.148+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe45020aa80 0x7fe45020aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:42:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.148+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe450210090 con 0x7fe45020aa80 2024-09-19T05:42:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.148+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe45020aa80 0x7fe45020aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:42:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.149+0000 7fe454ffa640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe45020aa80 0x7fe45020aec0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:42:58.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.149+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe450210090 con 0x7fe45020aa80 2024-09-19T05:42:58.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.163+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe45020b480 con 0x7fe45020aa80 2024-09-19T05:42:58.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.165+0000 7fe44d7fa640 1 -- 172.21.15.2:0/4233257346 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe45020b480 con 0x7fe45020aa80 2024-09-19T05:42:58.244 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:42:58.246 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:42:58.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.165+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4501545a0 msgr2=0x7fe4501f0b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.165+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4501545a0 0x7fe4501f0b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe450154980 msgr2=0x7fe4500c49c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe450154980 0x7fe4500c49c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4501f4140 msgr2=0x7fe450062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4501f4140 0x7fe450062630 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe4500ce280 msgr2=0x7fe4500ce6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe4500ce280 0x7fe4500ce6c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe45002cb50 msgr2=0x7fe45002cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe45002cb50 0x7fe45002cf70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe450204bf0 msgr2=0x7fe450206fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe450204bf0 0x7fe450206fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe450207a30 msgr2=0x7fe450209e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe450207a30 0x7fe450209e20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe45020aa80 msgr2=0x7fe45020aec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe45020aa80 0x7fe45020aec0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe4557fb640 1 -- 172.21.15.2:0/4233257346 reap_dead start 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe4557fb640 1 -- 172.21.15.2:0/4233257346 reap_dead start 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe4557fb640 1 -- 172.21.15.2:0/4233257346 reap_dead start 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe4557fb640 1 -- 172.21.15.2:0/4233257346 reap_dead start 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe43c06c2b0 msgr2=0x7fe43c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe43c06c2b0 0x7fe43c06e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe444034000 tx=0x7fe44402e000 comp rx=0 tx=0).stop 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb570 msgr2=0x7fe4500ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb570 0x7fe4500ca140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe44000ec10 tx=0x7fe44000c520 comp rx=0 tx=0).stop 2024-09-19T05:42:58.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 shutdown_connections 2024-09-19T05:42:58.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe43c06c2b0 0x7fe43c06e770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 --2- 172.21.15.2:0/4233257346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4500cb570 0x7fe4500ca140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 >> 172.21.15.2:0/4233257346 conn(0x7fe450067b30 msgr2=0x7fe450152d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:58.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 shutdown_connections 2024-09-19T05:42:58.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.166+0000 7fe45725e640 1 -- 172.21.15.2:0/4233257346 wait complete. 2024-09-19T05:42:58.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.335+0000 7f71e5ffb640 1 -- 172.21.15.2:0/1895114675 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v27) ==== 101+0+0 (secure 0 0 0) 0x7f71e005cc20 con 0x7f71f00cb910 2024-09-19T05:42:58.337 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:42:58.338 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:58.335+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f71d406c2b0 msgr2=0x7f71d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f71d406c2b0 0x7f71d406e770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009b40 tx=0x7f71ec009190 comp rx=0 tx=0).stop 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 msgr2=0x7f71f0156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0156430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c410 tx=0x7f71e000c8d0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 shutdown_connections 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f71d406c2b0 0x7f71d406e770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71f015aab0 0x7f71f01f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71f00cb910 0x7f71f0156430 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 --2- 172.21.15.2:0/1895114675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71f00cb530 0x7f71f015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:42:58.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 >> 172.21.15.2:0/1895114675 conn(0x7f71f0067950 msgr2=0x7f71f015a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:42:58.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 shutdown_connections 2024-09-19T05:42:58.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:42:58.337+0000 7f71f7a5e640 1 -- 172.21.15.2:0/1895114675 wait complete. 2024-09-19T05:42:58.384 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:58.384 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:58.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:42:58.438 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:58.437+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:58.496 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:58.497 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:58.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:42:58.547 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:58.545+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:58.640 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:58.638+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:58.649 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:58.649 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:58.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:42:58.741 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:58.739+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:58.803 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:58.803 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:58.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:42:58.842 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:42:58.840+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:42:58.916 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:58.916 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:58.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:42:58.969 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:42:58.967+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:42:59.028 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:59.029 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:59.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:42:59.079 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:42:59.077+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:42:59.144 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:59.143+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:59.182 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:59.182 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:59.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:42:59.275 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:59.274+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:59.334 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:59.334 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:59.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:42:59.346 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:59.344+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:59.447 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:59.446+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:59.487 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:59.487 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:59.496 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:42:59.548 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:42:59.547+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:42:59.617 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:59.617 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:59.626 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:42:59.649 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:42:59.648+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:42:59.735 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:59.735 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:59.744 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:42:59.788 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:59.786+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:42:59.851 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:42:59.849+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:42:59.852 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:42:59.852 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:42:59.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:42:59.951 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:42:59.950+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:00.009 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:00.009 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:00.018 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:00.059 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:00.057+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:00.154 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:00.152+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:00.165 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:00.165 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:00.174 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:00.249 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:00.255 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:00.253+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:00.322 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:00.322 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:00.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:00.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:43:00.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.356+0000 7f21492af640 1 Processor -- start 2024-09-19T05:43:00.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.357+0000 7f21492af640 1 -- start start 2024-09-19T05:43:00.359 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:00.357+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:00.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.357+0000 7f21492af640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f21440cd4b0 0x7f214415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.357+0000 7f21492af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd890 0x7f214406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.357+0000 7f21492af640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21440c9030 con 0x7f21440c6b90 2024-09-19T05:43:00.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.357+0000 7f21492af640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f21440c9310 con 0x7f21440cd4b0 2024-09-19T05:43:00.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.357+0000 7f21492af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f21440c95f0 con 0x7f21440cd890 2024-09-19T05:43:00.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f21437fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f21440c6b90 0x7f21440c47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:39002/0 (socket says 172.21.15.2:39002) 2024-09-19T05:43:00.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f21437fe640 1 -- 172.21.15.2:0/2576072630 learned_addr learned my addr 172.21.15.2:0/2576072630 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f21427fc640 1 --2- 172.21.15.2:0/2576072630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd890 0x7f214406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2576072630 >> v2:172.21.15.123:3300/0 conn(0x7f21440cd4b0 0x7f214415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f2141ffb640 1 -- 172.21.15.2:0/2576072630 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3156423529 0 0) 0x7f21440c9030 con 0x7f21440c6b90 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f2141ffb640 1 -- 172.21.15.2:0/2576072630 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2130003520 con 0x7f21440c6b90 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f21427fc640 1 -- 172.21.15.2:0/2576072630 >> v2:172.21.15.123:3300/0 conn(0x7f21440cd4b0 msgr2=0x7f214415acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f21427fc640 1 --2- 172.21.15.2:0/2576072630 >> v2:172.21.15.123:3300/0 conn(0x7f21440cd4b0 0x7f214415acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f21427fc640 1 -- 172.21.15.2:0/2576072630 >> v1:172.21.15.2:6789/0 conn(0x7f21440c6b90 legacy=0x7f21440c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f21427fc640 1 -- 172.21.15.2:0/2576072630 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21440c98d0 con 0x7f21440cd890 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.358+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2576072630 >> v2:172.21.15.123:3300/0 conn(0x7f21440cd4b0 0x7f214415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.359+0000 7f21427fc640 1 --2- 172.21.15.2:0/2576072630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd890 0x7f214406fc40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f212c0097a0 tx=0x7f212c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b8f30df7d81a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.359+0000 7f2141ffb640 1 -- 172.21.15.2:0/2576072630 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c037070 con 0x7f21440cd890 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.359+0000 7f2141ffb640 1 -- 172.21.15.2:0/2576072630 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212c02ae50 con 0x7f21440cd890 2024-09-19T05:43:00.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.359+0000 7f2141ffb640 1 -- 172.21.15.2:0/2576072630 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c032660 con 0x7f21440cd890 2024-09-19T05:43:00.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 -- 172.21.15.2:0/2576072630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd890 msgr2=0x7f214406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 --2- 172.21.15.2:0/2576072630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd890 0x7f214406fc40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f212c0097a0 tx=0x7f212c02a100 comp rx=0 tx=0).stop 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 -- 172.21.15.2:0/2576072630 shutdown_connections 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 --2- 172.21.15.2:0/2576072630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd890 0x7f214406fc40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 --2- 172.21.15.2:0/2576072630 >> v2:172.21.15.123:3300/0 conn(0x7f21440cd4b0 0x7f214415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 -- 172.21.15.2:0/2576072630 >> 172.21.15.2:0/2576072630 conn(0x7f2144072ab0 msgr2=0x7f2144072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 -- 172.21.15.2:0/2576072630 shutdown_connections 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.360+0000 7f21492af640 1 -- 172.21.15.2:0/2576072630 wait complete. 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.361+0000 7f21492af640 1 Processor -- start 2024-09-19T05:43:00.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.361+0000 7f21492af640 1 -- start start 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21492af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21440c6b90 0x7f21441f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21492af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd4b0 0x7f214415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21492af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21440cd890 0x7f214406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21492af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2144151600 con 0x7f21440cd890 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21492af640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21440c7630 con 0x7f21440c6b90 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21492af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2144151780 con 0x7f21440cd4b0 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21440cd890 0x7f214406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21437fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21440cd890 0x7f214406d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33594/0 (socket says 172.21.15.2:33594) 2024-09-19T05:43:00.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21437fe640 1 -- 172.21.15.2:0/2742976730 learned_addr learned my addr 172.21.15.2:0/2742976730 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21440c6b90 0x7f21441f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.362+0000 7f21427fc640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd4b0 0x7f214415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21437fe640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd4b0 msgr2=0x7f214415a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21437fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f21440cd4b0 0x7f214415a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21437fe640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21440c6b90 msgr2=0x7f21441f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21437fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21440c6b90 0x7f21441f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21437fe640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214414b3c0 con 0x7f21440cd890 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21437fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21440cd890 0x7f214406d5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c430 tx=0x7f213c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c017070 con 0x7f21440cd890 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c005ce0 con 0x7f21440cd890 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2144159250 con 0x7f21440cd890 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c007310 con 0x7f21440cd890 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.363+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214414b0e0 con 0x7f21440cd890 2024-09-19T05:43:00.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.364+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f214415b480 con 0x7f21440cd890 2024-09-19T05:43:00.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.365+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f213c01a070 con 0x7f21440cd890 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.366+0000 7f213b7fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f212406c2b0 0x7f212406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.366+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5709+0+0 (secure 0 0 0) 0x7f213c08a760 con 0x7f21440cd890 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.366+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v27) ==== 54+0+16 (secure 0 0 0) 0x7f213c090170 con 0x7f21440cd890 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.366+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f212406c2b0 0x7f212406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.366+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2144154bd0 0x7f214415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.366+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2144159d00 con 0x7f2144154bd0 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.367+0000 7f21427fc640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2144154bd0 0x7f214415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.367+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f212406c2b0 0x7f212406e770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2134009bd0 tx=0x7f213400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.367+0000 7f21427fc640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2144154bd0 0x7f214415acc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.368+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f212c032660 con 0x7f2144154bd0 2024-09-19T05:43:00.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.383+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f21441420a0 con 0x7f2144154bd0 2024-09-19T05:43:00.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.386+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21441420a0 con 0x7f2144154bd0 2024-09-19T05:43:00.388 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:00.388 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:00.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.386+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f214414d2a0 0x7f21441e9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.386+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2144141ae0 con 0x7f214414d2a0 2024-09-19T05:43:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.386+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f214414d2a0 0x7f21441e9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.386+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f214414d2a0 0x7f21441e9f20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.387+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2144141ae0 con 0x7f214414d2a0 2024-09-19T05:43:00.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.402+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f214414d680 con 0x7f214414d2a0 2024-09-19T05:43:00.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.403+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f214414d680 con 0x7f214414d2a0 2024-09-19T05:43:00.406 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:00.406 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:00.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.404+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f21441ea970 0x7f2144062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.404+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2144159590 con 0x7f21441ea970 2024-09-19T05:43:00.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.404+0000 7f21427fc640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f21441ea970 0x7f2144062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.404+0000 7f21427fc640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f21441ea970 0x7f2144062630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.405+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2144159590 con 0x7f21441ea970 2024-09-19T05:43:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.419+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2144062b70 con 0x7f21441ea970 2024-09-19T05:43:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.421+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2144062b70 con 0x7f21441ea970 2024-09-19T05:43:00.423 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:00.423 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:00.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.421+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f21440632d0 0x7f214406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.421+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f214406c150 con 0x7f21440632d0 2024-09-19T05:43:00.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.421+0000 7f21437fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f21440632d0 0x7f214406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.421+0000 7f21437fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f21440632d0 0x7f214406bc10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.422+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f214406c150 con 0x7f21440632d0 2024-09-19T05:43:00.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.436+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f214406c420 con 0x7f21440632d0 2024-09-19T05:43:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.438+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f214406c420 con 0x7f21440632d0 2024-09-19T05:43:00.440 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.438+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f214402c640 0x7f214402ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.438+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f214406c3a0 con 0x7f214402c640 2024-09-19T05:43:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.439+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f214402c640 0x7f214402ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.439+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f214402c640 0x7f214402ca80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.440+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f214406c3a0 con 0x7f214402c640 2024-09-19T05:43:00.449 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:00.449 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:00.449 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:00.449 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:00.456 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:00.454+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:00.462 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:00.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.462+0000 7f08d0dbd640 1 Processor -- start 2024-09-19T05:43:00.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.462+0000 7f08d0dbd640 1 -- start start 2024-09-19T05:43:00.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.463+0000 7f08d0dbd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f08cc1588f0 0x7f08cc0c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f214406c150 con 0x7f214402c640 2024-09-19T05:43:00.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.463+0000 7f08d0dbd640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc0cb6f0 con 0x7f08cc0cbb10 2024-09-19T05:43:00.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.463+0000 7f08d0dbd640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08cc0c9e00 con 0x7f08cc152ba0 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.463+0000 7f08d0dbd640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f08cc0c9bb0 con 0x7f08cc1588f0 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08c9d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f08cc0cbb10 0x7f08cc0718a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:39004/0 (socket says 172.21.15.2:39004) 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08c9d9b640 1 -- 172.21.15.2:0/2142619651 learned_addr learned my addr 172.21.15.2:0/2142619651 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08cad9d640 1 --2- 172.21.15.2:0/2142619651 >> v2:172.21.15.123:3300/0 conn(0x7f08cc1588f0 0x7f08cc0c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717076994 0 0) 0x7f08cc0cb6f0 con 0x7f08cc0cbb10 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08b8003520 con 0x7f08cc0cbb10 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416644342 0 0) 0x7f08cc0c9e00 con 0x7f08cc152ba0 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08cc0cb6f0 con 0x7f08cc152ba0 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1837171225 0 0) 0x7f08b8003520 con 0x7f08cc0cbb10 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08cad9d640 1 -- 172.21.15.2:0/2142619651 >> v1:172.21.15.123:6789/0 conn(0x7f08cc152ba0 legacy=0x7f08cc0c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08cad9d640 1 -- 172.21.15.2:0/2142619651 >> v1:172.21.15.2:6789/0 conn(0x7f08cc0cbb10 legacy=0x7f08cc0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.464+0000 7f08cad9d640 1 -- 172.21.15.2:0/2142619651 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc0ca0e0 con 0x7f08cc1588f0 2024-09-19T05:43:00.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.465+0000 7f08cad9d640 1 --2- 172.21.15.2:0/2142619651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc1588f0 0x7f08cc0c3ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009d90 tx=0x7f08c402a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=268e099f53fbcfce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.465+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4001670 con 0x7f08cc1588f0 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.465+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c4007540 con 0x7f08cc1588f0 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.465+0000 7f08c959a640 1 -- 172.21.15.2:0/2142619651 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4007910 con 0x7f08cc1588f0 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2142619651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc1588f0 msgr2=0x7f08cc0c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2142619651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc1588f0 0x7f08cc0c3ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009d90 tx=0x7f08c402a560 comp rx=0 tx=0).stop 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2142619651 shutdown_connections 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2142619651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc1588f0 0x7f08cc0c3ae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2142619651 >> 172.21.15.2:0/2142619651 conn(0x7f08cc06bdf0 msgr2=0x7f08cc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2142619651 shutdown_connections 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2142619651 wait complete. 2024-09-19T05:43:00.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.466+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f214406c150 con 0x7f214402c640 2024-09-19T05:43:00.471 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:00.471 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:00.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.467+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f214406c150 0x7f214402d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.467+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f214402da10 con 0x7f214406c150 2024-09-19T05:43:00.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.467+0000 7f21427fc640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f214406c150 0x7f214402d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 Processor -- start 2024-09-19T05:43:00.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f21427fc640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f214406c150 0x7f214402d4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 -- start start 2024-09-19T05:43:00.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0cbb10 0x7f08cc0c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f214402da10 con 0x7f214406c150 2024-09-19T05:43:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08cc152ba0 0x7f08cc1dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc1588f0 0x7f08cc148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc0ca3c0 con 0x7f08cc152ba0 2024-09-19T05:43:00.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08cc158d20 con 0x7f08cc0cbb10 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08d0dbd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f08cc0cb320 con 0x7f08cc1588f0 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.468+0000 7f08ca59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0cbb10 0x7f08cc0c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c9d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08cc152ba0 0x7f08cc1dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08ca59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0cbb10 0x7f08cc0c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:54354/0 (socket says 172.21.15.2:54354) 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08ca59c640 1 -- 172.21.15.2:0/2625860731 learned_addr learned my addr 172.21.15.2:0/2625860731 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08cad9d640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc1588f0 0x7f08cc148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c9d9b640 1 -- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc1588f0 msgr2=0x7f08cc148fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c9d9b640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc1588f0 0x7f08cc148fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c9d9b640 1 -- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0cbb10 msgr2=0x7f08cc0c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c9d9b640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0cbb10 0x7f08cc0c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c9d9b640 1 -- 172.21.15.2:0/2625860731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08cc0c76a0 con 0x7f08cc152ba0 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08ca59c640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0cbb10 0x7f08cc0c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:00.476 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c9d9b640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08cc152ba0 0x7f08cc1dfc40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08b4007fd0 tx=0x7f08b400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c2ffd640 1 -- 172.21.15.2:0/2625860731 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b40058d0 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c2ffd640 1 -- 172.21.15.2:0/2625860731 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b4005a70 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08cc151930 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08c2ffd640 1 -- 172.21.15.2:0/2625860731 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4005e60 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.469+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08cc0c63a0 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.470+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08cc0c9e00 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.471+0000 7f08c2ffd640 1 -- 172.21.15.2:0/2625860731 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f08b4007070 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.472+0000 7f08c2ffd640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f08ac06c2e0 0x7f08ac06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.472+0000 7f08c2ffd640 1 -- 172.21.15.2:0/2625860731 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5709+0+0 (secure 0 0 0) 0x7f08b408ee70 con 0x7f08cc152ba0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.472+0000 7f08ca59c640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f08ac06c2e0 0x7f08ac06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.473+0000 7f08ca59c640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f08ac06c2e0 0x7f08ac06e7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08bc007fd0 tx=0x7f08bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.476+0000 7f08c2ffd640 1 -- 172.21.15.2:0/2625860731 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f08b405ce60 con 0x7f08cc152ba0 2024-09-19T05:43:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.483+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f214402da10 con 0x7f214406c150 2024-09-19T05:43:00.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.485+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f214402da10 con 0x7f214406c150 2024-09-19T05:43:00.487 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.485+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f21442073d0 0x7f21442097c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.485+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f214420c590 con 0x7f21442073d0 2024-09-19T05:43:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.485+0000 7f21437fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f21442073d0 0x7f21442097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.486+0000 7f21437fe640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f21442073d0 0x7f21442097c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.487+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f214420c590 con 0x7f21442073d0 2024-09-19T05:43:00.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.501+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2144209d80 con 0x7f21442073d0 2024-09-19T05:43:00.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.503+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2144209d80 con 0x7f21442073d0 2024-09-19T05:43:00.505 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.503+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f214420a4a0 0x7f214420a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.503+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f214420f9e0 con 0x7f214420a4a0 2024-09-19T05:43:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.503+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f214420a4a0 0x7f214420a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.504+0000 7f2142ffd640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f214420a4a0 0x7f214420a8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:00.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.504+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f214420f9e0 con 0x7f214420a4a0 2024-09-19T05:43:00.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.518+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f214420aee0 con 0x7f214420a4a0 2024-09-19T05:43:00.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.520+0000 7f213b7fe640 1 -- 172.21.15.2:0/2742976730 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f214420aee0 con 0x7f214420a4a0 2024-09-19T05:43:00.522 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:00.523 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2144154bd0 msgr2=0x7f214415acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2144154bd0 0x7f214415acc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f214414d2a0 msgr2=0x7f21441e9f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f214414d2a0 0x7f21441e9f20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f21441ea970 msgr2=0x7f2144062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f21441ea970 0x7f2144062630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f21440632d0 msgr2=0x7f214406bc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f21440632d0 0x7f214406bc10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f214402c640 msgr2=0x7f214402ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21437fe640 1 -- 172.21.15.2:0/2742976730 reap_dead start 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f214402c640 0x7f214402ca80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21437fe640 1 -- 172.21.15.2:0/2742976730 reap_dead start 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f214406c150 msgr2=0x7f214402d4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f214406c150 0x7f214402d4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f21442073d0 msgr2=0x7f21442097c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f21442073d0 0x7f21442097c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f214420a4a0 msgr2=0x7f214420a8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f214420a4a0 0x7f214420a8e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f212406c2b0 msgr2=0x7f212406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f212406c2b0 0x7f212406e770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2134009bd0 tx=0x7f213400b040 comp rx=0 tx=0).stop 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21440cd890 msgr2=0x7f214406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 --2- 172.21.15.2:0/2742976730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21440cd890 0x7f214406d5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c430 tx=0x7f213c00c8f0 comp rx=0 tx=0).stop 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21437fe640 1 -- 172.21.15.2:0/2742976730 reap_dead start 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 shutdown_connections 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 >> 172.21.15.2:0/2742976730 conn(0x7f2144072ab0 msgr2=0x7f2144070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 shutdown_connections 2024-09-19T05:43:00.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.521+0000 7f21492af640 1 -- 172.21.15.2:0/2742976730 wait complete. 2024-09-19T05:43:00.557 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:00.556+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:00.580 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:00.580 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:00.580 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:00.580 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:00.580 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:00.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:00.658 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:00.657+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:00.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:00.663+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f08cc0721f0 con 0x7f08cc152ba0 2024-09-19T05:43:00.697 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:00.697 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:00.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:00.697 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:00.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.720379637000001, 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.720322073999998, 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:43:00.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:00.708 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.720249597999999, 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:43:00.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.72021234, 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:43:00.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.720172538, 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:00.723 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.719343377000001, 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:00.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:43:00.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:43:00.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.718918427999998, 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:43:00.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.714756427000001, 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013154900000000001, 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:00.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3099999999999997e-07 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980460+0000", 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6990000000000001e-06 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980470+0000", 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9420000000000007e-06 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980476+0000", 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0419999999999999e-06 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980510+0000", 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4498000000000001e-05 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980515+0000", 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6539999999999998e-06 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980587+0000", 2024-09-19T05:43:00.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2082999999999994e-05 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.714725439999999, 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:1 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.714452069, 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103356, 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980763+0000", 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6249999999999999e-06 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980770+0000", 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2880000000000004e-06 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980774+0000", 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6780000000000002e-06 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980803+0000", 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9512000000000001e-05 2024-09-19T05:43:00.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980807+0000", 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3610000000000001e-06 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980863+0000", 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6535e-05 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.714145306999999, 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010034, 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981070+0000", 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3730000000000001e-06 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981077+0000", 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6370000000000002e-06 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981080+0000", 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3419999999999999e-06 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981107+0000", 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6806000000000001e-05 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981111+0000", 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981167+0000", 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6246999999999999e-05 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:00.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.712143527999999, 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:43:00.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:43:00.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.711731768, 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:38 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.707078942999999, 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186071, 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988134+0000", 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0999999999999998e-07 2024-09-19T05:43:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988137+0000", 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3919999999999999e-06 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988147+0000", 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659999999999996e-06 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988151+0000", 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.797e-06 2024-09-19T05:43:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988194+0000", 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2911000000000003e-05 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988202+0000", 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988319+0000", 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117355 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.141348357, 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:40 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.139610040000001, 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178677, 2024-09-19T05:43:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555603+0000", 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0699999999999996e-07 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555608+0000", 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2240000000000003e-06 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555620+0000", 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.151e-05 2024-09-19T05:43:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555626+0000", 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8699999999999997e-06 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555665+0000", 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9356000000000001e-05 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555673+0000", 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4230000000000005e-06 2024-09-19T05:43:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555781+0000", 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010738699999999999 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.824681403, 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:43:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:43:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:43:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:43:00.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.822595993, 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:00.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:43:00.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:00.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:00.777 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:00.779 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:00.779 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:00.777+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:00.880 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:00.878+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:00.887 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:00.887 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:00.887 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:00.887 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:00.896 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:00.981 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:00.980+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:01.003 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:01.003 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:01.012 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:01.081 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:01.080+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:01.121 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:01.121 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:01.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:01.121 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:01.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.143221990000001, 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002950785, 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975069+0000", 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975082+0000", 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3139000000000001e-05 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975096+0000", 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4250000000000001e-05 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975106+0000", 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4949999999999997e-06 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975244+0000", 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013790100000000001 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122292 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975481+0000", 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011483499999999999 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977502+0000", 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020215849999999998 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977932+0000", 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000429877 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5309000000000003e-05 2024-09-19T05:43:01.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978013+0000", 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5796000000000001e-05 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.143143638000002, 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033652420000000001, 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-07 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975145+0000", 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:43:01.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975154+0000", 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0219999999999993e-06 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975161+0000", 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9870000000000002e-06 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975720+0000", 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055893000000000002 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975804+0000", 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4025999999999998e-05 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975919+0000", 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115009 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978071+0000", 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002151563 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978433+0000", 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036205599999999998 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:01.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7867999999999997e-05 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978506+0000", 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5397e-05 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:28 2.b 2:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.133 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.143112814999999, 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030203629999999999, 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2099999999999997e-07 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975175+0000", 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1109999999999999e-06 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975182+0000", 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975188+0000", 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4090000000000002e-06 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5392000000000001e-05 2024-09-19T05:43:01.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112596 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975488+0000", 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013297499999999999 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977490+0000", 2024-09-19T05:43:01.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020015580000000001 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978100+0000", 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060999199999999996 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978150+0000", 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0287000000000002e-05 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978192+0000", 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1635999999999999e-05 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:34 2.b 2:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.142852179999998, 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028916179999999999, 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975433+0000", 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9599999999999999e-07 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975438+0000", 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975448+0000", 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0145e-05 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975455+0000", 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9149999999999999e-06 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975979+0000", 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052412100000000001 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976090+0000", 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111307 2024-09-19T05:43:01.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978324+0000", 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022336130000000002 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.142791540000001, 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026148299999999998, 2024-09-19T05:43:01.143 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975496+0000", 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975504+0000", 2024-09-19T05:43:01.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975510+0000", 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975620+0000", 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109602 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975717+0000", 2024-09-19T05:43:01.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7676999999999993e-05 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978108+0000", 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023904070000000002 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:4 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.142494791000001, 2024-09-19T05:43:01.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022744829999999999, 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:01.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6600000000000005e-07 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975797+0000", 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3790000000000003e-06 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975805+0000", 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975812+0000", 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.029e-06 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975861+0000", 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8266999999999997e-05 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975882+0000", 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896000000000001e-05 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978029+0000", 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021471490000000001 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978064+0000", 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:43:01.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:5 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.141223244999999, 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001037481, 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:43:01.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977066+0000", 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2250000000000002e-06 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977087+0000", 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977095+0000", 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7889999999999995e-06 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977159+0000", 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977190+0000", 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1526999999999998e-05 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:01.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000881224 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978099+0000", 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6962999999999999e-05 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-19T05:43:01.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.140641668000001, 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046917399999999999, 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2399999999999997e-07 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977647+0000", 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977671+0000", 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3926000000000001e-05 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977679+0000", 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.593e-06 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977789+0000", 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109969 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978112+0000", 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032325799999999999 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.140563318000002, 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029909400000000002, 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:01.160 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977722+0000", 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977725+0000", 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3830000000000001e-06 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977737+0000", 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2194999999999999e-05 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977920+0000", 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017706699999999999 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977927+0000", 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978020+0000", 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3358999999999998e-05 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:28 2.b e22/19 ondisk, result = 0)", 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.140329907000002, 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024615999999999999, 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977960+0000", 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3630000000000004e-06 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977970+0000", 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:43:01.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977978+0000", 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0209999999999999e-06 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978088+0000", 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109578 2024-09-19T05:43:01.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9630000000000001e-06 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978201+0000", 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106018 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.b epoch 22/19 rep_tid 4 lcod 22'2)", 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.140293502999999, 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033492, 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:01.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977999+0000", 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8690000000000004e-06 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978007+0000", 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3929999999999999e-06 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978223+0000", 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000211051 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978326+0000", 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102874 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:25 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:01.172 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.140102507000002, 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033417400000000002, 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978186+0000", 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6349999999999999e-06 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978195+0000", 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7870000000000007e-06 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978367+0000", 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017261800000000001 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978420+0000", 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2339999999999997e-05 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978427+0000", 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.25e-06 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978516+0000", 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9038999999999999e-05 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:26 2.1f e22/19 2:fe07bba1:::100.00000000.inode:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:01.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.139540773, 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047866199999999998, 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978750+0000", 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4809999999999997e-06 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978759+0000", 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1300000000000007e-06 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978763+0000", 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3290000000000004e-06 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978801+0000", 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8127999999999998e-05 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978815+0000", 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3523000000000001e-05 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979197+0000", 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038223899999999998 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979222+0000", 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5219999999999999e-05 2024-09-19T05:43:01.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1f epoch 22/19 rep_tid 9 entries 22'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.139098349000001, 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033723899999999997, 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:01.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979187+0000", 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4270000000000002e-06 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979206+0000", 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313000000000001e-05 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979214+0000", 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420000000000001e-06 2024-09-19T05:43:01.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979253+0000", 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.892e-05 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979523+0000", 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000270547 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.185 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:01.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:01.187 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:01.185+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:01.286 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:01.285+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:01.301 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:01.301 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:01.301 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:01.301 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:01.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:01.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.341+0000 7f08c2ffd640 1 -- 172.21.15.2:0/2625860731 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v28) ==== 87+0+0 (secure 0 0 0) 0x7f08b4061160 con 0x7f08cc152ba0 2024-09-19T05:43:01.343 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.343+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f08ac06c2e0 msgr2=0x7f08ac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.343+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f08ac06c2e0 0x7f08ac06e7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08bc007fd0 tx=0x7f08bc009040 comp rx=0 tx=0).stop 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.343+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08cc152ba0 msgr2=0x7f08cc1dfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.343+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08cc152ba0 0x7f08cc1dfc40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f08b4007fd0 tx=0x7f08b400ecc0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 shutdown_connections 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f08ac06c2e0 0x7f08ac06e7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08cc1588f0 0x7f08cc148fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08cc152ba0 0x7f08cc1dfc40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 --2- 172.21.15.2:0/2625860731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08cc0cbb10 0x7f08cc0c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 >> 172.21.15.2:0/2625860731 conn(0x7f08cc06bdf0 msgr2=0x7f08cc151ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:01.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 shutdown_connections 2024-09-19T05:43:01.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.344+0000 7f08d0dbd640 1 -- 172.21.15.2:0/2625860731 wait complete. 2024-09-19T05:43:01.399 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:01.397+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:01.414 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:01.414 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:01.415 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:01.415 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:01.423 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:01.489 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:01.487+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.550335346000001, 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:43:01.530 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:43:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:43:01.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.549459688999999, 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.536 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:43:01.541 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548835509, 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:43:01.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:01.546 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548714771, 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:01.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548675482, 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:43:01.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548610982, 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.555 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:43:01.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:43:01.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548547564, 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:43:01.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548497908999998, 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.566 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548441740000001, 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:43:01.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548414748999999, 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.577 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:43:01.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:43:01.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548336301999999, 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:43:01.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:43:01.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548302621000001, 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:43:01.588 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:43:01.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:43:01.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.593 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548279610000002, 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:01.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:43:01.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548246214999999, 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.599 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:43:01.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:43:01.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:43:01.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548203535999999, 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.605 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:01.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:43:01.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:01.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.610 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548164370999999, 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:01.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:43:01.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:43:01.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.548132343999999, 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:01.616 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:43:01.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.547933811, 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.620 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:43:01.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:43:01.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:43:01.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.547862633000001, 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:43:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:43:01.627 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.541646475, 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:43:01.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:43:01.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:01.632 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:01.632 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:01.632 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:01.633 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-19T05:43:01.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T05:43:01.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:01.635 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:01.632+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:01.735 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:01.734+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:01.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.743+0000 7f2c22ff8640 1 Processor -- start 2024-09-19T05:43:01.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.743+0000 7f2c22ff8640 1 -- start start 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.744+0000 7f2c22ff8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2c1c154a10 0x7f2c1c15b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.744+0000 7f2c22ff8640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c1c15e040 con 0x7f2c1c0c75f0 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.744+0000 7f2c22ff8640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c1c15e320 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.744+0000 7f2c22ff8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f2c1c154df0 con 0x7f2c1c154a10 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.744+0000 7f2c20d94640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2c1c154a10 0x7f2c1c15b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.744+0000 7f2c21595640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f2c1c0c75f0 0x7f2c1c06c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:37446/0 (socket says 172.21.15.2:37446) 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.744+0000 7f2c21595640 1 -- 172.21.15.2:0/3609727277 learned_addr learned my addr 172.21.15.2:0/3609727277 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.745+0000 7f2c20d94640 1 -- 172.21.15.2:0/3609727277 >> v1:172.21.15.123:6789/0 conn(0x7f2c1c0c71a0 legacy=0x7f2c1c0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.745+0000 7f2c20d94640 1 -- 172.21.15.2:0/3609727277 >> v1:172.21.15.2:6789/0 conn(0x7f2c1c0c75f0 legacy=0x7f2c1c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.745+0000 7f2c1b7fe640 1 -- 172.21.15.2:0/3609727277 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3474356755 0 0) 0x7f2c1c15e040 con 0x7f2c1c0c75f0 2024-09-19T05:43:01.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.745+0000 7f2c20d94640 1 -- 172.21.15.2:0/3609727277 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c15e600 con 0x7f2c1c154a10 2024-09-19T05:43:01.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.745+0000 7f2c20d94640 1 --2- 172.21.15.2:0/3609727277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c154a10 0x7f2c1c15b3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0097a0 tx=0x7f2c0c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a0f8c293d5c663 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:01.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.745+0000 7f2c1b7fe640 1 -- 172.21.15.2:0/3609727277 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c037070 con 0x7f2c1c154a10 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.745+0000 7f2c1b7fe640 1 -- 172.21.15.2:0/3609727277 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0c032640 con 0x7f2c1c154a10 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c1b7fe640 1 -- 172.21.15.2:0/3609727277 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c032a10 con 0x7f2c1c154a10 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c22ff8640 1 -- 172.21.15.2:0/3609727277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c154a10 msgr2=0x7f2c1c15b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/3609727277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c154a10 0x7f2c1c15b3c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0097a0 tx=0x7f2c0c02a2c0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c22ff8640 1 -- 172.21.15.2:0/3609727277 shutdown_connections 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/3609727277 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c154a10 0x7f2c1c15b3c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c22ff8640 1 -- 172.21.15.2:0/3609727277 >> 172.21.15.2:0/3609727277 conn(0x7f2c1c0cd360 msgr2=0x7f2c1c0cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c22ff8640 1 -- 172.21.15.2:0/3609727277 shutdown_connections 2024-09-19T05:43:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.746+0000 7f2c22ff8640 1 -- 172.21.15.2:0/3609727277 wait complete. 2024-09-19T05:43:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.747+0000 7f2c22ff8640 1 Processor -- start 2024-09-19T05:43:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.747+0000 7f2c22ff8640 1 -- start start 2024-09-19T05:43:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c22ff8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c0c71a0 0x7f2c1c1f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c22ff8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c1c0c75f0 0x7f2c1c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c22ff8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c1c154a10 0x7f2c1c0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c22ff8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c0c8470 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c22ff8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1c15ebc0 con 0x7f2c1c0c75f0 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c22ff8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1c0c85f0 con 0x7f2c1c154a10 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c20d94640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c0c71a0 0x7f2c1c1f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c20d94640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c0c71a0 0x7f2c1c1f0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60390/0 (socket says 172.21.15.2:60390) 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c20d94640 1 -- 172.21.15.2:0/180602944 learned_addr learned my addr 172.21.15.2:0/180602944 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c1bfff640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c1c0c75f0 0x7f2c1c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.748+0000 7f2c21595640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c1c154a10 0x7f2c1c0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c20d94640 1 -- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c1c154a10 msgr2=0x7f2c1c0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c20d94640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c1c154a10 0x7f2c1c0c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c20d94640 1 -- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c1c0c75f0 msgr2=0x7f2c1c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c20d94640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c1c0c75f0 0x7f2c1c153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c20d94640 1 -- 172.21.15.2:0/180602944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1c149060 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c20d94640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c0c71a0 0x7f2c1c1f0b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0097a0 tx=0x7f2c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c197fa640 1 -- 172.21.15.2:0/180602944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c037050 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c197fa640 1 -- 172.21.15.2:0/180602944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0c038040 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c197fa640 1 -- 172.21.15.2:0/180602944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c004490 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1c154200 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.749+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1c149300 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.750+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c15b3c0 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.751+0000 7f2c197fa640 1 -- 172.21.15.2:0/180602944 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2c0c030900 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.751+0000 7f2c197fa640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c0806c2b0 0x7f2c0806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:01.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.751+0000 7f2c197fa640 1 -- 172.21.15.2:0/180602944 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f2c0c0abed0 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.751+0000 7f2c1bfff640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c0806c2b0 0x7f2c0806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:01.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.752+0000 7f2c1bfff640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c0806c2b0 0x7f2c0806e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0ca550 tx=0x7f2c10009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.755+0000 7f2c197fa640 1 -- 172.21.15.2:0/180602944 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2c0c07a110 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.779 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:01.780 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:01.780 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:01.780 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:01.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:01.873 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:01.871+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:01.931 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:01.932 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:01.932 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:01.932 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:01.932 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:01.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.931+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f2c1c158260 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.932+0000 7f2c197fa640 1 -- 172.21.15.2:0/180602944 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v28) ==== 81+0+34179 (secure 0 0 0) 0x7f2c0c07e410 con 0x7f2c1c0c71a0 2024-09-19T05:43:01.937 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:43:01.937 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 28, 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "896e162c-4658-4743-b70e-7ec2b5bce950", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-19T05:42:09.734568+0000", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-19T05:43:01.338954+0000", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-19T05:42:20.842106+0000", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-19T05:42:11.870630+0000", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5801984, 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-19T05:43:01.938 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "jewel", 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-19T05:43:01.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:21.863894+0000", 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:01.940 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:01.941 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "18", 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:01.942 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:01.943 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:01.944 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-19T05:43:01.945 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_metadata", 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:30.912746+0000", 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:01.946 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:01.947 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:01.948 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:01.949 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:01.950 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_bias": 4, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 16, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_priority": 5 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": "cephfs" 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.951 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 2.25, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 2.25, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2.25, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2.25, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.952 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_data", 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:31.312255+0000", 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:01.953 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 64, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 64, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 64, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 64, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 64, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:01.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "22", 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:01.955 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:01.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 64, 2024-09-19T05:43:01.957 INFO:teuthology.orchestra.run.smithi002.stdout: "target_size_ratio": 0.89999997615814209 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "data": "cephfs" 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.5, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.5, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:01.958 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "665c2ac5-a964-4f38-8fa6-f4e95cac9272", 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.961 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.962 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.963 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/288510134", 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/288510134", 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/288510134", 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/288510134", 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a4443f63-2c47-4c6f-a2df-6765b78afcc2", 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2074594312", 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2074594312", 2024-09-19T05:43:01.969 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2074594312", 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2074594312", 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "61465313-1ba1-4f26-b488-6ea73981419b", 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.970 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2644191985", 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2644191985", 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2644191985", 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2644191985", 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fa9d27d5-68d4-4467-a2a5-79f767a071aa", 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.976 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.977 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.978 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:01.979 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.980 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/3985782072", 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/3985782072", 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/3985782072", 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/3985782072", 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "85a040ff-ad8b-44ff-b82e-a36b05d83087", 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.981 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6816", 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6817", 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6818", 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6819", 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6822", 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6823", 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6820", 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6821", 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6817/2654957182", 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6819/2654957182", 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6823/2654957182", 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6821/2654957182", 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "f90f8fa4-4afc-4272-89eb-5e90123155d9", 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.987 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6800", 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6801", 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.988 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6802", 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6803", 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:01.989 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6806", 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6807", 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:01.990 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6804", 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6805", 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6801/3690741328", 2024-09-19T05:43:01.991 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6803/3690741328", 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6807/3690741328", 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6805/3690741328", 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "48561580-ba9a-4096-9722-16c308f414ee", 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.992 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6824", 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6825", 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6826", 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6827", 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6830", 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.995 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6831", 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6828", 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.996 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6829", 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6825/593730548", 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6827/593730548", 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6831/593730548", 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6829/593730548", 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "d1ee7357-25a5-432c-b96a-e808139a1cf3", 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:01.998 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6808", 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6809", 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:01.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6810", 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6811", 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6814", 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6815", 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6812", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6813", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6809/1657801827", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6811/1657801827", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6815/1657801827", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6813/1657801827", 2024-09-19T05:43:02.002 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.003 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.148620+0000", 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.921396+0000", 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.004 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.980898+0000", 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.005 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.184322+0000", 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.139258+0000", 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.479047+0000", 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.007 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.432111+0000", 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.402960+0000", 2024-09-19T05:43:02.008 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-19T05:43:02.009 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-19T05:43:02.010 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-19T05:43:02.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:02.011 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:02.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c0806c2b0 msgr2=0x7f2c0806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c0806c2b0 0x7f2c0806e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0ca550 tx=0x7f2c10009040 comp rx=0 tx=0).stop 2024-09-19T05:43:02.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c0c71a0 msgr2=0x7f2c1c1f0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c0c71a0 0x7f2c1c1f0b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0097a0 tx=0x7f2c0c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 shutdown_connections 2024-09-19T05:43:02.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2c0806c2b0 0x7f2c0806e770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c1c0c75f0 0x7f2c1c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c1c154a10 0x7f2c1c0c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 --2- 172.21.15.2:0/180602944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c1c0c71a0 0x7f2c1c1f0b10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 >> 172.21.15.2:0/180602944 conn(0x7f2c1c0cd360 msgr2=0x7f2c1c06cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:02.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 shutdown_connections 2024-09-19T05:43:02.013 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:01.934+0000 7f2c22ff8640 1 -- 172.21.15.2:0/180602944 wait complete. 2024-09-19T05:43:02.014 INFO:tasks.thrashosds.thrasher:j is {'epoch': 28, 'fsid': '896e162c-4658-4743-b70e-7ec2b5bce950', 'created': '2024-09-19T05:42:09.734568+0000', 'modified': '2024-09-19T05:43:01.338954+0000', 'last_up_change': '2024-09-19T05:42:20.842106+0000', 'last_in_change': '2024-09-19T05:42:11.870630+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:42:21.863894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:42:30.912746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:42:31.312255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '665c2ac5-a964-4f38-8fa6-f4e95cac9272', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 288510134}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 288510134}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 288510134}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 288510134}]}, 'public_addr': '172.21.15.2:6809/288510134', 'cluster_addr': '172.21.15.2:6811/288510134', 'heartbeat_back_addr': '172.21.15.2:6815/288510134', 'heartbeat_front_addr': '172.21.15.2:6813/288510134', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a4443f63-2c47-4c6f-a2df-6765b78afcc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2074594312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2074594312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2074594312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2074594312}]}, 'public_addr': '172.21.15.2:6817/2074594312', 'cluster_addr': '172.21.15.2:6819/2074594312', 'heartbeat_back_addr': '172.21.15.2:6823/2074594312', 'heartbeat_front_addr': '172.21.15.2:6821/2074594312', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61465313-1ba1-4f26-b488-6ea73981419b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2644191985}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2644191985}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2644191985}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2644191985}]}, 'public_addr': '172.21.15.2:6825/2644191985', 'cluster_addr': '172.21.15.2:6827/2644191985', 'heartbeat_back_addr': '172.21.15.2:6831/2644191985', 'heartbeat_front_addr': '172.21.15.2:6829/2644191985', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fa9d27d5-68d4-4467-a2a5-79f767a071aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 3985782072}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 3985782072}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 3985782072}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 3985782072}]}, 'public_addr': '172.21.15.2:6801/3985782072', 'cluster_addr': '172.21.15.2:6803/3985782072', 'heartbeat_back_addr': '172.21.15.2:6807/3985782072', 'heartbeat_front_addr': '172.21.15.2:6805/3985782072', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85a040ff-ad8b-44ff-b82e-a36b05d83087', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 2654957182}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 2654957182}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 2654957182}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 2654957182}]}, 'public_addr': '172.21.15.123:6817/2654957182', 'cluster_addr': '172.21.15.123:6819/2654957182', 'heartbeat_back_addr': '172.21.15.123:6823/2654957182', 'heartbeat_front_addr': '172.21.15.123:6821/2654957182', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f90f8fa4-4afc-4272-89eb-5e90123155d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 3690741328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 3690741328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 3690741328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 3690741328}]}, 'public_addr': '172.21.15.123:6801/3690741328', 'cluster_addr': '172.21.15.123:6803/3690741328', 'heartbeat_back_addr': '172.21.15.123:6807/3690741328', 'heartbeat_front_addr': '172.21.15.123:6805/3690741328', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '48561580-ba9a-4096-9722-16c308f414ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 593730548}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 593730548}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 593730548}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 593730548}]}, 'public_addr': '172.21.15.123:6825/593730548', 'cluster_addr': '172.21.15.123:6827/593730548', 'heartbeat_back_addr': '172.21.15.123:6831/593730548', 'heartbeat_front_addr': '172.21.15.123:6829/593730548', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd1ee7357-25a5-432c-b96a-e808139a1cf3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 1657801827}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 1657801827}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 1657801827}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 1657801827}]}, 'public_addr': '172.21.15.123:6809/1657801827', 'cluster_addr': '172.21.15.123:6811/1657801827', 'heartbeat_back_addr': '172.21.15.123:6815/1657801827', 'heartbeat_front_addr': '172.21.15.123:6813/1657801827', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.148620+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.921396+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.980898+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.184322+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.139258+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.479047+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.432111+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.402960+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T05:43:02.014 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-19T05:43:02.015 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:02.015 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:02.013+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:02.115 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:02.114+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:02.160 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.639762341999997, 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.161 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:43:02.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.639404374000001, 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.166 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:43:02.170 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.638903982999999, 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:43:02.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:43:02.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:43:02.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.63191775, 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:02.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:43:02.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:43:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:43:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:43:02.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:43:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:43:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:43:02.181 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.631459941999999, 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:43:02.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:43:02.186 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.608746322000002, 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:02.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:43:02.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.193 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.608251377000002, 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:43:02.194 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:43:02.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:43:02.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:43:02.200 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.046448421000001, 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:43:02.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:02.206 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.045690645000001, 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:02.211 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.045109598000003, 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:43:02.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.216 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.044474551999997, 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:43:02.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.038668045999998, 2024-09-19T05:43:02.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.038011873999999, 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.226 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:43:02.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.561463199999999, 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:43:02.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:43:02.235 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.561094885999999, 2024-09-19T05:43:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:43:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.236 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.240 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.560715975999997, 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.241 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.560019123000004, 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:02.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 37.554234508999997, 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.250 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.254 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.183119495, 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:43:02.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:43:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:43:02.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.183076411999998, 2024-09-19T05:43:02.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.263 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:02.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:02.265 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:02.263+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:02.366 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:02.364+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:02.409 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:02.409 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:02.409 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:02.417 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:02.466 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:02.464+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:02.521 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:02.521 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:02.521 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:02.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:02.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:02.574 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:02.573+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:02.633 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:02.633 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 24.655746992000001, 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:02.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:02.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:02.637 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:02.642 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.650+0000 7fe47bda1640 1 Processor -- start 2024-09-19T05:43:02.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.650+0000 7fe47bda1640 1 -- start start 2024-09-19T05:43:02.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47bda1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47406ec00 0x7fe4740b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47bda1640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe4740ce3d0 0x7fe4740ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47bda1640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe474070280 con 0x7fe4740b0ae0 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47bda1640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe474070520 con 0x7fe4740ce3d0 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47bda1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe4740707c0 con 0x7fe47406ec00 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47a33e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe4740ce3d0 0x7fe4740ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47a33e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe4740ce3d0 0x7fe4740ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60392/0 (socket says 172.21.15.2:60392) 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47a33e640 1 -- 172.21.15.2:0/836434422 learned_addr learned my addr 172.21.15.2:0/836434422 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47933c640 1 --2- 172.21.15.2:0/836434422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47406ec00 0x7fe4740b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47a33e640 1 -- 172.21.15.2:0/836434422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47406ec00 msgr2=0x7fe4740b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47a33e640 1 --2- 172.21.15.2:0/836434422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47406ec00 0x7fe4740b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47a33e640 1 -- 172.21.15.2:0/836434422 >> v1:172.21.15.123:6789/0 conn(0x7fe4740b0ae0 legacy=0x7fe4740c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.651+0000 7fe47a33e640 1 -- 172.21.15.2:0/836434422 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe474070b30 con 0x7fe4740ce3d0 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.652+0000 7fe47a33e640 1 --2- 172.21.15.2:0/836434422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4740ce3d0 0x7fe4740ce880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009e50 tx=0x7fe46c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ef2bf598a1b0b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.652+0000 7fe478b3b640 1 -- 172.21.15.2:0/836434422 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c036070 con 0x7fe4740ce3d0 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.652+0000 7fe478b3b640 1 -- 172.21.15.2:0/836434422 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c031600 con 0x7fe4740ce3d0 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.652+0000 7fe478b3b640 1 -- 172.21.15.2:0/836434422 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0319d0 con 0x7fe4740ce3d0 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 -- 172.21.15.2:0/836434422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4740ce3d0 msgr2=0x7fe4740ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 --2- 172.21.15.2:0/836434422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4740ce3d0 0x7fe4740ce880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009e50 tx=0x7fe46c02a7a0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 -- 172.21.15.2:0/836434422 shutdown_connections 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 --2- 172.21.15.2:0/836434422 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4740ce3d0 0x7fe4740ce880 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 --2- 172.21.15.2:0/836434422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47406ec00 0x7fe4740b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 -- 172.21.15.2:0/836434422 >> 172.21.15.2:0/836434422 conn(0x7fe47406c240 msgr2=0x7fe474067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 -- 172.21.15.2:0/836434422 shutdown_connections 2024-09-19T05:43:02.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.653+0000 7fe47bda1640 1 -- 172.21.15.2:0/836434422 wait complete. 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.654+0000 7fe47bda1640 1 Processor -- start 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.654+0000 7fe47bda1640 1 -- start start 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47bda1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47406ec00 0x7fe4740af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47bda1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4740b0ae0 0x7fe4740cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47bda1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4740ce3d0 0x7fe4741575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47bda1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe4740c9af0 con 0x7fe47406ec00 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47bda1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4740c9810 con 0x7fe4740ce3d0 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47bda1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe47415d150 con 0x7fe4740b0ae0 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47406ec00 0x7fe4740af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47406ec00 0x7fe4740af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60402/0 (socket says 172.21.15.2:60402) 2024-09-19T05:43:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 -- 172.21.15.2:0/327061288 learned_addr learned my addr 172.21.15.2:0/327061288 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4740ce3d0 0x7fe4741575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4740b0ae0 0x7fe4740cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4740b0ae0 msgr2=0x7fe4740cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4740b0ae0 0x7fe4740cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4740ce3d0 msgr2=0x7fe4741575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4740ce3d0 0x7fe4741575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe479b3d640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4741f0a40 con 0x7fe47406ec00 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe4740b0ae0 0x7fe4740cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.655+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4740ce3d0 0x7fe4741575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.656+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47406ec00 0x7fe4740af830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe47000c400 tx=0x7fe47000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.656+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4700052d0 con 0x7fe47406ec00 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.656+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe470005490 con 0x7fe47406ec00 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.656+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4741f0d00 con 0x7fe47406ec00 2024-09-19T05:43:02.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.656+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470007260 con 0x7fe47406ec00 2024-09-19T05:43:02.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.656+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4741f12c0 con 0x7fe47406ec00 2024-09-19T05:43:02.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.657+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe47406e390 con 0x7fe47406ec00 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.657+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe470007400 con 0x7fe47406ec00 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.658+0000 7fe46a7fc640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe45806c2b0 0x7fe45806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.658+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7fe47008b3e0 con 0x7fe47406ec00 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.658+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v28) ==== 54+0+16 (secure 0 0 0) 0x7fe47406e390 con 0x7fe47406ec00 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.658+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe45806c2b0 0x7fe45806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.658+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4740aea00 0x7fe4741447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.658+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4740ce880 con 0x7fe4740aea00 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.658+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4740aea00 0x7fe4741447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.659+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe45806c2b0 0x7fe45806e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4740cdb30 tx=0x7fe464009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.659+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4740aea00 0x7fe4741447d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.660+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4740ce880 con 0x7fe4740aea00 2024-09-19T05:43:02.668 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:02.666+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:02.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.674+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe47406dd40 con 0x7fe4740aea00 2024-09-19T05:43:02.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.676+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe47406dd40 con 0x7fe4740aea00 2024-09-19T05:43:02.678 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:02.678 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:02.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.676+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe4740aede0 0x7fe474145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.676+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe4740c84a0 con 0x7fe4740aede0 2024-09-19T05:43:02.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.676+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe4740aede0 0x7fe474145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.676+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe4740aede0 0x7fe474145220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.677+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4740c84a0 con 0x7fe4740aede0 2024-09-19T05:43:02.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.691+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4740afd70 con 0x7fe4740aede0 2024-09-19T05:43:02.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.693+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4740afd70 con 0x7fe4740aede0 2024-09-19T05:43:02.695 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:02.696 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:02.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.693+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4741f4130 0x7fe4741f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.693+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe47415a030 con 0x7fe4741f4130 2024-09-19T05:43:02.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.693+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4741f4130 0x7fe4741f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.694+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4741f4130 0x7fe4741f4510 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.694+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe47415a030 con 0x7fe4741f4130 2024-09-19T05:43:02.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.708+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4741f4a50 con 0x7fe4741f4130 2024-09-19T05:43:02.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.710+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4741f4a50 con 0x7fe4741f4130 2024-09-19T05:43:02.712 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.710+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe474062f00 0x7fe4740cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.710+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe474063320 con 0x7fe474062f00 2024-09-19T05:43:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.710+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe474062f00 0x7fe4740cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.710+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe474062f00 0x7fe4740cc380 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.711+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe474063320 con 0x7fe474062f00 2024-09-19T05:43:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.724+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe474063320 con 0x7fe474062f00 2024-09-19T05:43:02.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.726+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe474063320 con 0x7fe474062f00 2024-09-19T05:43:02.728 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:02.729 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:02.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.726+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe4740ccdd0 0x7fe47402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.726+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe474063320 con 0x7fe4740ccdd0 2024-09-19T05:43:02.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.726+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe4740ccdd0 0x7fe47402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.727+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe4740ccdd0 0x7fe47402c640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.728+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe474063320 con 0x7fe4740ccdd0 2024-09-19T05:43:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.741+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe474063320 con 0x7fe4740ccdd0 2024-09-19T05:43:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.743+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe474063320 con 0x7fe4740ccdd0 2024-09-19T05:43:02.745 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.743+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe47402d090 0x7fe47402d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.743+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe47402da10 con 0x7fe47402d090 2024-09-19T05:43:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.744+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe47402d090 0x7fe47402d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.744+0000 7fe47933c640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe47402d090 0x7fe47402d4d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.745+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe47402da10 con 0x7fe47402d090 2024-09-19T05:43:02.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.759+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe47402da10 con 0x7fe47402d090 2024-09-19T05:43:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.760+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe47402da10 con 0x7fe47402d090 2024-09-19T05:43:02.763 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.761+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe474207720 0x7fe474209b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.761+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe47420c9f0 con 0x7fe474207720 2024-09-19T05:43:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.761+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe474207720 0x7fe474209b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.761+0000 7fe47a33e640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe474207720 0x7fe474209b10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.762+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe47420c9f0 con 0x7fe474207720 2024-09-19T05:43:02.769 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:02.767+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:02.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.776+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4740756b0 con 0x7fe474207720 2024-09-19T05:43:02.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.777+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4740756b0 con 0x7fe474207720 2024-09-19T05:43:02.780 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.778+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe47420a560 0x7fe47420a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.778+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe47420fdc0 con 0x7fe47420a560 2024-09-19T05:43:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.778+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe47420a560 0x7fe47420a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.779+0000 7fe479b3d640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe47420a560 0x7fe47420a9c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:02.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.780+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe47420fdc0 con 0x7fe47420a560 2024-09-19T05:43:02.787 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:02.787 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:02.787 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.793+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe47420b010 con 0x7fe47420a560 2024-09-19T05:43:02.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe46a7fc640 1 -- 172.21.15.2:0/327061288 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe47420b010 con 0x7fe47420a560 2024-09-19T05:43:02.797 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4740aea00 msgr2=0x7fe4741447d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe4740aea00 0x7fe4741447d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe4740aede0 msgr2=0x7fe474145220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe4740aede0 0x7fe474145220 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4741f4130 msgr2=0x7fe4741f4510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe4741f4130 0x7fe4741f4510 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe474062f00 msgr2=0x7fe4740cc380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe474062f00 0x7fe4740cc380 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe4740ccdd0 msgr2=0x7fe47402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe4740ccdd0 0x7fe47402c640 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47a33e640 1 -- 172.21.15.2:0/327061288 reap_dead start 2024-09-19T05:43:02.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47a33e640 1 -- 172.21.15.2:0/327061288 reap_dead start 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe47402d090 msgr2=0x7fe47402d4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe47402d090 0x7fe47402d4d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe474207720 msgr2=0x7fe474209b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe474207720 0x7fe474209b10 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe47420a560 msgr2=0x7fe47420a9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe47420a560 0x7fe47420a9c0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe45806c2b0 msgr2=0x7fe45806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe45806c2b0 0x7fe45806e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4740cdb30 tx=0x7fe464009190 comp rx=0 tx=0).stop 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47406ec00 msgr2=0x7fe4740af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 --2- 172.21.15.2:0/327061288 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47406ec00 0x7fe4740af830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe47000c400 tx=0x7fe47000c8c0 comp rx=0 tx=0).stop 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47a33e640 1 -- 172.21.15.2:0/327061288 reap_dead start 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47a33e640 1 -- 172.21.15.2:0/327061288 reap_dead start 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 shutdown_connections 2024-09-19T05:43:02.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 >> 172.21.15.2:0/327061288 conn(0x7fe47406c240 msgr2=0x7fe474067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:02.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.795+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 shutdown_connections 2024-09-19T05:43:02.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:02.796+0000 7fe47bda1640 1 -- 172.21.15.2:0/327061288 wait complete. 2024-09-19T05:43:02.881 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:02.879+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:02.940 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:02.940 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:02.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:02.971 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:02.969+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:03.071 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:03.070+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:03.092 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:03.092 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:03.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:03.173 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:03.171+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:03.211 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:03.211 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:03.220 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:03.309 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:03.306+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:03.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:43:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:03.371 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:03.374 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:03.372+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:03.380 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.462+0000 7faea1c8b640 1 Processor -- start 2024-09-19T05:43:03.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.462+0000 7faea1c8b640 1 -- start start 2024-09-19T05:43:03.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7faea1c8b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fae9c152b80 0x7fae9c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7faea1c8b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c0c9000 0x7fae9c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7faea1c8b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9c0ca3a0 con 0x7fae9c071110 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7faea1c8b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fae9c0c94f0 con 0x7fae9c152b80 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7faea1c8b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fae9c0c3750 con 0x7fae9c0c9000 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9b7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fae9c152b80 0x7fae9c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9affd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c0c9000 0x7fae9c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9bfff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fae9c071110 0x7fae9c15beb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:54512/0 (socket says 172.21.15.2:54512) 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9b7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fae9c152b80 0x7fae9c1591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:48344/0 (socket says 172.21.15.2:48344) 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9affd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c0c9000 0x7fae9c0ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:52224/0 (socket says 172.21.15.2:52224) 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9bfff640 1 -- 172.21.15.2:0/1719963473 learned_addr learned my addr 172.21.15.2:0/1719963473 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9a7fc640 1 -- 172.21.15.2:0/1719963473 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673053561 0 0) 0x7fae9c0ca3a0 con 0x7fae9c071110 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9affd640 1 -- 172.21.15.2:0/1719963473 >> v2:172.21.15.123:3300/0 conn(0x7fae9c152b80 msgr2=0x7fae9c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9affd640 1 --2- 172.21.15.2:0/1719963473 >> v2:172.21.15.123:3300/0 conn(0x7fae9c152b80 0x7fae9c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:03.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9affd640 1 -- 172.21.15.2:0/1719963473 >> v1:172.21.15.123:6789/0 conn(0x7fae9c071110 legacy=0x7fae9c15beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:03.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.463+0000 7fae9affd640 1 -- 172.21.15.2:0/1719963473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c0c3a30 con 0x7fae9c0c9000 2024-09-19T05:43:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.464+0000 7fae9affd640 1 --2- 172.21.15.2:0/1719963473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c0c9000 0x7fae9c0ce570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae900097a0 tx=0x7fae9002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=166a3a1ed673445 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.464+0000 7fae9a7fc640 1 -- 172.21.15.2:0/1719963473 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90037070 con 0x7fae9c0c9000 2024-09-19T05:43:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.464+0000 7fae9a7fc640 1 -- 172.21.15.2:0/1719963473 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae90032440 con 0x7fae9c0c9000 2024-09-19T05:43:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.464+0000 7fae9a7fc640 1 -- 172.21.15.2:0/1719963473 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90032810 con 0x7fae9c0c9000 2024-09-19T05:43:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 -- 172.21.15.2:0/1719963473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c0c9000 msgr2=0x7fae9c0ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 --2- 172.21.15.2:0/1719963473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c0c9000 0x7fae9c0ce570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae900097a0 tx=0x7fae9002a180 comp rx=0 tx=0).stop 2024-09-19T05:43:03.468 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 -- 172.21.15.2:0/1719963473 shutdown_connections 2024-09-19T05:43:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 --2- 172.21.15.2:0/1719963473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c0c9000 0x7fae9c0ce570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 --2- 172.21.15.2:0/1719963473 >> v2:172.21.15.123:3300/0 conn(0x7fae9c152b80 0x7fae9c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 -- 172.21.15.2:0/1719963473 >> 172.21.15.2:0/1719963473 conn(0x7fae9c06c240 msgr2=0x7fae9c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 -- 172.21.15.2:0/1719963473 shutdown_connections 2024-09-19T05:43:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.465+0000 7faea1c8b640 1 -- 172.21.15.2:0/1719963473 wait complete. 2024-09-19T05:43:03.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.466+0000 7faea1c8b640 1 Processor -- start 2024-09-19T05:43:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.466+0000 7faea1c8b640 1 -- start start 2024-09-19T05:43:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7faea1c8b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c071110 0x7fae9c14ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:03.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7faea1c8b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0c9000 0x7fae9c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7faea1c8b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c152b80 0x7fae9c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7faea1c8b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c158cb0 con 0x7fae9c0c9000 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7faea1c8b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c1589d0 con 0x7fae9c071110 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7faea1c8b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fae9c152f60 con 0x7fae9c152b80 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0c9000 0x7fae9c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0c9000 0x7fae9c1568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60410/0 (socket says 172.21.15.2:60410) 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 -- 172.21.15.2:0/917582985 learned_addr learned my addr 172.21.15.2:0/917582985 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9b7fe640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c071110 0x7fae9c14ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9bfff640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c152b80 0x7fae9c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 -- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c152b80 msgr2=0x7fae9c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c152b80 0x7fae9c06e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 -- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c071110 msgr2=0x7fae9c14ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:03.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c071110 0x7fae9c14ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.467+0000 7fae9affd640 1 -- 172.21.15.2:0/917582985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c158330 con 0x7fae9c0c9000 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.468+0000 7fae9bfff640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c152b80 0x7fae9c06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.468+0000 7fae9affd640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0c9000 0x7fae9c1568e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae90035040 tx=0x7fae900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.468+0000 7fae83fff640 1 -- 172.21.15.2:0/917582985 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90037040 con 0x7fae9c0c9000 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.468+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9c148290 con 0x7fae9c0c9000 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.468+0000 7fae83fff640 1 -- 172.21.15.2:0/917582985 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae90030500 con 0x7fae9c0c9000 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.468+0000 7fae83fff640 1 -- 172.21.15.2:0/917582985 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90030910 con 0x7fae9c0c9000 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.468+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9c148b60 con 0x7fae9c0c9000 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.469+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9c0c4020 con 0x7fae9c0c9000 2024-09-19T05:43:03.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.470+0000 7fae83fff640 1 -- 172.21.15.2:0/917582985 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fae90044020 con 0x7fae9c0c9000 2024-09-19T05:43:03.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.470+0000 7fae83fff640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fae8806c2b0 0x7fae8806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:03.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.470+0000 7fae83fff640 1 -- 172.21.15.2:0/917582985 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7fae900ab700 con 0x7fae9c0c9000 2024-09-19T05:43:03.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.470+0000 7fae9b7fe640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fae8806c2b0 0x7fae8806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:03.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.471+0000 7fae9b7fe640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fae8806c2b0 0x7fae8806e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009ad0 tx=0x7fae8c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:03.475 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:03.473+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:03.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.475+0000 7fae83fff640 1 -- 172.21.15.2:0/917582985 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fae9c0c4020 con 0x7fae9c0c9000 2024-09-19T05:43:03.505 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:43:03.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:43:03.530 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:03.530 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:03.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:03.578 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:03.576+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:03.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:03.662+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fae9c0cb740 con 0x7fae9c0c9000 2024-09-19T05:43:03.678 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:03.676+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:03.704 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:03.704 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:03.713 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:03.780 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:03.779+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:03.860 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:03.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:03.913 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:03.911+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:03.977 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:03.981 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:03.979+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:03.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:04.082 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:04.080+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:04.133 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:04.133 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:04.142 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:04.183 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:04.181+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:04.284 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:04.282+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:04.289 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:04.289 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:04.298 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:04.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.347+0000 7fae83fff640 1 -- 172.21.15.2:0/917582985 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v29) ==== 97+0+0 (secure 0 0 0) 0x7fae90079e80 con 0x7fae9c0c9000 2024-09-19T05:43:04.349 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.349+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fae8806c2b0 msgr2=0x7fae8806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.349+0000 7faea1c8b640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fae8806c2b0 0x7fae8806e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009ad0 tx=0x7fae8c008040 comp rx=0 tx=0).stop 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.349+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0c9000 msgr2=0x7fae9c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.349+0000 7faea1c8b640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0c9000 0x7fae9c1568e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae90035040 tx=0x7fae900046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.349+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 shutdown_connections 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.349+0000 7faea1c8b640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fae8806c2b0 0x7fae8806e770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.350+0000 7faea1c8b640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fae9c152b80 0x7fae9c06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.350+0000 7faea1c8b640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0c9000 0x7fae9c1568e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.350+0000 7faea1c8b640 1 --2- 172.21.15.2:0/917582985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae9c071110 0x7fae9c14ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.350+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 >> 172.21.15.2:0/917582985 conn(0x7fae9c06c240 msgr2=0x7fae9c150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:04.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.350+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 shutdown_connections 2024-09-19T05:43:04.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.350+0000 7faea1c8b640 1 -- 172.21.15.2:0/917582985 wait complete. 2024-09-19T05:43:04.405 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:04.403+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:04.450 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:04.450 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:04.458 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:04.500 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:04.498+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:04.587 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:04.585+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:04.607 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:04.607 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:04.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:04.688 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:04.687+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:04.763 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:04.763 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:04.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:04.789 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:04.788+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:04.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:04.881 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:04.882 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:04.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:04.891 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:04.888+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:04.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.914+0000 7f8d1fdd8640 1 Processor -- start 2024-09-19T05:43:04.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.914+0000 7f8d1fdd8640 1 -- start start 2024-09-19T05:43:04.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1fdd8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d180c6800 0x7f8d180c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1fdd8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6be0 0x7f8d180c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1fdd8640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d181514c0 0x7f8d18151950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1fdd8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8d180c7910 con 0x7f8d180c6800 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1fdd8640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8d18151e90 con 0x7f8d181514c0 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1fdd8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8d180c4cc0 con 0x7f8d180c6be0 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d180c6800 0x7f8d180c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d180c6800 0x7f8d180c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60412/0 (socket says 172.21.15.2:60412) 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 -- 172.21.15.2:0/863255989 learned_addr learned my addr 172.21.15.2:0/863255989 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1d373640 1 --2- 172.21.15.2:0/863255989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6be0 0x7f8d180c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1e375640 1 --2- 172.21.15.2:0/863255989 >> v2:172.21.15.123:3300/0 conn(0x7f8d181514c0 0x7f8d18151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 -- 172.21.15.2:0/863255989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6be0 msgr2=0x7f8d180c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 --2- 172.21.15.2:0/863255989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6be0 0x7f8d180c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 -- 172.21.15.2:0/863255989 >> v2:172.21.15.123:3300/0 conn(0x7f8d181514c0 msgr2=0x7f8d18151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 --2- 172.21.15.2:0/863255989 >> v2:172.21.15.123:3300/0 conn(0x7f8d181514c0 0x7f8d18151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.915+0000 7f8d1db74640 1 -- 172.21.15.2:0/863255989 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d180c85e0 con 0x7f8d180c6800 2024-09-19T05:43:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.916+0000 7f8d1db74640 1 --2- 172.21.15.2:0/863255989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6800 0x7f8d180c7df0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d180cb2a0 tx=0x7f8d1402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f2f084714f08b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.916+0000 7f8d1cb72640 1 -- 172.21.15.2:0/863255989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14037070 con 0x7f8d180c6800 2024-09-19T05:43:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.916+0000 7f8d1cb72640 1 -- 172.21.15.2:0/863255989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d14032440 con 0x7f8d180c6800 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.916+0000 7f8d1cb72640 1 -- 172.21.15.2:0/863255989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14032810 con 0x7f8d180c6800 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/863255989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6800 msgr2=0x7f8d180c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/863255989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6800 0x7f8d180c7df0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d180cb2a0 tx=0x7f8d1402a2a0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/863255989 shutdown_connections 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/863255989 >> v2:172.21.15.123:3300/0 conn(0x7f8d181514c0 0x7f8d18151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/863255989 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6be0 0x7f8d180c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/863255989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6800 0x7f8d180c7df0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/863255989 >> 172.21.15.2:0/863255989 conn(0x7f8d18072ad0 msgr2=0x7f8d18072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:04.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/863255989 shutdown_connections 2024-09-19T05:43:04.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.917+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/863255989 wait complete. 2024-09-19T05:43:04.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.918+0000 7f8d1fdd8640 1 Processor -- start 2024-09-19T05:43:04.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.918+0000 7f8d1fdd8640 1 -- start start 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1fdd8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6800 0x7f8d18144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1fdd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6be0 0x7f8d181ee840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1fdd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d181514c0 0x7f8d181eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1fdd8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8d180c8e80 con 0x7f8d180c6be0 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1fdd8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d180c8ba0 con 0x7f8d181514c0 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1fdd8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8d180c4fa0 con 0x7f8d180c6800 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6be0 0x7f8d181ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6be0 0x7f8d181ee840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60416/0 (socket says 172.21.15.2:60416) 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 -- 172.21.15.2:0/2949580543 learned_addr learned my addr 172.21.15.2:0/2949580543 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6800 0x7f8d18144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1e375640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d181514c0 0x7f8d181eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6800 msgr2=0x7f8d18144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d180c6800 0x7f8d18144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d181514c0 msgr2=0x7f8d181eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d181514c0 0x7f8d181eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.919+0000 7f8d1d373640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d180ce2c0 con 0x7f8d180c6be0 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.920+0000 7f8d1d373640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6be0 0x7f8d181ee840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800e890 tx=0x7f8d0800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.920+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08017070 con 0x7f8d180c6be0 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.920+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0800c8e0 con 0x7f8d180c6be0 2024-09-19T05:43:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.920+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d18145720 con 0x7f8d180c6be0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.920+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0800ccb0 con 0x7f8d180c6be0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.920+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d180cda20 con 0x7f8d180c6be0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.921+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8d180c48f0 con 0x7f8d180c6be0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.921+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8d0801a070 con 0x7f8d180c6be0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.922+0000 7f8d0e7fc640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8cfc06c2b0 0x7f8cfc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.922+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f8d0808b620 con 0x7f8d180c6be0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.922+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v29) ==== 54+0+16 (secure 0 0 0) 0x7f8d08091170 con 0x7f8d180c6be0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.922+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8cfc06c2b0 0x7f8cfc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.922+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d180cbd00 0x7f8d18145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.922+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d181ee1c0 con 0x7f8d180cbd00 2024-09-19T05:43:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.922+0000 7f8d1e375640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d180cbd00 0x7f8d18145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.923+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8cfc06c2b0 0x7f8cfc06e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d14034000 tx=0x7f8d1402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.923+0000 7f8d1e375640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d180cbd00 0x7f8d18145a90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.923+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d181ee1c0 con 0x7f8d180cbd00 2024-09-19T05:43:04.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.939+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d180c9f40 con 0x7f8d180cbd00 2024-09-19T05:43:04.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.941+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d180c9f40 con 0x7f8d180cbd00 2024-09-19T05:43:04.943 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:04.943 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:04.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.941+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d180ccef0 0x7f8d18148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.941+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8d181ee3f0 con 0x7f8d180ccef0 2024-09-19T05:43:04.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.941+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d180ccef0 0x7f8d18148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.941+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d180ccef0 0x7f8d18148fd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.942+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d181ee3f0 con 0x7f8d180ccef0 2024-09-19T05:43:04.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.957+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d18144bc0 con 0x7f8d180ccef0 2024-09-19T05:43:04.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.959+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d18144bc0 con 0x7f8d180ccef0 2024-09-19T05:43:04.962 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.959+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d181f2000 0x7f8d18062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.959+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8d181f2460 con 0x7f8d181f2000 2024-09-19T05:43:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.959+0000 7f8d1d373640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d181f2000 0x7f8d18062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.960+0000 7f8d1d373640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d181f2000 0x7f8d18062630 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.960+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d181f2460 con 0x7f8d181f2000 2024-09-19T05:43:04.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.974+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d18062b70 con 0x7f8d181f2000 2024-09-19T05:43:04.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.976+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d18062b70 con 0x7f8d181f2000 2024-09-19T05:43:04.978 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:04.979 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:04.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.976+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1814ef50 0x7f8d1814f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:04.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.976+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8d18062b70 con 0x7f8d1814ef50 2024-09-19T05:43:04.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.976+0000 7f8d1e375640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1814ef50 0x7f8d1814f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:04.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.977+0000 7f8d1e375640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1814ef50 0x7f8d1814f3d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:04.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:04.977+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d18062b70 con 0x7f8d1814ef50 2024-09-19T05:43:04.991 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:04.989+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:05.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.003+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d18062b70 con 0x7f8d1814ef50 2024-09-19T05:43:05.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.005+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d18062b70 con 0x7f8d1814ef50 2024-09-19T05:43:05.007 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:05.008 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:05.008 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:05.009 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:05.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.005+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d1814fc40 0x7f8d1802cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:05.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.005+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8d18062b70 con 0x7f8d1814fc40 2024-09-19T05:43:05.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.006+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d1814fc40 0x7f8d1802cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:05.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.006+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d1814fc40 0x7f8d1802cb50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:05.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.007+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d18062b70 con 0x7f8d1814fc40 2024-09-19T05:43:05.018 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:05.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.020+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d18062b70 con 0x7f8d1814fc40 2024-09-19T05:43:05.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.022+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d18062b70 con 0x7f8d1814fc40 2024-09-19T05:43:05.024 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:05.025 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:05.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.022+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d1802d5a0 0x7f8d18204ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:05.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.022+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8d18062b70 con 0x7f8d1802d5a0 2024-09-19T05:43:05.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.023+0000 7f8d1d373640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d1802d5a0 0x7f8d18204ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:05.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.023+0000 7f8d1d373640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d1802d5a0 0x7f8d18204ac0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:05.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.024+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d18062b70 con 0x7f8d1802d5a0 2024-09-19T05:43:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.050+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d18062b70 con 0x7f8d1802d5a0 2024-09-19T05:43:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.051+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d18062b70 con 0x7f8d1802d5a0 2024-09-19T05:43:05.053 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:05.054 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:05.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.051+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d18205510 0x7f8d18207900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:05.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.051+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8d1802db70 con 0x7f8d18205510 2024-09-19T05:43:05.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.052+0000 7f8d1e375640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d18205510 0x7f8d18207900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:05.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.052+0000 7f8d1e375640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d18205510 0x7f8d18207900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:05.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.053+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d1802db70 con 0x7f8d18205510 2024-09-19T05:43:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.070+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d18207e40 con 0x7f8d18205510 2024-09-19T05:43:05.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.072+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d18207e40 con 0x7f8d18205510 2024-09-19T05:43:05.074 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.072+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d18208560 0x7f8d182089c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.072+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8d1820d8c0 con 0x7f8d18208560 2024-09-19T05:43:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.072+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d18208560 0x7f8d182089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.073+0000 7f8d1db74640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d18208560 0x7f8d182089c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.073+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d1820d8c0 con 0x7f8d18208560 2024-09-19T05:43:05.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.088+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d1820c7c0 con 0x7f8d18208560 2024-09-19T05:43:05.091 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:05.089+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:05.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d0e7fc640 1 -- 172.21.15.2:0/2949580543 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d1820c7c0 con 0x7f8d18208560 2024-09-19T05:43:05.092 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:05.094 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:05.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d180cbd00 msgr2=0x7f8d18145a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d180cbd00 0x7f8d18145a90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d180ccef0 msgr2=0x7f8d18148fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d180ccef0 0x7f8d18148fd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d181f2000 msgr2=0x7f8d18062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d181f2000 0x7f8d18062630 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1814ef50 msgr2=0x7f8d1814f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1814ef50 0x7f8d1814f3d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d1814fc40 msgr2=0x7f8d1802cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1e375640 1 -- 172.21.15.2:0/2949580543 reap_dead start 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d1814fc40 0x7f8d1802cb50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1e375640 1 -- 172.21.15.2:0/2949580543 reap_dead start 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d1802d5a0 msgr2=0x7f8d18204ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d1802d5a0 0x7f8d18204ac0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d18205510 msgr2=0x7f8d18207900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d18205510 0x7f8d18207900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d18208560 msgr2=0x7f8d182089c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d18208560 0x7f8d182089c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.090+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8cfc06c2b0 msgr2=0x7f8cfc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8cfc06c2b0 0x7f8cfc06e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d14034000 tx=0x7f8d1402e000 comp rx=0 tx=0).stop 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6be0 msgr2=0x7f8d181ee840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1fdd8640 1 --2- 172.21.15.2:0/2949580543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d180c6be0 0x7f8d181ee840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800e890 tx=0x7f8d0800ed50 comp rx=0 tx=0).stop 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1e375640 1 -- 172.21.15.2:0/2949580543 reap_dead start 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 shutdown_connections 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 >> 172.21.15.2:0/2949580543 conn(0x7f8d18072ad0 msgr2=0x7f8d18150190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 shutdown_connections 2024-09-19T05:43:05.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:05.091+0000 7f8d1fdd8640 1 -- 172.21.15.2:0/2949580543 wait complete. 2024-09-19T05:43:05.161 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:05.162 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:05.162 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:05.170 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:05.215 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:05.214+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:05.275 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:05.276 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:05.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:05.276 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:05.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.276 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.276 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:05.276 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.297783872, 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:43:05.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.282 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.296908214999998, 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:05.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:43:05.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:43:05.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.289 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.296284034999999, 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:05.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:43:05.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:43:05.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:05.294 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.296163297, 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:43:05.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:43:05.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.296124008, 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:43:05.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:43:05.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:43:05.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.296059507999999, 2024-09-19T05:43:05.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:05.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:43:05.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:43:05.308 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295996089999999, 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:43:05.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:43:05.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:43:05.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295946435000001, 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.315 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:43:05.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:43:05.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:43:05.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295890266000001, 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:43:05.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:43:05.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.328 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295863274999999, 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.329 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:43:05.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:43:05.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:43:05.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:43:05.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295784827999999, 2024-09-19T05:43:05.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:43:05.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.339 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295751147000001, 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295728136000001, 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.345 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:43:05.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:43:05.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:43:05.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295694740999998, 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.351 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:43:05.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:43:05.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295652061999998, 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.364 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295612897000002, 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:43:05.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:05.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:05.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295580869999998, 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:43:05.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295382336999999, 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.374 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:43:05.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:43:05.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.295311159000001, 2024-09-19T05:43:05.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.289095001, 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.382 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:43:05.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.388 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.388 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:05.389 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:05.390 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:05.388+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:05.490 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:05.488+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:05.533 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:05.533 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:05.533 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:05.533 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:05.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:05.590 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:05.589+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:05.643 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:05.644 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:05.644 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:05.644 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:05.644 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:05.652 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:05.693 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:05.691+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:05.792 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:05.791+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:05.796 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:05.796 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:05.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.275676914999998, 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:43:05.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:05.801 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.275318947000002, 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:43:05.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.274818556, 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:43:05.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.810 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.267832323, 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:43:05.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:43:05.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.267374515, 2024-09-19T05:43:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:05.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:43:05.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.244660895000003, 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.820 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.826 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.244165950000003, 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:43:05.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.831 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.682362994000002, 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:43:05.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:43:05.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.681605218000001, 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:43:05.836 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.840 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.681024170999997, 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:43:05.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.680389124999998, 2024-09-19T05:43:05.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:05.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:43:05.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.849 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.674582618999999, 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:43:05.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:43:05.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.673926446999999, 2024-09-19T05:43:05.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:43:05.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.197377772999999, 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:43:05.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.197009459, 2024-09-19T05:43:05.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:05.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:43:05.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:05.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.196630548999998, 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:43:05.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.195933695999997, 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:05.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:43:05.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 41.190149081999998, 2024-09-19T05:43:05.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.881 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.819034068000001, 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:43:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:43:05.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 27.818990984999999, 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:05.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:43:05.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:05.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:05.891 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:05.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:05.893 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:05.891+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:05.994 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:05.992+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:06.035 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:06.036 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:06.036 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:06.036 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:06.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:06.095 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:06.093+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:06.146 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:06.147 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:06.147 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:06.147 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:06.147 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:06.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:06.196 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:06.195+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:06.257 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:06.257 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:06.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:06.257 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:06.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.257 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.279914216000002, 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.261 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:06.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:06.311 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:06.309+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:06.364 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:43:06.371 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:06.371 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:06.371 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:06.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:06.421 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:06.419+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.466+0000 7f7131fef640 1 Processor -- start 2024-09-19T05:43:06.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.466+0000 7f7131fef640 1 -- start start 2024-09-19T05:43:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.467+0000 7f7131fef640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f712c154c20 0x7f712c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.467+0000 7f7131fef640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c155000 0x7f712c073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.467+0000 7f7131fef640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f712c0ccbc0 con 0x7f712c0caf40 2024-09-19T05:43:06.470 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.467+0000 7f7131fef640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f712c0cc600 con 0x7f712c154c20 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.467+0000 7f7131fef640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f712c0cc8e0 con 0x7f712c155000 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.467+0000 7f712b7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f712c154c20 0x7f712c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.468+0000 7f712bfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f712c0caf40 0x7f712c15aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:37448/0 (socket says 172.21.15.2:37448) 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.468+0000 7f712bfff640 1 -- 172.21.15.2:0/278754473 learned_addr learned my addr 172.21.15.2:0/278754473 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.468+0000 7f7123fff640 1 --2- 172.21.15.2:0/278754473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c155000 0x7f712c073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.468+0000 7f712b7fe640 1 -- 172.21.15.2:0/278754473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c155000 msgr2=0x7f712c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.468+0000 7f712b7fe640 1 --2- 172.21.15.2:0/278754473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c155000 0x7f712c073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.468+0000 7f712b7fe640 1 -- 172.21.15.2:0/278754473 >> v1:172.21.15.2:6789/0 conn(0x7f712c0caf40 legacy=0x7f712c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.468+0000 7f712b7fe640 1 -- 172.21.15.2:0/278754473 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c0cbc60 con 0x7f712c154c20 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.469+0000 7f712b7fe640 1 --2- 172.21.15.2:0/278754473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c154c20 0x7f712c0cd9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f711c0097a0 tx=0x7f711c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0748025df5e0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.469+0000 7f712affd640 1 -- 172.21.15.2:0/278754473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c037070 con 0x7f712c154c20 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.469+0000 7f712affd640 1 -- 172.21.15.2:0/278754473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711c032440 con 0x7f712c154c20 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.469+0000 7f712affd640 1 -- 172.21.15.2:0/278754473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c032810 con 0x7f712c154c20 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.469+0000 7f7131fef640 1 -- 172.21.15.2:0/278754473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c154c20 msgr2=0x7f712c0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.469+0000 7f7131fef640 1 --2- 172.21.15.2:0/278754473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c154c20 0x7f712c0cd9e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f711c0097a0 tx=0x7f711c02a180 comp rx=0 tx=0).stop 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.470+0000 7f7131fef640 1 -- 172.21.15.2:0/278754473 shutdown_connections 2024-09-19T05:43:06.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.470+0000 7f7131fef640 1 --2- 172.21.15.2:0/278754473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c155000 0x7f712c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.470+0000 7f7131fef640 1 --2- 172.21.15.2:0/278754473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c154c20 0x7f712c0cd9e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.470+0000 7f7131fef640 1 -- 172.21.15.2:0/278754473 >> 172.21.15.2:0/278754473 conn(0x7f712c06d0e0 msgr2=0x7f712c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:06.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.470+0000 7f7131fef640 1 -- 172.21.15.2:0/278754473 shutdown_connections 2024-09-19T05:43:06.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.470+0000 7f7131fef640 1 -- 172.21.15.2:0/278754473 wait complete. 2024-09-19T05:43:06.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 Processor -- start 2024-09-19T05:43:06.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 -- start start 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c0caf40 0x7f712c1e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c154c20 0x7f712c1e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c155000 0x7f712c1e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f712c15b1f0 con 0x7f712c0caf40 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f712c0cb430 con 0x7f712c155000 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f7131fef640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f712c15b370 con 0x7f712c154c20 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.471+0000 7f712b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c0caf40 0x7f712c1e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c0caf40 0x7f712c1e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60432/0 (socket says 172.21.15.2:60432) 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 -- 172.21.15.2:0/3072964581 learned_addr learned my addr 172.21.15.2:0/3072964581 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f7123fff640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c154c20 0x7f712c1e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712bfff640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c155000 0x7f712c1e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 -- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c154c20 msgr2=0x7f712c1e1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c154c20 0x7f712c1e1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 -- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c155000 msgr2=0x7f712c1e1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c155000 0x7f712c1e1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 -- 172.21.15.2:0/3072964581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c1e7900 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f7123fff640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c154c20 0x7f712c1e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f712b7fe640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c0caf40 0x7f712c1e5a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f711c035040 tx=0x7f711c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f7128ff9640 1 -- 172.21.15.2:0/3072964581 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c037040 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f7128ff9640 1 -- 172.21.15.2:0/3072964581 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711c030500 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c1e7ba0 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f7128ff9640 1 -- 172.21.15.2:0/3072964581 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c030910 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.472+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1e4070 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.473+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c0c5770 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.474+0000 7f7128ff9640 1 -- 172.21.15.2:0/3072964581 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f711c044020 con 0x7f712c0caf40 2024-09-19T05:43:06.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.474+0000 7f7128ff9640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f710c06c2b0 0x7f710c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:06.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.475+0000 7f7128ff9640 1 -- 172.21.15.2:0/3072964581 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f711c0ab5a0 con 0x7f712c0caf40 2024-09-19T05:43:06.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.475+0000 7f7123fff640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f710c06c2b0 0x7f710c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:06.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.475+0000 7f7123fff640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f710c06c2b0 0x7f710c06e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f712c1e7140 tx=0x7f7114009110 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:06.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.479+0000 7f7128ff9640 1 -- 172.21.15.2:0/3072964581 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f711c0796c0 con 0x7f712c0caf40 2024-09-19T05:43:06.499 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:06.498+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:06.524 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:06.524 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:06.524 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:06.524 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:06.524 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:06.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:06.617 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:06.615+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:06.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:06.663+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f712c151350 con 0x7f712c0caf40 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.701000109999999, 2024-09-19T05:43:06.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062186530000000002, 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:06.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974705+0000", 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1270000000000002e-06 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974716+0000", 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1306e-05 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974736+0000", 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9293e-05 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974749+0000", 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3059e-05 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022932900000000001 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975163+0000", 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018497500000000001 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975314+0000", 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151328 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979256+0000", 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039412179999999998 2024-09-19T05:43:06.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571296 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980877+0000", 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0387999999999999e-05 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980919+0000", 2024-09-19T05:43:06.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1334000000000001e-05 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.700877301999999, 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062599709999999996, 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974824+0000", 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2500000000000004e-07 2024-09-19T05:43:06.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974829+0000", 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.66e-06 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974844+0000", 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5160000000000001e-05 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974852+0000", 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9570000000000002e-06 2024-09-19T05:43:06.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975748+0000", 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089547100000000001 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975835+0000", 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7190999999999999e-05 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975997+0000", 2024-09-19T05:43:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016230800000000001 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979261+0000", 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032635310000000001 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980973+0000", 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017117199999999999 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981027+0000", 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4732999999999999e-05 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981083+0000", 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5414999999999997e-05 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:30 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:06.689 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.700816771, 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040710980000000004, 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:06.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974884+0000", 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6400000000000001e-07 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974899+0000", 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0584000000000001e-05 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974917+0000", 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9799999999999998e-06 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974994+0000", 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6612000000000004e-05 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975176+0000", 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018201400000000001 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975321+0000", 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014440899999999999 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977845+0000", 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002524409 2024-09-19T05:43:06.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978841+0000", 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099560399999999998 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978896+0000", 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5075999999999999e-05 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978954+0000", 2024-09-19T05:43:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8743999999999999e-05 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.700748826000002, 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040869110000000004, 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974952+0000", 2024-09-19T05:43:06.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974957+0000", 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7260000000000002e-06 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974966+0000", 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974972+0000", 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0970000000000002e-06 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975780+0000", 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080748199999999999 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975894+0000", 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113867 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976037+0000", 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014333 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978068+0000", 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020306439999999999 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978965+0000", 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089710699999999996 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979003+0000", 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8797999999999997e-05 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979038+0000", 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4668000000000001e-05 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.700 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.700717107999999, 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062404959999999999, 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3900000000000005e-07 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974999+0000", 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5817000000000001e-05 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975006+0000", 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5509999999999996e-06 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975012+0000", 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8189999999999997e-06 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976240+0000", 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012283870000000001 2024-09-19T05:43:06.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976367+0000", 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126465 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976487+0000", 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120548 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:06.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980396+0000", 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003908702 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981127+0000", 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073081199999999995 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981182+0000", 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5921999999999998e-05 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981223+0000", 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0933999999999998e-05 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1e epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:7bf7e7b5:::200.00000006:head by mds.0.4:37 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.699591653999999, 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028842130000000001, 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.706 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976110+0000", 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2869999999999999e-06 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976123+0000", 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3224000000000001e-05 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976140+0000", 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.755e-05 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976157+0000", 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6464e-05 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976275+0000", 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117602 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978993+0000", 2024-09-19T05:43:06.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027180860000000002 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.698384776000001, 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001691115, 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977316+0000", 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209999999999999e-06 2024-09-19T05:43:06.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977337+0000", 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0890000000000002e-05 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977356+0000", 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032999999999999e-05 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977365+0000", 2024-09-19T05:43:06.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7649999999999999e-06 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977425+0000", 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0044999999999999e-05 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977449+0000", 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4006999999999999e-05 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978959+0000", 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015099379999999999 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979006+0000", 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7315999999999999e-05 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:30 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:06.714 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.697964669000001, 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118336, 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:06.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977736+0000", 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.096e-06 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0979999999999999e-06 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977756+0000", 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3891e-05 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977765+0000", 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7579999999999998e-06 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977830+0000", 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5184999999999993e-05 2024-09-19T05:43:06.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977839+0000", 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4770000000000003e-06 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977854+0000", 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4831000000000001e-05 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.718 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:27 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.697727118, 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101439, 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977974+0000", 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6899999999999996e-07 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977979+0000", 2024-09-19T05:43:06.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1399999999999999e-06 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977993+0000", 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4215e-05 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978001+0000", 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.721 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978055+0000", 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3711000000000001e-05 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978062+0000", 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978074+0000", 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2238999999999999e-05 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.722 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.696483068999999, 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024064609999999999, 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:06.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979218+0000", 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979227+0000", 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979239+0000", 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979247+0000", 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8369999999999997e-06 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979307+0000", 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0438e-05 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979326+0000", 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8938999999999999e-05 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981594+0000", 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022679710000000001 2024-09-19T05:43:06.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981623+0000", 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9329999999999999e-05 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.727 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.696182389000001, 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021274990000000001, 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7800000000000001e-07 2024-09-19T05:43:06.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979524+0000", 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8019999999999997e-06 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979534+0000", 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8859999999999999e-06 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:06.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8189999999999992e-06 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979610+0000", 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8433e-05 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979641+0000", 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1233999999999999e-05 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981625+0000", 2024-09-19T05:43:06.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001983964 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981645+0000", 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9783e-05 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:06.732 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.696121417000001, 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020817100000000001, 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979598+0000", 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9072e-05 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979609+0000", 2024-09-19T05:43:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0360000000000001e-05 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979615+0000", 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8700000000000003e-06 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979866+0000", 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000250454 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979884+0000", 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7725000000000001e-05 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981647+0000", 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017632629999999999 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981660+0000", 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3502999999999999e-05 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.736 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.695634997999999, 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016104190000000001, 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980066+0000", 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8400000000000005e-07 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980070+0000", 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8010000000000003e-06 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980077+0000", 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.08e-06 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980082+0000", 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8450000000000002e-06 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980146+0000", 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3589000000000004e-05 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980166+0000", 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0514e-05 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981661+0000", 2024-09-19T05:43:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014949539999999999 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981675+0000", 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.695584784000001, 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018022209999999999, 2024-09-19T05:43:06.741 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980116+0000", 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980126+0000", 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980143+0000", 2024-09-19T05:43:06.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6407999999999999e-05 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980150+0000", 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980385+0000", 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023545400000000001 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980403+0000", 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7652999999999999e-05 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981869+0000", 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014661679999999999 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981917+0000", 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319000000000003e-05 2024-09-19T05:43:06.745 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.694985832, 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000222196, 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980715+0000", 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:43:06.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980721+0000", 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2620000000000004e-06 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980735+0000", 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3756e-05 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980759+0000", 2024-09-19T05:43:06.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980813+0000", 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4249999999999997e-05 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980821+0000", 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7540000000000006e-06 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980936+0000", 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115574 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.69491777, 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030787300000000001, 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980783+0000", 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980786+0000", 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5279999999999999e-06 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980795+0000", 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0790000000000007e-06 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980802+0000", 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4910000000000002e-06 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980962+0000", 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015999100000000001 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980968+0000", 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.874e-06 2024-09-19T05:43:06.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981090+0000", 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012244099999999999 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:29 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.694887864000002, 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041785999999999999, 2024-09-19T05:43:06.754 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1600000000000002e-07 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980816+0000", 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6660000000000001e-06 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980823+0000", 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6789999999999999e-06 2024-09-19T05:43:06.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6480000000000002e-06 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981115+0000", 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028752699999999999 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:06.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981230+0000", 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108546 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:06.758 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.694865233000002, 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011264039999999999, 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:06.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980840+0000", 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1540000000000001e-06 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980848+0000", 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980853+0000", 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9030000000000003e-06 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980907+0000", 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4224999999999997e-05 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980937+0000", 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9852000000000001e-05 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981931+0000", 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099441099999999995 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981961+0000", 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0144000000000001e-05 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.762 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.69434699, 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013984010000000001, 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:06.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981354+0000", 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0160000000000001e-06 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981361+0000", 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981373+0000", 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1439e-05 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981385+0000", 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2041999999999999e-05 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981446+0000", 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1446999999999996e-05 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981469+0000", 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982708+0000", 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012396359999999999 2024-09-19T05:43:06.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982751+0000", 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3198000000000002e-05 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:06.767 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:06.769 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:06.767+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:06.769 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:06.869 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:06.867+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:06.877 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:06.878 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:06.878 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:06.878 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:06.886 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:06.972 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:06.970+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:07.015 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:07.015 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:07.016 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:07.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-19T05:43:07.042 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:07.043 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:07.043 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:07.043 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:07.043 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:07.053 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:07.094 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:07.092+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:07.105 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:07.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.120+0000 7fed1532c640 1 Processor -- start 2024-09-19T05:43:07.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.121+0000 7fed1532c640 1 -- start start 2024-09-19T05:43:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed1532c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10154c20 0x7fed100cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed1532c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fed100caf40 0x7fed1015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed1532c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed100ccbc0 con 0x7fed10155000 2024-09-19T05:43:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed1532c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fed100cc600 con 0x7fed100caf40 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed1532c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fed100cc8e0 con 0x7fed10154c20 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed0effd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10154c20 0x7fed100cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed0f7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fed100caf40 0x7fed1015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed0effd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10154c20 0x7fed100cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:52266/0 (socket says 172.21.15.2:52266) 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.122+0000 7fed0effd640 1 -- 172.21.15.2:0/2438235814 learned_addr learned my addr 172.21.15.2:0/2438235814 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.123+0000 7fed0effd640 1 -- 172.21.15.2:0/2438235814 >> v2:172.21.15.123:3300/0 conn(0x7fed100caf40 msgr2=0x7fed1015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.123+0000 7fed0effd640 1 --2- 172.21.15.2:0/2438235814 >> v2:172.21.15.123:3300/0 conn(0x7fed100caf40 0x7fed1015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.123+0000 7fed0effd640 1 -- 172.21.15.2:0/2438235814 >> v1:172.21.15.123:6789/0 conn(0x7fed10155000 legacy=0x7fed10073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.123+0000 7fed0effd640 1 -- 172.21.15.2:0/2438235814 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed100cbc60 con 0x7fed10154c20 2024-09-19T05:43:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.124+0000 7fed0effd640 1 --2- 172.21.15.2:0/2438235814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10154c20 0x7fed100cd9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed00009870 tx=0x7fed0002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd73a909f2f1ca69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.124+0000 7fed0dffb640 1 -- 172.21.15.2:0/2438235814 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00037070 con 0x7fed10154c20 2024-09-19T05:43:07.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.124+0000 7fed0dffb640 1 -- 172.21.15.2:0/2438235814 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed00032440 con 0x7fed10154c20 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.124+0000 7fed0dffb640 1 -- 172.21.15.2:0/2438235814 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00032810 con 0x7fed10154c20 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 -- 172.21.15.2:0/2438235814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10154c20 msgr2=0x7fed100cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 --2- 172.21.15.2:0/2438235814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10154c20 0x7fed100cd9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed00009870 tx=0x7fed0002a270 comp rx=0 tx=0).stop 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 -- 172.21.15.2:0/2438235814 shutdown_connections 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 --2- 172.21.15.2:0/2438235814 >> v2:172.21.15.123:3300/0 conn(0x7fed100caf40 0x7fed1015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 --2- 172.21.15.2:0/2438235814 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10154c20 0x7fed100cd9e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 -- 172.21.15.2:0/2438235814 >> 172.21.15.2:0/2438235814 conn(0x7fed1006d0e0 msgr2=0x7fed1006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 -- 172.21.15.2:0/2438235814 shutdown_connections 2024-09-19T05:43:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.125+0000 7fed1532c640 1 -- 172.21.15.2:0/2438235814 wait complete. 2024-09-19T05:43:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.127+0000 7fed1532c640 1 Processor -- start 2024-09-19T05:43:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.127+0000 7fed1532c640 1 -- start start 2024-09-19T05:43:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed1532c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed100caf40 0x7fed100b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed1532c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed10154c20 0x7fed100cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed1532c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10155000 0x7fed101f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed0effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed100caf40 0x7fed100b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed0e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed10154c20 0x7fed100cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed0effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed100caf40 0x7fed100b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:48414/0 (socket says 172.21.15.2:48414) 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed0e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed10154c20 0x7fed100cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60442/0 (socket says 172.21.15.2:60442) 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed0effd640 1 -- 172.21.15.2:0/3201595915 learned_addr learned my addr 172.21.15.2:0/3201595915 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed10073a50 con 0x7fed10154c20 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed100cb430 con 0x7fed100caf40 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.128+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fed10073bd0 con 0x7fed10155000 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fed0f7fe640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10155000 0x7fed101f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fed0effd640 1 -- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10155000 msgr2=0x7fed101f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fed0effd640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10155000 0x7fed101f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fed0effd640 1 -- 172.21.15.2:0/3201595915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed10154c20 msgr2=0x7fed100cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fed0effd640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed10154c20 0x7fed100cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fed0effd640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed101f4400 con 0x7fed100caf40 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fed0effd640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed100caf40 0x7fed100b59f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed00009990 tx=0x7fed000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.129+0000 7fecf77fe640 1 -- 172.21.15.2:0/3201595915 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00037040 con 0x7fed100caf40 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.130+0000 7fecf77fe640 1 -- 172.21.15.2:0/3201595915 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed00042070 con 0x7fed100caf40 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.130+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed101f46e0 con 0x7fed100caf40 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.130+0000 7fecf77fe640 1 -- 172.21.15.2:0/3201595915 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed00030940 con 0x7fed100caf40 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.130+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed101f4ca0 con 0x7fed100caf40 2024-09-19T05:43:07.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.131+0000 7fecf77fe640 1 -- 172.21.15.2:0/3201595915 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fed00030ae0 con 0x7fed100caf40 2024-09-19T05:43:07.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.131+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed100c5770 con 0x7fed100caf40 2024-09-19T05:43:07.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.132+0000 7fecf77fe640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecfc06c2e0 0x7fecfc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.132+0000 7fecf77fe640 1 -- 172.21.15.2:0/3201595915 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fed000abb20 con 0x7fed100caf40 2024-09-19T05:43:07.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.132+0000 7fed0e7fc640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecfc06c2e0 0x7fecfc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.137 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.132+0000 7fed0e7fc640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecfc06c2e0 0x7fecfc06e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed10073710 tx=0x7fed04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.137+0000 7fecf77fe640 1 -- 172.21.15.2:0/3201595915 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fed00079c30 con 0x7fed100caf40 2024-09-19T05:43:07.172 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:07.170+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:07.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.227+0000 7fe97f882640 1 Processor -- start 2024-09-19T05:43:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.227+0000 7fe97f882640 1 -- start start 2024-09-19T05:43:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97f882640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9780cd4e0 0x7fe9780b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97f882640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe97815aa90 0x7fe9780b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97f882640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9780cd8c0 0x7fe97815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97f882640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe97815eec0 con 0x7fe97815aa90 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97f882640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe97815ebe0 con 0x7fe9780cd8c0 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97f882640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9780b2e00 con 0x7fe9780cd4e0 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97ce1d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe97815aa90 0x7fe9780b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.228+0000 7fe97d61e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9780cd4e0 0x7fe9780b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97d61e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9780cd4e0 0x7fe9780b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:52286/0 (socket says 172.21.15.2:52286) 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97d61e640 1 -- 172.21.15.2:0/1565370115 learned_addr learned my addr 172.21.15.2:0/1565370115 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:07.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97de1f640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9780cd8c0 0x7fe97815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97d61e640 1 -- 172.21.15.2:0/1565370115 >> v2:172.21.15.123:3300/0 conn(0x7fe9780cd8c0 msgr2=0x7fe97815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97d61e640 1 --2- 172.21.15.2:0/1565370115 >> v2:172.21.15.123:3300/0 conn(0x7fe9780cd8c0 0x7fe97815df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97d61e640 1 -- 172.21.15.2:0/1565370115 >> v2:172.21.15.2:3300/0 conn(0x7fe97815aa90 msgr2=0x7fe9780b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97d61e640 1 --2- 172.21.15.2:0/1565370115 >> v2:172.21.15.2:3300/0 conn(0x7fe97815aa90 0x7fe9780b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.229+0000 7fe97d61e640 1 -- 172.21.15.2:0/1565370115 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9780b30e0 con 0x7fe9780cd4e0 2024-09-19T05:43:07.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.230+0000 7fe97d61e640 1 --2- 172.21.15.2:0/1565370115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9780cd4e0 0x7fe9780b3bc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe97815edf0 tx=0x7fe97402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263a1711c6994a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.230+0000 7fe96ffff640 1 -- 172.21.15.2:0/1565370115 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974037070 con 0x7fe9780cd4e0 2024-09-19T05:43:07.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.230+0000 7fe96ffff640 1 -- 172.21.15.2:0/1565370115 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe974032440 con 0x7fe9780cd4e0 2024-09-19T05:43:07.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.230+0000 7fe96ffff640 1 -- 172.21.15.2:0/1565370115 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974032810 con 0x7fe9780cd4e0 2024-09-19T05:43:07.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 -- 172.21.15.2:0/1565370115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9780cd4e0 msgr2=0x7fe9780b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 --2- 172.21.15.2:0/1565370115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9780cd4e0 0x7fe9780b3bc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe97815edf0 tx=0x7fe97402a1c0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 -- 172.21.15.2:0/1565370115 shutdown_connections 2024-09-19T05:43:07.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 --2- 172.21.15.2:0/1565370115 >> v2:172.21.15.123:3300/0 conn(0x7fe9780cd8c0 0x7fe97815df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 --2- 172.21.15.2:0/1565370115 >> v2:172.21.15.2:3300/0 conn(0x7fe97815aa90 0x7fe9780b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 --2- 172.21.15.2:0/1565370115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9780cd4e0 0x7fe9780b3bc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 -- 172.21.15.2:0/1565370115 >> 172.21.15.2:0/1565370115 conn(0x7fe978067b30 msgr2=0x7fe978067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:07.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 -- 172.21.15.2:0/1565370115 shutdown_connections 2024-09-19T05:43:07.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.231+0000 7fe97f882640 1 -- 172.21.15.2:0/1565370115 wait complete. 2024-09-19T05:43:07.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.232+0000 7fe97f882640 1 Processor -- start 2024-09-19T05:43:07.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.232+0000 7fe97f882640 1 -- start start 2024-09-19T05:43:07.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97f882640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9780cd4e0 0x7fe978157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97f882640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9780cd8c0 0x7fe97815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97f882640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe97815aa90 0x7fe97815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97f882640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe97815e470 con 0x7fe9780cd4e0 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97f882640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9780b36a0 con 0x7fe9780cd8c0 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97f882640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe97815e5f0 con 0x7fe97815aa90 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9780cd4e0 0x7fe978157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9780cd4e0 0x7fe978157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60458/0 (socket says 172.21.15.2:60458) 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 -- 172.21.15.2:0/3409861670 learned_addr learned my addr 172.21.15.2:0/3409861670 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97de1f640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe97815aa90 0x7fe97815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9780cd8c0 0x7fe97815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe97815aa90 msgr2=0x7fe97815cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe97815aa90 0x7fe97815cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9780cd8c0 msgr2=0x7fe97815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9780cd8c0 0x7fe97815caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97d61e640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97814e5f0 con 0x7fe9780cd4e0 2024-09-19T05:43:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.233+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9780cd8c0 0x7fe97815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.234+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9780cd4e0 0x7fe978157770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe974000c00 tx=0x7fe9740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.234+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974041020 con 0x7fe9780cd4e0 2024-09-19T05:43:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.234+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97403d620 con 0x7fe9780cd4e0 2024-09-19T05:43:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.234+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9781f44a0 con 0x7fe9780cd4e0 2024-09-19T05:43:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.234+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974037040 con 0x7fe9780cd4e0 2024-09-19T05:43:07.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.234+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe978158d00 con 0x7fe9780cd4e0 2024-09-19T05:43:07.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.235+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe97815df30 con 0x7fe9780cd4e0 2024-09-19T05:43:07.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.236+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe974044020 con 0x7fe9780cd4e0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.236+0000 7fe96dffb640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe95c06c2b0 0x7fe95c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.236+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fe9740abb00 con 0x7fe9780cd4e0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.237+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe95c06c2b0 0x7fe95c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.237+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v29) ==== 54+0+16 (secure 0 0 0) 0x7fe97404b030 con 0x7fe9780cd4e0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.237+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe978150ef0 0x7fe9781f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.237+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9780cb3c0 con 0x7fe978150ef0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.237+0000 7fe97de1f640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe978150ef0 0x7fe9781f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.237+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe95c06c2b0 0x7fe95c06e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9780b5f30 tx=0x7fe968008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.237+0000 7fe97de1f640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe978150ef0 0x7fe9781f3850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.238+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9780cb3c0 con 0x7fe978150ef0 2024-09-19T05:43:07.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.252+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe978068160 con 0x7fe978150ef0 2024-09-19T05:43:07.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.254+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe978068160 con 0x7fe978150ef0 2024-09-19T05:43:07.256 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:07.257 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:07.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.254+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe9781f02e0 0x7fe9781f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.254+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe9780cfec0 con 0x7fe9781f02e0 2024-09-19T05:43:07.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.254+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe9781f02e0 0x7fe9781f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.257 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:07.257 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:07.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.255+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe9781f02e0 0x7fe9781f0740 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.255+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9780cfec0 con 0x7fe9781f02e0 2024-09-19T05:43:07.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:07.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.270+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe9781500f0 con 0x7fe9781f02e0 2024-09-19T05:43:07.273 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:07.271+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:07.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.272+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe9781500f0 con 0x7fe9781f02e0 2024-09-19T05:43:07.275 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:07.276 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:07.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.273+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe978062630 0x7fe978062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.273+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe978062f90 con 0x7fe978062630 2024-09-19T05:43:07.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.273+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe978062630 0x7fe978062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.273+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe978062630 0x7fe978062a50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.274+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe978062f90 con 0x7fe978062630 2024-09-19T05:43:07.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.290+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe978062f90 con 0x7fe978062630 2024-09-19T05:43:07.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.292+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe978062f90 con 0x7fe978062630 2024-09-19T05:43:07.295 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:07.295 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:07.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.293+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe97806ce80 0x7fe97806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.293+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe97806d800 con 0x7fe97806ce80 2024-09-19T05:43:07.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.293+0000 7fe97de1f640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe97806ce80 0x7fe97806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.293+0000 7fe97de1f640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe97806ce80 0x7fe97806d2c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.294+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe97806d800 con 0x7fe97806ce80 2024-09-19T05:43:07.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.322+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe97802c910 con 0x7fe97806ce80 2024-09-19T05:43:07.327 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.325+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe97802c910 con 0x7fe97806ce80 2024-09-19T05:43:07.327 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:07.328 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:07.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.325+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe97802d3f0 0x7fe978204bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.325+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe97806d800 con 0x7fe97802d3f0 2024-09-19T05:43:07.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.326+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe97802d3f0 0x7fe978204bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.326+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe97802d3f0 0x7fe978204bd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.327+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe97806d800 con 0x7fe97802d3f0 2024-09-19T05:43:07.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.354+0000 7f7128ff9640 1 -- 172.21.15.2:0/3072964581 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v30) ==== 91+0+0 (secure 0 0 0) 0x7f711c07d9c0 con 0x7f712c0caf40 2024-09-19T05:43:07.357 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:43:07.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.357+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe97806d800 con 0x7fe97802d3f0 2024-09-19T05:43:07.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f710c06c2b0 msgr2=0x7f710c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f710c06c2b0 0x7f710c06e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f712c1e7140 tx=0x7f7114009110 comp rx=0 tx=0).stop 2024-09-19T05:43:07.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c0caf40 msgr2=0x7f712c1e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c0caf40 0x7f712c1e5a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f711c035040 tx=0x7f711c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 shutdown_connections 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f710c06c2b0 0x7f710c06e770 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f712c1e7140 tx=0x7f7114009110 comp rx=0 tx=0).stop 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f712c155000 0x7f712c1e1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f712c154c20 0x7f712c1e1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 --2- 172.21.15.2:0/3072964581 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f712c0caf40 0x7f712c1e5a10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 >> 172.21.15.2:0/3072964581 conn(0x7f712c06d0e0 msgr2=0x7f712c0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 shutdown_connections 2024-09-19T05:43:07.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.358+0000 7f7131fef640 1 -- 172.21.15.2:0/3072964581 wait complete. 2024-09-19T05:43:07.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.360+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe97806d800 con 0x7fe97802d3f0 2024-09-19T05:43:07.362 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:07.362 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:07.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.360+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe97802d850 0x7fe978207630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.360+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe97802c890 con 0x7fe97802d850 2024-09-19T05:43:07.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.360+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe97802d850 0x7fe978207630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.361+0000 7fe97ce1d640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe97802d850 0x7fe978207630 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.362+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe97802c890 con 0x7fe97802d850 2024-09-19T05:43:07.373 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:07.371+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:07.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.392+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe97802cae0 con 0x7fe97802d850 2024-09-19T05:43:07.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.393+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe97802cae0 con 0x7fe97802d850 2024-09-19T05:43:07.396 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:07.396 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:07.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.394+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe978208080 0x7fe97820a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.394+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe97820d240 con 0x7fe978208080 2024-09-19T05:43:07.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.394+0000 7fe97de1f640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe978208080 0x7fe97820a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.395+0000 7fe97de1f640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe978208080 0x7fe97820a470 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.395+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe97820d240 con 0x7fe978208080 2024-09-19T05:43:07.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.414+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd out", "ids": ["1"]} v 0) -- 0x7fed1014ea70 con 0x7fed100caf40 2024-09-19T05:43:07.417 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-19T05:43:07.415+0000 7fe7b68ac640 -1 mon.a@0(leader).osd e30 definitely_dead 0 2024-09-19T05:43:07.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.427+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe97820a9b0 con 0x7fe978208080 2024-09-19T05:43:07.429 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:07.429 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:07.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.429+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe97820a9b0 con 0x7fe978208080 2024-09-19T05:43:07.431 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:07.431 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.429+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe97820b030 0x7fe97820d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.429+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe9782101f0 con 0x7fe97820b030 2024-09-19T05:43:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.429+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe97820b030 0x7fe97820d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.430+0000 7fe97d61e640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe97820b030 0x7fe97820d420 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.431+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9782101f0 con 0x7fe97820b030 2024-09-19T05:43:07.440 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:07.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.447+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe97820d960 con 0x7fe97820b030 2024-09-19T05:43:07.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.449+0000 7fe96dffb640 1 -- 172.21.15.2:0/3409861670 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe97820d960 con 0x7fe97820b030 2024-09-19T05:43:07.452 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:07.453 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:07.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe978150ef0 msgr2=0x7fe9781f3850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe978150ef0 0x7fe9781f3850 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe9781f02e0 msgr2=0x7fe9781f0740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe9781f02e0 0x7fe9781f0740 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe978062630 msgr2=0x7fe978062a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe978062630 0x7fe978062a50 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe97806ce80 msgr2=0x7fe97806d2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe97806ce80 0x7fe97806d2c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe97802d3f0 msgr2=0x7fe978204bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe97802d3f0 0x7fe978204bd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe97802d850 msgr2=0x7fe978207630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe97802d850 0x7fe978207630 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe978208080 msgr2=0x7fe97820a470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe978208080 0x7fe97820a470 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe97820b030 msgr2=0x7fe97820d420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe97820b030 0x7fe97820d420 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe95c06c2b0 msgr2=0x7fe95c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe95c06c2b0 0x7fe95c06e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9780b5f30 tx=0x7fe968008040 comp rx=0 tx=0).stop 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9780cd4e0 msgr2=0x7fe978157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 --2- 172.21.15.2:0/3409861670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9780cd4e0 0x7fe978157770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe974000c00 tx=0x7fe9740046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97de1f640 1 -- 172.21.15.2:0/3409861670 reap_dead start 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97de1f640 1 -- 172.21.15.2:0/3409861670 reap_dead start 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97de1f640 1 -- 172.21.15.2:0/3409861670 reap_dead start 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97de1f640 1 -- 172.21.15.2:0/3409861670 reap_dead start 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97de1f640 1 -- 172.21.15.2:0/3409861670 reap_dead start 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97de1f640 1 -- 172.21.15.2:0/3409861670 reap_dead start 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 shutdown_connections 2024-09-19T05:43:07.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 >> 172.21.15.2:0/3409861670 conn(0x7fe978067b30 msgr2=0x7fe9780cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:07.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 shutdown_connections 2024-09-19T05:43:07.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:07.450+0000 7fe97f882640 1 -- 172.21.15.2:0/3409861670 wait complete. 2024-09-19T05:43:07.474 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:07.473+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:07.559 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:07.559 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:07.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:07.613 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:07.611+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:07.676 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:07.675+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:07.681 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:07.681 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:07.690 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:07.777 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:07.776+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:07.854 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:07.854 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:07.864 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:07.908 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:07.906+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:07.979 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:07.979 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:07.979 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:07.977+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:07.988 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:08.081 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:08.079+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:08.138 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:08.139 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:08.147 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:08.182 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:08.180+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:08.283 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:08.281+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:08.298 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:08.298 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:08.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:08.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.365+0000 7fecf77fe640 1 -- 172.21.15.2:0/3201595915 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["1"]}]=0 marked out osd.1. v31) ==== 87+0+0 (secure 0 0 0) 0x7fed0007df30 con 0x7fed100caf40 2024-09-19T05:43:08.369 INFO:teuthology.orchestra.run.smithi002.stderr:marked out osd.1. 2024-09-19T05:43:08.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecfc06c2e0 msgr2=0x7fecfc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:08.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecfc06c2e0 0x7fecfc06e7a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed10073710 tx=0x7fed04009040 comp rx=0 tx=0).stop 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed100caf40 msgr2=0x7fed100b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed100caf40 0x7fed100b59f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed00009990 tx=0x7fed000046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 shutdown_connections 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecfc06c2e0 0x7fecfc06e7a0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed10073710 tx=0x7fed04009040 comp rx=0 tx=0).stop 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed10155000 0x7fed101f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed10154c20 0x7fed100cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 --2- 172.21.15.2:0/3201595915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed100caf40 0x7fed100b59f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:08.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.369+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 >> 172.21.15.2:0/3201595915 conn(0x7fed1006d0e0 msgr2=0x7fed10067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:08.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.370+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 shutdown_connections 2024-09-19T05:43:08.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:08.370+0000 7fed1532c640 1 -- 172.21.15.2:0/3201595915 wait complete. 2024-09-19T05:43:08.384 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:08.381+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:08.427 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:08.428 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:08.437 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:08.485 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:08.483+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:08.585 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:08.584+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:08.590 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:08.590 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:08.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:08.687 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:08.684+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:08.720 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:08.721 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:08.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:08.814 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:08.811+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:08.875 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:08.876 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:08.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:08.888 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:08.887+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:08.989 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:08.988+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:09.028 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:09.028 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:09.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:09.090 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:09.087+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:09.141 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:09.142 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:09.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:09.192 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:09.189+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:09.292 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:09.290+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:09.293 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:09.294 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:09.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:09.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:43:09.393 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:09.390+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:09.413 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:09.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:09.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:09.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.486+0000 7f6ff1d86640 1 Processor -- start 2024-09-19T05:43:09.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.486+0000 7f6ff1d86640 1 -- start start 2024-09-19T05:43:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6ff1d86640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec067d10 0x7f6fec0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6ff1d86640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6fec15cb90 0x7f6fec0c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6ff1d86640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fec0b4930 con 0x7f6fec0680f0 2024-09-19T05:43:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6ff1d86640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6fec0b4650 con 0x7f6fec15cb90 2024-09-19T05:43:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6ff1d86640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6fec0c90c0 con 0x7f6fec067d10 2024-09-19T05:43:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6feb7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec067d10 0x7f6fec0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6febfff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6fec15cb90 0x7f6fec0c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6febfff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6fec15cb90 0x7f6fec0c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:48448/0 (socket says 172.21.15.2:48448) 2024-09-19T05:43:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6feb7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec067d10 0x7f6fec0b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:52312/0 (socket says 172.21.15.2:52312) 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6febfff640 1 -- 172.21.15.2:0/1571360422 learned_addr learned my addr 172.21.15.2:0/1571360422 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.487+0000 7f6feaffd640 1 --1- 172.21.15.2:0/1571360422 >> v1:172.21.15.123:6789/0 conn(0x7f6fec0680f0 0x7f6fec15c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:54530/0 (socket says 172.21.15.2:54530) 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.488+0000 7f6febfff640 1 -- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec067d10 msgr2=0x7f6fec0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.488+0000 7f6febfff640 1 --2- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec067d10 0x7f6fec0b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.488+0000 7f6febfff640 1 -- 172.21.15.2:0/1571360422 >> v1:172.21.15.123:6789/0 conn(0x7f6fec0680f0 legacy=0x7f6fec15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.488+0000 7f6febfff640 1 -- 172.21.15.2:0/1571360422 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec0c93a0 con 0x7f6fec15cb90 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.488+0000 7f6fea7fc640 1 -- 172.21.15.2:0/1571360422 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789813461 0 0) 0x7f6fec0b4930 con 0x7f6fec0680f0 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.488+0000 7f6feb7fe640 1 --2- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec067d10 0x7f6fec0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.489+0000 7f6febfff640 1 --2- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec15cb90 0x7f6fec0c35b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009e50 tx=0x7f6fe402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db1747c13e7633a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.489+0000 7f6fea7fc640 1 -- 172.21.15.2:0/1571360422 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4036070 con 0x7f6fec15cb90 2024-09-19T05:43:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.489+0000 7f6fea7fc640 1 -- 172.21.15.2:0/1571360422 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe4031600 con 0x7f6fec15cb90 2024-09-19T05:43:09.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.489+0000 7f6fea7fc640 1 -- 172.21.15.2:0/1571360422 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40319d0 con 0x7f6fec15cb90 2024-09-19T05:43:09.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 -- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec15cb90 msgr2=0x7f6fec0c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec15cb90 0x7f6fec0c35b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009e50 tx=0x7f6fe402a710 comp rx=0 tx=0).stop 2024-09-19T05:43:09.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 -- 172.21.15.2:0/1571360422 shutdown_connections 2024-09-19T05:43:09.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec15cb90 0x7f6fec0c35b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/1571360422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec067d10 0x7f6fec0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 -- 172.21.15.2:0/1571360422 >> 172.21.15.2:0/1571360422 conn(0x7f6fec070fc0 msgr2=0x7f6fec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 -- 172.21.15.2:0/1571360422 shutdown_connections 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.490+0000 7f6ff1d86640 1 -- 172.21.15.2:0/1571360422 wait complete. 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.491+0000 7f6ff1d86640 1 Processor -- start 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.491+0000 7f6ff1d86640 1 -- start start 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6ff1d86640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec067d10 0x7f6fec0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6ff1d86640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fec0680f0 0x7f6fec1ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6ff1d86640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec15cb90 0x7f6fec144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6ff1d86640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec0b41e0 con 0x7f6fec0680f0 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6ff1d86640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec0b4060 con 0x7f6fec067d10 2024-09-19T05:43:09.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6ff1d86640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6fec0b4360 con 0x7f6fec15cb90 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6feaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fec0680f0 0x7f6fec1ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6feaffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fec0680f0 0x7f6fec1ef390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60462/0 (socket says 172.21.15.2:60462) 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6feb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec067d10 0x7f6fec0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6feaffd640 1 -- 172.21.15.2:0/893785811 learned_addr learned my addr 172.21.15.2:0/893785811 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.492+0000 7f6febfff640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec15cb90 0x7f6fec144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6feb7fe640 1 -- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec15cb90 msgr2=0x7f6fec144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6feb7fe640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec15cb90 0x7f6fec144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6feb7fe640 1 -- 172.21.15.2:0/893785811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fec0680f0 msgr2=0x7f6fec1ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6feb7fe640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fec0680f0 0x7f6fec1ef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6feb7fe640 1 -- 172.21.15.2:0/893785811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec0c7360 con 0x7f6fec067d10 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6febfff640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec15cb90 0x7f6fec144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6feb7fe640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec067d10 0x7f6fec0b20b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00e8a0 tx=0x7f6fdc00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6fd3fff640 1 -- 172.21.15.2:0/893785811 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc017070 con 0x7f6fec067d10 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6fd3fff640 1 -- 172.21.15.2:0/893785811 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc00c8e0 con 0x7f6fec067d10 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec0c7620 con 0x7f6fec067d10 2024-09-19T05:43:09.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6fd3fff640 1 -- 172.21.15.2:0/893785811 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00ccd0 con 0x7f6fec067d10 2024-09-19T05:43:09.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.493+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec0c7be0 con 0x7f6fec067d10 2024-09-19T05:43:09.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.494+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec0b3a90 con 0x7f6fec067d10 2024-09-19T05:43:09.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.495+0000 7f6fd3fff640 1 -- 172.21.15.2:0/893785811 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6fdc01a070 con 0x7f6fec067d10 2024-09-19T05:43:09.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.495+0000 7f6fd3fff640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fd806c2b0 0x7f6fd806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.495+0000 7f6fd3fff640 1 -- 172.21.15.2:0/893785811 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5734+0+0 (secure 0 0 0) 0x7f6fdc08b4b0 con 0x7f6fec067d10 2024-09-19T05:43:09.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.495+0000 7f6feaffd640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fd806c2b0 0x7f6fd806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.496+0000 7f6feaffd640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fd806c2b0 0x7f6fd806e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0007c30 tx=0x7f6fe0007b80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.500+0000 7f6fd3fff640 1 -- 172.21.15.2:0/893785811 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6fdc059640 con 0x7f6fec067d10 2024-09-19T05:43:09.506 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:09.504+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:09.567 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:09.568 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:09.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.570+0000 7f8230ff8640 1 Processor -- start 2024-09-19T05:43:09.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.570+0000 7f8230ff8640 1 -- start start 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.571+0000 7f8230ff8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.571+0000 7f8230ff8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f822c0cd8c0 0x7f822c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.571+0000 7f8230ff8640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822c15ebe0 con 0x7f822c0cd4e0 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.571+0000 7f8230ff8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f822c15e9b0 con 0x7f822c0cd8c0 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.571+0000 7f8230ff8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f822c15eec0 con 0x7f822c15aa90 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.571+0000 7f822ad9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f822c0cd8c0 0x7f822c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.571+0000 7f822a59c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f822c0cd4e0 0x7f822c0b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:37464/0 (socket says 172.21.15.2:37464) 2024-09-19T05:43:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.572+0000 7f822a59c640 1 -- 172.21.15.2:0/1305828116 learned_addr learned my addr 172.21.15.2:0/1305828116 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.572+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1305828116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.572+0000 7f822ad9d640 1 -- 172.21.15.2:0/1305828116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 msgr2=0x7f822c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.572+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1305828116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.572+0000 7f822ad9d640 1 -- 172.21.15.2:0/1305828116 >> v1:172.21.15.2:6789/0 conn(0x7f822c0cd4e0 legacy=0x7f822c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.572+0000 7f822ad9d640 1 -- 172.21.15.2:0/1305828116 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0b2e00 con 0x7f822c0cd8c0 2024-09-19T05:43:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.572+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1305828116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:09.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.573+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1305828116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c15e470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82240097d0 tx=0x7f822402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3131d1b2b7efcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.573+0000 7f822959a640 1 -- 172.21.15.2:0/1305828116 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224037070 con 0x7f822c0cd8c0 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.573+0000 7f822959a640 1 -- 172.21.15.2:0/1305828116 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822402ad40 con 0x7f822c0cd8c0 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.573+0000 7f822959a640 1 -- 172.21.15.2:0/1305828116 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224032660 con 0x7f822c0cd8c0 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 -- 172.21.15.2:0/1305828116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 msgr2=0x7f822c15e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1305828116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c15e470 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82240097d0 tx=0x7f822402a0b0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 -- 172.21.15.2:0/1305828116 shutdown_connections 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1305828116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c15e470 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1305828116 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 -- 172.21.15.2:0/1305828116 >> 172.21.15.2:0/1305828116 conn(0x7f822c067b30 msgr2=0x7f822c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 -- 172.21.15.2:0/1305828116 shutdown_connections 2024-09-19T05:43:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.574+0000 7f8230ff8640 1 -- 172.21.15.2:0/1305828116 wait complete. 2024-09-19T05:43:09.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.575+0000 7f8230ff8640 1 Processor -- start 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8230ff8640 1 -- start start 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8230ff8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c0cd4e0 0x7f822c0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8230ff8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8230ff8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8230ff8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b36a0 con 0x7f822c0cd8c0 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8230ff8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b33c0 con 0x7f822c0cd4e0 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8230ff8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f822c0b42d0 con 0x7f822c15aa90 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8229d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8229d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c0c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60474/0 (socket says 172.21.15.2:60474) 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f8229d9b640 1 -- 172.21.15.2:0/1691962197 learned_addr learned my addr 172.21.15.2:0/1691962197 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.576+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c0cd4e0 0x7f822c0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8229d9b640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 msgr2=0x7f822c0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f822c15aa90 0x7f822c0c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8229d9b640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c0cd4e0 msgr2=0x7f822c0c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f822c0cd4e0 0x7f822c0c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8229d9b640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0c3ee0 con 0x7f822c0cd8c0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c0c63b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f822000e990 tx=0x7f822000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220017070 con 0x7f822c0cd8c0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822000c9e0 con 0x7f822c0cd8c0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c0c41c0 con 0x7f822c0cd8c0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822000cdb0 con 0x7f822c0cd8c0 2024-09-19T05:43:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.577+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c0c4780 con 0x7f822c0cd8c0 2024-09-19T05:43:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.578+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f822c076430 con 0x7f822c0cd8c0 2024-09-19T05:43:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.579+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f822001a070 con 0x7f822c0cd8c0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.579+0000 7f821affd640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f820c06c2b0 0x7f820c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.580+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5734+0+0 (secure 0 0 0) 0x7f822008b6b0 con 0x7f822c0cd8c0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.580+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f820c06c2b0 0x7f820c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.580+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v32) ==== 54+0+16 (secure 0 0 0) 0x7f822c076430 con 0x7f822c0cd8c0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.580+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f822c1f09c0 0x7f822c0cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.580+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f822c0cc990 con 0x7f822c1f09c0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.580+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f820c06c2b0 0x7f820c06e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f821c0012e0 tx=0x7f821c001810 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.580+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f822c1f09c0 0x7f822c0cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.581+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f822c1f09c0 0x7f822c0cbc30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.581+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c0cc990 con 0x7f822c1f09c0 2024-09-19T05:43:09.596 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:09.594+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:09.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.596+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c0b3bc0 con 0x7f822c1f09c0 2024-09-19T05:43:09.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.598+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c0b3bc0 con 0x7f822c1f09c0 2024-09-19T05:43:09.600 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:09.601 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:09.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.598+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f822c0c4950 0x7f822c1f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.598+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f822c1e6a10 con 0x7f822c0c4950 2024-09-19T05:43:09.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.598+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f822c0c4950 0x7f822c1f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.599+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f822c0c4950 0x7f822c1f1500 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.599+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c1e6a10 con 0x7f822c0c4950 2024-09-19T05:43:09.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.614+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c149900 con 0x7f822c0c4950 2024-09-19T05:43:09.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.616+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c149900 con 0x7f822c0c4950 2024-09-19T05:43:09.618 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:09.618 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:09.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.616+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f822c1f4410 0x7f822c1f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.616+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f822c1f4d70 con 0x7f822c1f4410 2024-09-19T05:43:09.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.616+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f822c1f4410 0x7f822c1f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.616+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f822c1f4410 0x7f822c1f4830 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.617+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c1f4d70 con 0x7f822c1f4410 2024-09-19T05:43:09.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.631+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c0b4aa0 con 0x7f822c1f4410 2024-09-19T05:43:09.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.632+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c0b4aa0 con 0x7f822c1f4410 2024-09-19T05:43:09.634 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.632+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f822c062f00 0x7f822c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.632+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f822c0b4aa0 con 0x7f822c062f00 2024-09-19T05:43:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.633+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f822c062f00 0x7f822c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.633+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f822c062f00 0x7f822c0ce310 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.634+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c0b4aa0 con 0x7f822c062f00 2024-09-19T05:43:09.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.666+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c0b4aa0 con 0x7f822c062f00 2024-09-19T05:43:09.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.668+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c0b4aa0 con 0x7f822c062f00 2024-09-19T05:43:09.670 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.668+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f822c0ced60 0x7f822c02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.668+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f822c0cf1e0 con 0x7f822c0ced60 2024-09-19T05:43:09.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.668+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f822c0ced60 0x7f822c02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.668+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f822c0ced60 0x7f822c02c640 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.669+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c0cf1e0 con 0x7f822c0ced60 2024-09-19T05:43:09.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.683+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c0cf1e0 con 0x7f822c0ced60 2024-09-19T05:43:09.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.685+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c0cf1e0 con 0x7f822c0ced60 2024-09-19T05:43:09.687 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.685+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f822c02d090 0x7f822c02d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.685+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f822c02da30 con 0x7f822c02d090 2024-09-19T05:43:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.685+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f822c02d090 0x7f822c02d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.685+0000 7f8229d9b640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f822c02d090 0x7f822c02d4f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.686+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c02da30 con 0x7f822c02d090 2024-09-19T05:43:09.696 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:09.694+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:09.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.699+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c02da30 con 0x7f822c02d090 2024-09-19T05:43:09.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.701+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c02da30 con 0x7f822c02d090 2024-09-19T05:43:09.703 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:09.704 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:09.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.701+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f822c2077d0 0x7f822c209bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.701+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f822c20ca10 con 0x7f822c2077d0 2024-09-19T05:43:09.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.702+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f822c2077d0 0x7f822c209bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.702+0000 7f822ad9d640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f822c2077d0 0x7f822c209bc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.703+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c20ca10 con 0x7f822c2077d0 2024-09-19T05:43:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.717+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c20a180 con 0x7f822c2077d0 2024-09-19T05:43:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.719+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c20a180 con 0x7f822c2077d0 2024-09-19T05:43:09.721 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.719+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f822c20c000 0x7f822c20c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.719+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f822c20fdb0 con 0x7f822c20c000 2024-09-19T05:43:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.720+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f822c20c000 0x7f822c20c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.720+0000 7f822a59c640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f822c20c000 0x7f822c20c440 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:09.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.721+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f822c20fdb0 con 0x7f822c20c000 2024-09-19T05:43:09.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.721+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f6fec0ca750 con 0x7f6fec067d10 2024-09-19T05:43:09.725 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:09.725 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:09.726 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:09.726 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:09.726 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:09.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:09.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.734+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f822c20ec30 con 0x7f822c20c000 2024-09-19T05:43:09.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.736+0000 7f821affd640 1 -- 172.21.15.2:0/1691962197 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f822c20ec30 con 0x7f822c20c000 2024-09-19T05:43:09.739 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f822c1f09c0 msgr2=0x7f822c0cbc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f822c1f09c0 0x7f822c0cbc30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f822c0c4950 msgr2=0x7f822c1f1500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f822c0c4950 0x7f822c1f1500 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f822c1f4410 msgr2=0x7f822c1f4830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f822c1f4410 0x7f822c1f4830 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f822c062f00 msgr2=0x7f822c0ce310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f822ad9d640 1 -- 172.21.15.2:0/1691962197 reap_dead start 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f822c062f00 0x7f822c0ce310 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f822c0ced60 msgr2=0x7f822c02c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f822c0ced60 0x7f822c02c640 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f822c02d090 msgr2=0x7f822c02d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f822c02d090 0x7f822c02d4f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f822c2077d0 msgr2=0x7f822c209bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f822c2077d0 0x7f822c209bc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f822c20c000 msgr2=0x7f822c20c440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f822c20c000 0x7f822c20c440 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f822ad9d640 1 -- 172.21.15.2:0/1691962197 reap_dead start 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f820c06c2b0 msgr2=0x7f820c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f820c06c2b0 0x7f820c06e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f821c0012e0 tx=0x7f821c001810 comp rx=0 tx=0).stop 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 msgr2=0x7f822c0c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c0c63b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f822000e990 tx=0x7f822000ee50 comp rx=0 tx=0).stop 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 shutdown_connections 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f820c06c2b0 0x7f820c06e770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 --2- 172.21.15.2:0/1691962197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f822c0cd8c0 0x7f822c0c63b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 >> 172.21.15.2:0/1691962197 conn(0x7f822c067b30 msgr2=0x7f822c0cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 shutdown_connections 2024-09-19T05:43:09.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:09.737+0000 7f8230ff8640 1 -- 172.21.15.2:0/1691962197 wait complete. 2024-09-19T05:43:09.818 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:09.816+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 31.901716729, 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:09.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:09.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:09.882 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:09.883 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:09.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:09.898 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:09.896+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:09.998 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:09.996+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:10.032 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:10.033 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:10.033 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:10.033 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:10.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:10.099 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:10.097+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:10.185 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:10.185 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:10.185 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:10.185 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:10.185 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:10.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:10.200 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:10.198+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:10.298 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.320963945000003, 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062186530000000002, 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974705+0000", 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1270000000000002e-06 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974716+0000", 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1306e-05 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974736+0000", 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9293e-05 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974749+0000", 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3059e-05 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022932900000000001 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975163+0000", 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018497500000000001 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975314+0000", 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151328 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979256+0000", 2024-09-19T05:43:10.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039412179999999998 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571296 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980877+0000", 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0387999999999999e-05 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980919+0000", 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1334000000000001e-05 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.320841137000002, 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062599709999999996, 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.304 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974824+0000", 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2500000000000004e-07 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974829+0000", 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.66e-06 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974844+0000", 2024-09-19T05:43:10.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5160000000000001e-05 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974852+0000", 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9570000000000002e-06 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975748+0000", 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089547100000000001 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975835+0000", 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7190999999999999e-05 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975997+0000", 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016230800000000001 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979261+0000", 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032635310000000001 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980973+0000", 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017117199999999999 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981027+0000", 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4732999999999999e-05 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981083+0000", 2024-09-19T05:43:10.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5414999999999997e-05 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:30 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.320780606, 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040710980000000004, 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974884+0000", 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6400000000000001e-07 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974899+0000", 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0584000000000001e-05 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974917+0000", 2024-09-19T05:43:10.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9799999999999998e-06 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974994+0000", 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6612000000000004e-05 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975176+0000", 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018201400000000001 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975321+0000", 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014440899999999999 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977845+0000", 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002524409 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978841+0000", 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099560399999999998 2024-09-19T05:43:10.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978896+0000", 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5075999999999999e-05 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978954+0000", 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8743999999999999e-05 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.320712661000002, 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040869110000000004, 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974952+0000", 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974957+0000", 2024-09-19T05:43:10.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7260000000000002e-06 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974966+0000", 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974972+0000", 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0970000000000002e-06 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975780+0000", 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080748199999999999 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975894+0000", 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113867 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976037+0000", 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014333 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978068+0000", 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020306439999999999 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978965+0000", 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089710699999999996 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979003+0000", 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8797999999999997e-05 2024-09-19T05:43:10.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979038+0000", 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4668000000000001e-05 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.320680942999999, 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062404959999999999, 2024-09-19T05:43:10.321 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3900000000000005e-07 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974999+0000", 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5817000000000001e-05 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975006+0000", 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5509999999999996e-06 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975012+0000", 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8189999999999997e-06 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976240+0000", 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012283870000000001 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976367+0000", 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126465 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:10.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976487+0000", 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120548 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980396+0000", 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003908702 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981127+0000", 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073081199999999995 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981182+0000", 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5921999999999998e-05 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981223+0000", 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0933999999999998e-05 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1e epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:7bf7e7b5:::200.00000006:head by mds.0.4:37 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.319555489000003, 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028842130000000001, 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976110+0000", 2024-09-19T05:43:10.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2869999999999999e-06 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976123+0000", 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3224000000000001e-05 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976140+0000", 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.755e-05 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976157+0000", 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6464e-05 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976275+0000", 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117602 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978993+0000", 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027180860000000002 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.332 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.318348610999998, 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001691115, 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977316+0000", 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209999999999999e-06 2024-09-19T05:43:10.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977337+0000", 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0890000000000002e-05 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977356+0000", 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032999999999999e-05 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977365+0000", 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7649999999999999e-06 2024-09-19T05:43:10.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977425+0000", 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0044999999999999e-05 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977449+0000", 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4006999999999999e-05 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978959+0000", 2024-09-19T05:43:10.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015099379999999999 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979006+0000", 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7315999999999999e-05 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:30 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.317928504000001, 2024-09-19T05:43:10.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118336, 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977736+0000", 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.096e-06 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0979999999999999e-06 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977756+0000", 2024-09-19T05:43:10.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3891e-05 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977765+0000", 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7579999999999998e-06 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977830+0000", 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5184999999999993e-05 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977839+0000", 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4770000000000003e-06 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977854+0000", 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4831000000000001e-05 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.341 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:27 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.317690953000003, 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101439, 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:10.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977974+0000", 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6899999999999996e-07 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977979+0000", 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1399999999999999e-06 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977993+0000", 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4215e-05 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978001+0000", 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:10.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978055+0000", 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3711000000000001e-05 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978062+0000", 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978074+0000", 2024-09-19T05:43:10.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2238999999999999e-05 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.316446904000003, 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024064609999999999, 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979218+0000", 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:10.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979227+0000", 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979239+0000", 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979247+0000", 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8369999999999997e-06 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979307+0000", 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0438e-05 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979326+0000", 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8938999999999999e-05 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981594+0000", 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022679710000000001 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981623+0000", 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9329999999999999e-05 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:10.350 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.316146224000001, 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021274990000000001, 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7800000000000001e-07 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979524+0000", 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8019999999999997e-06 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979534+0000", 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8859999999999999e-06 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8189999999999992e-06 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979610+0000", 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8433e-05 2024-09-19T05:43:10.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979641+0000", 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1233999999999999e-05 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981625+0000", 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001983964 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981645+0000", 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9783e-05 2024-09-19T05:43:10.354 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.316085252000001, 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020817100000000001, 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979598+0000", 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9072e-05 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979609+0000", 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0360000000000001e-05 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979615+0000", 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8700000000000003e-06 2024-09-19T05:43:10.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979866+0000", 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000250454 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979884+0000", 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7725000000000001e-05 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981647+0000", 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017632629999999999 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981660+0000", 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3502999999999999e-05 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:10.359 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.315598833000003, 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016104190000000001, 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980066+0000", 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8400000000000005e-07 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980070+0000", 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8010000000000003e-06 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980077+0000", 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.08e-06 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980082+0000", 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8450000000000002e-06 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980146+0000", 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3589000000000004e-05 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980166+0000", 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0514e-05 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981661+0000", 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014949539999999999 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981675+0000", 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:43:10.363 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.315548618999998, 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018022209999999999, 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980116+0000", 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980126+0000", 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980143+0000", 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6407999999999999e-05 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980150+0000", 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:10.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980385+0000", 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023545400000000001 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980403+0000", 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7652999999999999e-05 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981869+0000", 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014661679999999999 2024-09-19T05:43:10.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981917+0000", 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319000000000003e-05 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.314949667, 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000222196, 2024-09-19T05:43:10.368 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980715+0000", 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980721+0000", 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2620000000000004e-06 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980735+0000", 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3756e-05 2024-09-19T05:43:10.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980759+0000", 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980813+0000", 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4249999999999997e-05 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980821+0000", 2024-09-19T05:43:10.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7540000000000006e-06 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980936+0000", 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115574 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.372 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.314881604999997, 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030787300000000001, 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980783+0000", 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980786+0000", 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5279999999999999e-06 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980795+0000", 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0790000000000007e-06 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980802+0000", 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4910000000000002e-06 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980962+0000", 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015999100000000001 2024-09-19T05:43:10.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980968+0000", 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.874e-06 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981090+0000", 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012244099999999999 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:29 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.314851699000002, 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041785999999999999, 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1600000000000002e-07 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980816+0000", 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6660000000000001e-06 2024-09-19T05:43:10.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980823+0000", 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6789999999999999e-06 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6480000000000002e-06 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981115+0000", 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028752699999999999 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981230+0000", 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108546 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.380 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.314829068000002, 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011264039999999999, 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:10.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980840+0000", 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1540000000000001e-06 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980848+0000", 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980853+0000", 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9030000000000003e-06 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980907+0000", 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4224999999999997e-05 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980937+0000", 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9852000000000001e-05 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981931+0000", 2024-09-19T05:43:10.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099441099999999995 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981961+0000", 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0144000000000001e-05 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 32.314310825, 2024-09-19T05:43:10.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013984010000000001, 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:10.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981354+0000", 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0160000000000001e-06 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981361+0000", 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981373+0000", 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1439e-05 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981385+0000", 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2041999999999999e-05 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981446+0000", 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1446999999999996e-05 2024-09-19T05:43:10.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981469+0000", 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982708+0000", 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012396359999999999 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982751+0000", 2024-09-19T05:43:10.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3198000000000002e-05 2024-09-19T05:43:10.390 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.390 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.390 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.390 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:10.390 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:10.390 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:10.392 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:10.389+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:10.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:10.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.373+0000 7f6fd3fff640 1 -- 172.21.15.2:0/893785811 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v33) ==== 101+0+0 (secure 0 0 0) 0x7f6fdc05d940 con 0x7f6fec067d10 2024-09-19T05:43:10.393 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.375+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fd806c2b0 msgr2=0x7f6fd806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.375+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fd806c2b0 0x7f6fd806e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0007c30 tx=0x7f6fe0007b80 comp rx=0 tx=0).stop 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec067d10 msgr2=0x7f6fec0b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec067d10 0x7f6fec0b20b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc00e8a0 tx=0x7f6fdc00ed60 comp rx=0 tx=0).stop 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 shutdown_connections 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fd806c2b0 0x7f6fd806e770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fec15cb90 0x7f6fec144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fec0680f0 0x7f6fec1ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 --2- 172.21.15.2:0/893785811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fec067d10 0x7f6fec0b20b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 >> 172.21.15.2:0/893785811 conn(0x7f6fec070fc0 msgr2=0x7f6fec150b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 shutdown_connections 2024-09-19T05:43:10.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:10.376+0000 7f6ff1d86640 1 -- 172.21.15.2:0/893785811 wait complete. 2024-09-19T05:43:10.492 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:10.490+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:10.548 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:10.548 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:10.557 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:10.593 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:10.590+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:10.665 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:10.665 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:10.665 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:10.665 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:10.665 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:10.673 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:10.693 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:10.692+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:10.787 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.810979461999999, 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029411070000000001, 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974719+0000", 2024-09-19T05:43:10.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1449e-05 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974732+0000", 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3788000000000001e-05 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974743+0000", 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0831e-05 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974751+0000", 2024-09-19T05:43:10.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1659999999999994e-06 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974913+0000", 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016199899999999999 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975022+0000", 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108857 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:10.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011757200000000001 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976884+0000", 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017442969999999999 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977555+0000", 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067127799999999998 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977604+0000", 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8068999999999997e-05 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977648+0000", 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4801e-05 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.810912944999998, 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034736569999999998, 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974776+0000", 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974783+0000", 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2940000000000002e-06 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974786+0000", 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6959999999999998e-06 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975441+0000", 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065484099999999997 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975514+0000", 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2608999999999995e-05 2024-09-19T05:43:10.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975640+0000", 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125869 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977680+0000", 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020402739999999999 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978154+0000", 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047386900000000003 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978206+0000", 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2250000000000003e-05 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978247+0000", 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369000000000003e-05 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.810878682999999, 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034986209999999999, 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8999999999999998e-07 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974818+0000", 2024-09-19T05:43:10.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6870000000000006e-06 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974826+0000", 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9639999999999999e-06 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974869+0000", 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3380999999999997e-05 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975009+0000", 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013971099999999999 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131235 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:10.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976896+0000", 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001756482 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978210+0000", 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013140090000000001 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978275+0000", 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000004e-05 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978307+0000", 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899000000000001e-05 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.810845086999997, 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035825869999999999, 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:10.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974846+0000", 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974856+0000", 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0701e-05 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974862+0000", 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0669999999999997e-06 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976006+0000", 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001143205 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976080+0000", 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4290999999999997e-05 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976213+0000", 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013278500000000001 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977693+0000", 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014800810000000001 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978354+0000", 2024-09-19T05:43:10.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000660971 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978385+0000", 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1423000000000001e-05 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978424+0000", 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9131000000000002e-05 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.810814651000001, 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037252449999999999, 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974873+0000", 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2200000000000004e-07 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974876+0000", 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.965e-06 2024-09-19T05:43:10.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974882+0000", 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3729999999999998e-06 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039999999999995e-06 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974947+0000", 2024-09-19T05:43:10.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6505000000000002e-05 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975044+0000", 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6933000000000002e-05 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975162+0000", 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117879 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977622+0000", 2024-09-19T05:43:10.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024597519999999999 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978546+0000", 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092433599999999999 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978574+0000", 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7923e-05 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978597+0000", 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3553e-05 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:30 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.810106269999999, 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002124711, 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.816 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975581+0000", 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0470000000000001e-06 2024-09-19T05:43:10.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975592+0000", 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.097e-05 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975603+0000", 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0891e-05 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975611+0000", 2024-09-19T05:43:10.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4569999999999999e-06 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975662+0000", 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1424000000000001e-05 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975687+0000", 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4913999999999999e-05 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977661+0000", 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019738020000000002 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977705+0000", 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4206000000000003e-05 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:24 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:10.820 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.809324007999997, 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012912539999999999, 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:10.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2799999999999995e-07 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976370+0000", 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8369999999999999e-06 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976381+0000", 2024-09-19T05:43:10.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0744000000000001e-05 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976388+0000", 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2320000000000004e-06 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976435+0000", 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6726e-05 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976452+0000", 2024-09-19T05:43:10.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6629999999999998e-05 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977623+0000", 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001170894 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977654+0000", 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1362999999999998e-05 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.809134641999997, 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001404389, 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976553+0000", 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-07 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976558+0000", 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-19T05:43:10.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976568+0000", 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976575+0000", 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976621+0000", 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6136e-05 2024-09-19T05:43:10.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976640+0000", 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8604e-05 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977917+0000", 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00127725 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977956+0000", 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9249000000000003e-05 2024-09-19T05:43:10.828 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.809084362, 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099816800000000002, 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976603+0000", 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976610+0000", 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0210000000000002e-06 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5128e-05 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976638+0000", 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2995e-05 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976709+0000", 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0659000000000002e-05 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977601+0000", 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089194000000000003 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.832 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:25 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.808860095999997, 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001150365, 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976836+0000", 2024-09-19T05:43:10.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3049999999999998e-06 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976845+0000", 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4020000000000002e-06 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976850+0000", 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6890000000000001e-06 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:10.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5437000000000004e-05 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976933+0000", 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640999999999999e-05 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977941+0000", 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001008175 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6264999999999999e-05 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:16 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.837 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.808295893, 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000264987, 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977392+0000", 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3600000000000002e-07 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977396+0000", 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6190000000000002e-06 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977412+0000", 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5919000000000001e-05 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977420+0000", 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977543+0000", 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012284899999999999 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977550+0000", 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977656+0000", 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010567499999999999 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:17 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.807615187000003, 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183666, 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:10.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:43:10.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978076+0000", 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.019e-06 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978089+0000", 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2828e-05 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4389999999999997e-06 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978141+0000", 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5649999999999998e-05 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978149+0000", 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5820000000000004e-06 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978255+0000", 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010655299999999999 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:20 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.807551533000002, 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176971, 2024-09-19T05:43:10.846 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978136+0000", 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2300000000000001e-07 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978139+0000", 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978148+0000", 2024-09-19T05:43:10.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978155+0000", 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0169999999999999e-06 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978198+0000", 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3251999999999999e-05 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978205+0000", 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2250000000000003e-06 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978312+0000", 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010694800000000001 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:21 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:10.850 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.807517214999997, 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000262632, 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978173+0000", 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8169999999999999e-06 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978180+0000", 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4839999999999998e-06 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978185+0000", 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978347+0000", 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016165400000000001 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978351+0000", 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1389999999999997e-06 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978432+0000", 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260999999999993e-05 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-19T05:43:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.807234260999998, 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000148286, 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978453+0000", 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978458+0000", 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978472+0000", 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3733e-05 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978479+0000", 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8040000000000004e-06 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978539+0000", 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0038999999999999e-05 2024-09-19T05:43:10.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978543+0000", 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2699999999999998e-06 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978601+0000", 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7881999999999999e-05 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.858 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.790857925000001, 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001029049, 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994831+0000", 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2299999999999998e-06 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994844+0000", 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.291e-05 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994875+0000", 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0898999999999997e-05 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994899+0000", 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3972999999999999e-05 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994994+0000", 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5068000000000005e-05 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995015+0000", 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1166e-05 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995800+0000", 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078543299999999999 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995858+0000", 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000001e-05 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.862 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.863 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.863 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:10.863 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:10.863 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:10.864 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:10.865 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:10.862+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:10.965 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:10.963+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:11.017 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:11.017 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:11.026 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:11.066 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:11.064+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:11.137 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:11.137 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:11.137 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:11.137 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:11.137 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:11.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:11.190 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:11.188+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:11.258 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.258 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.376325874999999, 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:11.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:11.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:11.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.737241066999999, 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:11.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:43:11.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:43:11.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:11.271 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.737198538000001, 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:43:11.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:43:11.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.737080978999998, 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:11.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:43:11.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:43:11.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.707823644000001, 2024-09-19T05:43:11.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:11.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:43:11.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.707766706000001, 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:11.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:43:11.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:43:11.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.553706974000001, 2024-09-19T05:43:11.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:43:11.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:11.302 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.553634643000002, 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:11.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:43:11.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:43:11.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:11.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.553621094, 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:11.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:43:11.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:11.314 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.148454192999999, 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:11.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:43:11.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.148375596000001, 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:43:11.320 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:11.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:43:11.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:43:11.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.148363658000001, 2024-09-19T05:43:11.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:11.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:43:11.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:43:11.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:11.332 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.148352578000001, 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:11.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:43:11.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.148278013000002, 2024-09-19T05:43:11.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:11.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:43:11.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:43:11.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:43:11.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:11.344 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.136851473, 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:11.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:43:11.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:43:11.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 47.136789331000003, 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:11.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:43:11.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.659565909000001, 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:11.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:43:11.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.659544545000003, 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:43:11.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.659526843000002, 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:43:11.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:11.375 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.659473458000001, 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:11.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:43:11.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.381 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:11.382 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:11.383 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:11.381+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:11.483 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:11.481+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:11.534 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:11.534 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:11.534 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:11.534 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:11.543 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:11.584 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:11.583+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:11.685 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:11.683+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:11.691 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:11.691 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:11.692 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:11.692 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:11.692 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:11.701 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:11.751 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:11.786 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:11.784+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:11.861 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:11.861 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.884042157000003, 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:43:11.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:11.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.883984593999998, 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:43:11.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:43:11.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:11.875 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.883912117999998, 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:43:11.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:43:11.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:11.881 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.883874859999999, 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:43:11.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:43:11.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.883835058000003, 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:43:11.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:43:11.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.883005896999997, 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:43:11.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.882580947999998, 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:43:11.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:43:11.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.878418947, 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013154900000000001, 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3099999999999997e-07 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980460+0000", 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6990000000000001e-06 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980470+0000", 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9420000000000007e-06 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980476+0000", 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0419999999999999e-06 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980510+0000", 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4498000000000001e-05 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980515+0000", 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6539999999999998e-06 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980587+0000", 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2082999999999994e-05 2024-09-19T05:43:11.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.878387959999998, 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:43:11.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:43:11.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:43:11.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:1 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.878114588999999, 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103356, 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.908 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:43:11.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980763+0000", 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6249999999999999e-06 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980770+0000", 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2880000000000004e-06 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980774+0000", 2024-09-19T05:43:11.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6780000000000002e-06 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980803+0000", 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9512000000000001e-05 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980807+0000", 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3610000000000001e-06 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980863+0000", 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6535e-05 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.877807826999998, 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010034, 2024-09-19T05:43:11.912 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981070+0000", 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3730000000000001e-06 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981077+0000", 2024-09-19T05:43:11.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6370000000000002e-06 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981080+0000", 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3419999999999999e-06 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981107+0000", 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6806000000000001e-05 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981111+0000", 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981167+0000", 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6246999999999999e-05 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:43:11.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.875806048000001, 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.920 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.875394288000003, 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:43:11.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:43:11.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:38 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.870741463000002, 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186071, 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988134+0000", 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0999999999999998e-07 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988137+0000", 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3919999999999999e-06 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988147+0000", 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659999999999996e-06 2024-09-19T05:43:11.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988151+0000", 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.797e-06 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988194+0000", 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2911000000000003e-05 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988202+0000", 2024-09-19T05:43:11.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988319+0000", 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117355 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.305010877000001, 2024-09-19T05:43:11.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:11.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:43:11.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:40 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.30327256, 2024-09-19T05:43:11.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178677, 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:11.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555603+0000", 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0699999999999996e-07 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555608+0000", 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2240000000000003e-06 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555620+0000", 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.151e-05 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555626+0000", 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8699999999999997e-06 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555665+0000", 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9356000000000001e-05 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555673+0000", 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4230000000000005e-06 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555781+0000", 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010738699999999999 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.988343922999999, 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:43:11.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:11.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:11.946 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.986258512999999, 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:11.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:43:11.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 18'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@18'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:11.950 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.3815363620000001, 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084064700000000003, 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:11.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477339+0000", 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000002e-07 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477650+0000", 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031168299999999999 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477668+0000", 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7745e-05 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477689+0000", 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0644000000000001e-05 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477763+0000", 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3919999999999997e-05 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477772+0000", 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2609999999999999e-06 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.478179+0000", 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00040692799999999998 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 18'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@18'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.3792043860000001, 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010886400000000001, 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479671+0000", 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7300000000000005e-07 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480001+0000", 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033000999999999999 2024-09-19T05:43:11.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480025+0000", 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4053000000000002e-05 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480044+0000", 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.925e-05 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480142+0000", 2024-09-19T05:43:11.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7955000000000007e-05 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480156+0000", 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3553999999999999e-05 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480759+0000", 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060294500000000002 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:11.959 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:11.959 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:11.960 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:11.965 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:11.958+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.889+0000 7fa53df2c640 1 Processor -- start 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.889+0000 7fa53df2c640 1 -- start start 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa53df2c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa538073940 0x7fa538151590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa53df2c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa538071a10 0x7fa538151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa53df2c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5380c3820 con 0x7fa538073560 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa53df2c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fa5380c35d0 con 0x7fa538073940 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa53df2c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa538072a90 con 0x7fa538071a10 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa536ffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa538073940 0x7fa538151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa5377fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa538073560 0x7fa538157700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:57864/0 (socket says 172.21.15.2:57864) 2024-09-19T05:43:11.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa5377fe640 1 -- 172.21.15.2:0/3061818137 learned_addr learned my addr 172.21.15.2:0/3061818137 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa537fff640 1 --2- 172.21.15.2:0/3061818137 >> v2:172.21.15.123:3300/0 conn(0x7fa538071a10 0x7fa538151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa536ffd640 1 -- 172.21.15.2:0/3061818137 >> v2:172.21.15.123:3300/0 conn(0x7fa538071a10 msgr2=0x7fa538151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa536ffd640 1 --2- 172.21.15.2:0/3061818137 >> v2:172.21.15.123:3300/0 conn(0x7fa538071a10 0x7fa538151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa536ffd640 1 -- 172.21.15.2:0/3061818137 >> v1:172.21.15.2:6789/0 conn(0x7fa538073560 legacy=0x7fa538157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa5367fc640 1 -- 172.21.15.2:0/3061818137 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3165625818 0 0) 0x7fa5380c3820 con 0x7fa538073560 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.890+0000 7fa536ffd640 1 -- 172.21.15.2:0/3061818137 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa538072440 con 0x7fa538073940 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.891+0000 7fa536ffd640 1 --2- 172.21.15.2:0/3061818137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073940 0x7fa538151590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa53806efc0 tx=0x7fa52c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4516e6c3b0b60c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.891+0000 7fa5367fc640 1 -- 172.21.15.2:0/3061818137 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c037070 con 0x7fa538073940 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.891+0000 7fa5367fc640 1 -- 172.21.15.2:0/3061818137 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c032440 con 0x7fa538073940 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.891+0000 7fa5367fc640 1 -- 172.21.15.2:0/3061818137 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c032810 con 0x7fa538073940 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 -- 172.21.15.2:0/3061818137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073940 msgr2=0x7fa538151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:11.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 --2- 172.21.15.2:0/3061818137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073940 0x7fa538151590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa53806efc0 tx=0x7fa52c02a1a0 comp rx=0 tx=0).stop 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 -- 172.21.15.2:0/3061818137 shutdown_connections 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 --2- 172.21.15.2:0/3061818137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073940 0x7fa538151590 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 --2- 172.21.15.2:0/3061818137 >> v2:172.21.15.123:3300/0 conn(0x7fa538071a10 0x7fa538151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 -- 172.21.15.2:0/3061818137 >> 172.21.15.2:0/3061818137 conn(0x7fa5380cc0d0 msgr2=0x7fa5380cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 -- 172.21.15.2:0/3061818137 shutdown_connections 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.892+0000 7fa53df2c640 1 -- 172.21.15.2:0/3061818137 wait complete. 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.894+0000 7fa53df2c640 1 Processor -- start 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.894+0000 7fa53df2c640 1 -- start start 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa53df2c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa538071a10 0x7fa5381ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa53df2c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073560 0x7fa538147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa53df2c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa538073940 0x7fa538153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa53df2c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380c3b00 con 0x7fa538073560 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa53df2c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa53806f100 con 0x7fa538071a10 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa53df2c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa5380c3c80 con 0x7fa538073940 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa536ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073560 0x7fa538147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa536ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073560 0x7fa538147b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48842/0 (socket says 172.21.15.2:48842) 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa536ffd640 1 -- 172.21.15.2:0/1285962263 learned_addr learned my addr 172.21.15.2:0/1285962263 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa538071a10 0x7fa5381ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.895+0000 7fa537fff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa538073940 0x7fa538153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa536ffd640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa538073940 msgr2=0x7fa538153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa536ffd640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa538073940 0x7fa538153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa536ffd640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa538071a10 msgr2=0x7fa5381ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa536ffd640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa538071a10 0x7fa5381ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:11.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa536ffd640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5380c94c0 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa536ffd640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073560 0x7fa538147b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c040020 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c038040 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c037070 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5380c9a10 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.896+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381f1980 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.897+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa5380c4120 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.898+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa52c043020 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.898+0000 7fa51ffff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa52406c2b0 0x7fa52406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.898+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa52c0aba70 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.898+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v34) ==== 54+0+16 (secure 0 0 0) 0x7fa5380c4120 con 0x7fa538073560 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.898+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa52406c2b0 0x7fa52406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.899+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa5381f2460 0x7fa5381f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.899+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa538072f00 con 0x7fa5381f2460 2024-09-19T05:43:11.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.899+0000 7fa537fff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa5381f2460 0x7fa5381f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.899+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa52406c2b0 0x7fa52406e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa528000f60 tx=0x7fa528007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.899+0000 7fa537fff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa5381f2460 0x7fa5381f1bc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.900+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa538072f00 con 0x7fa5381f2460 2024-09-19T05:43:11.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.914+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa5380cc5e0 con 0x7fa5381f2460 2024-09-19T05:43:11.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.916+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa5380cc5e0 con 0x7fa5381f2460 2024-09-19T05:43:11.972 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:11.972 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.916+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa538154160 0x7fa538154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.916+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa538150d80 con 0x7fa538154160 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.916+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa538154160 0x7fa538154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.916+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa538154160 0x7fa538154540 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.917+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa538150d80 con 0x7fa538154160 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.932+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa538148070 con 0x7fa538154160 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.934+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa538148070 con 0x7fa538154160 2024-09-19T05:43:11.973 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:11.974 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:11.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.934+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa53806f920 0x7fa53806fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.934+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa538148220 con 0x7fa53806f920 2024-09-19T05:43:11.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.935+0000 7fa536ffd640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa53806f920 0x7fa53806fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.935+0000 7fa536ffd640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa53806f920 0x7fa53806fd00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.936+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa52c0abe50 con 0x7fa53806f920 2024-09-19T05:43:11.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.950+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa538070240 con 0x7fa53806f920 2024-09-19T05:43:11.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.952+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa538070240 con 0x7fa53806f920 2024-09-19T05:43:11.975 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:11.976 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.952+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa538062f00 0x7fa538145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.952+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa538070240 con 0x7fa538062f00 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.952+0000 7fa537fff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa538062f00 0x7fa538145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.952+0000 7fa537fff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa538062f00 0x7fa538145780 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.953+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa538070240 con 0x7fa538062f00 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.967+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa538063590 con 0x7fa538062f00 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.968+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa538063590 con 0x7fa538062f00 2024-09-19T05:43:11.977 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:11.978 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:11.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.969+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa5381461d0 0x7fa53802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.969+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa538146610 con 0x7fa5381461d0 2024-09-19T05:43:11.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.969+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa5381461d0 0x7fa53802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.970+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa5381461d0 0x7fa53802c640 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.970+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa538146610 con 0x7fa5381461d0 2024-09-19T05:43:11.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.986+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa538146610 con 0x7fa5381461d0 2024-09-19T05:43:11.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.988+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa538146610 con 0x7fa5381461d0 2024-09-19T05:43:11.990 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:11.990 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:11.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.988+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa53802d090 0x7fa53802d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:11.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.988+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa53802dab0 con 0x7fa53802d090 2024-09-19T05:43:11.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.988+0000 7fa536ffd640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa53802d090 0x7fa53802d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:11.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.989+0000 7fa536ffd640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa53802d090 0x7fa53802d4f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:11.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:11.989+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa53802dab0 con 0x7fa53802d090 2024-09-19T05:43:12.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.010+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa53802dab0 con 0x7fa53802d090 2024-09-19T05:43:12.014 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.012+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa53802dab0 con 0x7fa53802d090 2024-09-19T05:43:12.014 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:12.015 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:12.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.012+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa538205110 0x7fa538205550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.012+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa53820a670 con 0x7fa538205110 2024-09-19T05:43:12.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.012+0000 7fa537fff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa538205110 0x7fa538205550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.013+0000 7fa537fff640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa538205110 0x7fa538205550 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:12.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.014+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa53820a670 con 0x7fa538205110 2024-09-19T05:43:12.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.027+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa538205ba0 con 0x7fa538205110 2024-09-19T05:43:12.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.029+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa538205ba0 con 0x7fa538205110 2024-09-19T05:43:12.032 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:12.032 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:12.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.029+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa5382062c0 0x7fa5382066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.029+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa538206db0 con 0x7fa5382062c0 2024-09-19T05:43:12.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.030+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa5382062c0 0x7fa5382066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.030+0000 7fa5377fe640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa5382062c0 0x7fa5382066e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:12.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.031+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa538206db0 con 0x7fa5382062c0 2024-09-19T05:43:12.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.044+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa538206db0 con 0x7fa5382062c0 2024-09-19T05:43:12.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa51ffff640 1 -- 172.21.15.2:0/1285962263 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa538206db0 con 0x7fa5382062c0 2024-09-19T05:43:12.049 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:12.050 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:12.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa5381f2460 msgr2=0x7fa5381f1bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa5381f2460 0x7fa5381f1bc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa538154160 msgr2=0x7fa538154540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa538154160 0x7fa538154540 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa53806f920 msgr2=0x7fa53806fd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa53806f920 0x7fa53806fd00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa538062f00 msgr2=0x7fa538145780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa538062f00 0x7fa538145780 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa5381461d0 msgr2=0x7fa53802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa5381461d0 0x7fa53802c640 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa537fff640 1 -- 172.21.15.2:0/1285962263 reap_dead start 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa53802d090 msgr2=0x7fa53802d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa53802d090 0x7fa53802d4f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa537fff640 1 -- 172.21.15.2:0/1285962263 reap_dead start 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa538205110 msgr2=0x7fa538205550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa538205110 0x7fa538205550 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa5382062c0 msgr2=0x7fa5382066e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa5382062c0 0x7fa5382066e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa537fff640 1 -- 172.21.15.2:0/1285962263 reap_dead start 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa52406c2b0 msgr2=0x7fa52406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa52406c2b0 0x7fa52406e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa528000f60 tx=0x7fa528007d50 comp rx=0 tx=0).stop 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073560 msgr2=0x7fa538147b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073560 0x7fa538147b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa52c000c00 tx=0x7fa52c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 shutdown_connections 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa52406c2b0 0x7fa52406e770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 --2- 172.21.15.2:0/1285962263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa538073560 0x7fa538147b30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 >> 172.21.15.2:0/1285962263 conn(0x7fa5380cc0d0 msgr2=0x7fa5380ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 shutdown_connections 2024-09-19T05:43:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.047+0000 7fa53df2c640 1 -- 172.21.15.2:0/1285962263 wait complete. 2024-09-19T05:43:12.061 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:12.059+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:12.094 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:12.094 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:12.103 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:12.162 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:12.160+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:12.254 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:12.254 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:12.262 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:12.261+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:12.264 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:12.363 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:12.361+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:12.372 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:12.372 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:12.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:12.396 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:43:12.464 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:12.462+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:12.492 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:12.493 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:12.502 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:12.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.508+0000 7fa9ecd5f640 1 Processor -- start 2024-09-19T05:43:12.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.508+0000 7fa9ecd5f640 1 -- start start 2024-09-19T05:43:12.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9ecd5f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070910 0x7fa9e8073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9ecd5f640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa9e8070cf0 0x7fa9e8151170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9ecd5f640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e80c6430 con 0x7fa9e8073300 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9ecd5f640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e815b530 con 0x7fa9e8070cf0 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9ecd5f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e8073d90 con 0x7fa9e8070910 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e6d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa9e8070cf0 0x7fa9e8151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e5d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa9e8073300 0x7fa9e815ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:57870/0 (socket says 172.21.15.2:57870) 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e5d9b640 1 -- 172.21.15.2:0/3938944723 learned_addr learned my addr 172.21.15.2:0/3938944723 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e659c640 1 --2- 172.21.15.2:0/3938944723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070910 0x7fa9e8073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e6d9d640 1 -- 172.21.15.2:0/3938944723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070910 msgr2=0x7fa9e8073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e6d9d640 1 --2- 172.21.15.2:0/3938944723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070910 0x7fa9e8073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e6d9d640 1 -- 172.21.15.2:0/3938944723 >> v1:172.21.15.2:6789/0 conn(0x7fa9e8073300 legacy=0x7fa9e815ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.509+0000 7fa9e6d9d640 1 -- 172.21.15.2:0/3938944723 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8157830 con 0x7fa9e8070cf0 2024-09-19T05:43:12.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.510+0000 7fa9e6d9d640 1 --2- 172.21.15.2:0/3938944723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8070cf0 0x7fa9e8151170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000bfa0 tx=0x7fa9e002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24740648e8c19325 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.510+0000 7fa9e559a640 1 -- 172.21.15.2:0/3938944723 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0039070 con 0x7fa9e8070cf0 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.510+0000 7fa9e559a640 1 -- 172.21.15.2:0/3938944723 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0009540 con 0x7fa9e8070cf0 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.510+0000 7fa9e559a640 1 -- 172.21.15.2:0/3938944723 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0009910 con 0x7fa9e8070cf0 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/3938944723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8070cf0 msgr2=0x7fa9e8151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/3938944723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8070cf0 0x7fa9e8151170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000bfa0 tx=0x7fa9e002c860 comp rx=0 tx=0).stop 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/3938944723 shutdown_connections 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/3938944723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8070cf0 0x7fa9e8151170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/3938944723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070910 0x7fa9e8073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/3938944723 >> 172.21.15.2:0/3938944723 conn(0x7fa9e80cd4d0 msgr2=0x7fa9e80cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/3938944723 shutdown_connections 2024-09-19T05:43:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.511+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/3938944723 wait complete. 2024-09-19T05:43:12.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.512+0000 7fa9ecd5f640 1 Processor -- start 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.512+0000 7fa9ecd5f640 1 -- start start 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9ecd5f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e8070910 0x7fa9e8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9ecd5f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070cf0 0x7fa9e81ef790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9ecd5f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8073300 0x7fa9e81ec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9ecd5f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8157dd0 con 0x7fa9e8073300 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9ecd5f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8157af0 con 0x7fa9e8070910 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9ecd5f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e81585d0 con 0x7fa9e8070cf0 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8073300 0x7fa9e81ec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e6d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8073300 0x7fa9e81ec0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48854/0 (socket says 172.21.15.2:48854) 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e6d9d640 1 -- 172.21.15.2:0/2707726012 learned_addr learned my addr 172.21.15.2:0/2707726012 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e659c640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e8070910 0x7fa9e8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e5d9b640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070cf0 0x7fa9e81ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e6d9d640 1 -- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070cf0 msgr2=0x7fa9e81ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e6d9d640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070cf0 0x7fa9e81ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e6d9d640 1 -- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e8070910 msgr2=0x7fa9e8159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.513+0000 7fa9e6d9d640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e8070910 0x7fa9e8159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9e6d9d640 1 -- 172.21.15.2:0/2707726012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e815a8a0 con 0x7fa9e8073300 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9e659c640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e8070910 0x7fa9e8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9e6d9d640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8073300 0x7fa9e81ec0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000d110 tx=0x7fa9e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9d6ffd640 1 -- 172.21.15.2:0/2707726012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0039050 con 0x7fa9e8073300 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9d6ffd640 1 -- 172.21.15.2:0/2707726012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0045070 con 0x7fa9e8073300 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e81eba70 con 0x7fa9e8073300 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9d6ffd640 1 -- 172.21.15.2:0/2707726012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00408b0 con 0x7fa9e8073300 2024-09-19T05:43:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.514+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e8152fa0 con 0x7fa9e8073300 2024-09-19T05:43:12.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.515+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e8152180 con 0x7fa9e8073300 2024-09-19T05:43:12.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.516+0000 7fa9d6ffd640 1 -- 172.21.15.2:0/2707726012 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa9e0040a50 con 0x7fa9e8073300 2024-09-19T05:43:12.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.516+0000 7fa9d6ffd640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa9c806c2e0 0x7fa9c806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:12.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.516+0000 7fa9d6ffd640 1 -- 172.21.15.2:0/2707726012 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa9e00ae2c0 con 0x7fa9e8073300 2024-09-19T05:43:12.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.517+0000 7fa9e659c640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa9c806c2e0 0x7fa9c806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:12.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.517+0000 7fa9e659c640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa9c806c2e0 0x7fa9c806e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009e90 tx=0x7fa9d8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.521+0000 7fa9d6ffd640 1 -- 172.21.15.2:0/2707726012 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa9e007c3e0 con 0x7fa9e8073300 2024-09-19T05:43:12.565 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:12.564+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:12.604 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:12.605 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:12.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:12.697 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:12.696+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:12.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:12.711+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fa9e80c4240 con 0x7fa9e8073300 2024-09-19T05:43:12.758 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:12.758 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:12.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:12.767 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:12.765+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:12.867 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:12.866+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:12.910 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:12.911 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:12.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:13.003 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:13.002+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:13.063 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.063 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.069 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:13.067+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:13.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:13.170 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:13.168+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:13.213 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.214 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.222 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:13.271 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:13.269+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:13.325 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.325 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:13.372 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:13.370+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:13.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.383+0000 7fa9d6ffd640 1 -- 172.21.15.2:0/2707726012 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v36) ==== 87+0+0 (secure 0 0 0) 0x7fa9e00806e0 con 0x7fa9e8073300 2024-09-19T05:43:13.385 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:43:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa9c806c2e0 msgr2=0x7fa9c806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa9c806c2e0 0x7fa9c806e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009e90 tx=0x7fa9d8009190 comp rx=0 tx=0).stop 2024-09-19T05:43:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8073300 msgr2=0x7fa9e81ec0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8073300 0x7fa9e81ec0d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000d110 tx=0x7fa9e0004a40 comp rx=0 tx=0).stop 2024-09-19T05:43:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 shutdown_connections 2024-09-19T05:43:13.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa9c806c2e0 0x7fa9c806e7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:13.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9e8073300 0x7fa9e81ec0d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:13.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa9e8070cf0 0x7fa9e81ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:13.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 --2- 172.21.15.2:0/2707726012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9e8070910 0x7fa9e8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:13.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 >> 172.21.15.2:0/2707726012 conn(0x7fa9e80cd4d0 msgr2=0x7fa9e8156790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:13.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 shutdown_connections 2024-09-19T05:43:13.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:13.386+0000 7fa9ecd5f640 1 -- 172.21.15.2:0/2707726012 wait complete. 2024-09-19T05:43:13.392 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:13.392 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:13.392 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-19T05:43:13.393 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T05:43:13.393 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T05:43:13.393 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-19T05:43:13.438 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.438 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.446 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:13.473 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:13.471+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:13.504 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:13.504 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "" 2024-09-19T05:43:13.504 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:13.549 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.549 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:13.601 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:13.600+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:13.661 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.661 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.670 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:13.711 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:13.709+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:13.775 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:13.773+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:13.813 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.813 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.821 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:13.907 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:13.905+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:13.965 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:13.965 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:13.973 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:13.976 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:13.974+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:14.062 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:14.077 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:14.075+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:14.084 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:14.084 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:14.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:14.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.163+0000 7fc543b2c640 1 Processor -- start 2024-09-19T05:43:14.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.164+0000 7fc543b2c640 1 -- start start 2024-09-19T05:43:14.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.164+0000 7fc543b2c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0cc2e0 0x7fc53c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.164+0000 7fc543b2c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc53c15bc10 con 0x7fc53c15e200 2024-09-19T05:43:14.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.164+0000 7fc543b2c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc53c0b3100 con 0x7fc53c0ca560 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc543b2c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c15b070 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5410c7640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fc53c15e200 0x7fc53c0cb700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:57874/0 (socket says 172.21.15.2:57874) 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5410c7640 1 -- 172.21.15.2:0/2831487595 learned_addr learned my addr 172.21.15.2:0/2831487595 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5420c9640 1 --2- 172.21.15.2:0/2831487595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0cc2e0 0x7fc53c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5408c6640 1 -- 172.21.15.2:0/2831487595 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631295831 0 0) 0x7fc52c001110 con 0x7fc53c15e200 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5408c6640 1 -- 172.21.15.2:0/2831487595 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc528003520 con 0x7fc53c15e200 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5408c6640 1 -- 172.21.15.2:0/2831487595 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347743214 0 0) 0x7fc53c0b3100 con 0x7fc53c0ca560 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5408c6640 1 -- 172.21.15.2:0/2831487595 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc52c001110 con 0x7fc53c0ca560 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5420c9640 1 -- 172.21.15.2:0/2831487595 >> v1:172.21.15.123:6789/0 conn(0x7fc53c0ca560 legacy=0x7fc53c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5420c9640 1 -- 172.21.15.2:0/2831487595 >> v1:172.21.15.2:6789/0 conn(0x7fc53c15e200 legacy=0x7fc53c0cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.165+0000 7fc5420c9640 1 -- 172.21.15.2:0/2831487595 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c0ca940 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.166+0000 7fc5420c9640 1 --2- 172.21.15.2:0/2831487595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0cc2e0 0x7fc53c0cc770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0b32d0 tx=0x7fc53402a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6adb79d532cb2e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.166+0000 7fc5408c6640 1 -- 172.21.15.2:0/2831487595 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534007740 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.166+0000 7fc5408c6640 1 -- 172.21.15.2:0/2831487595 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5340078e0 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.166+0000 7fc5408c6640 1 -- 172.21.15.2:0/2831487595 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534007cb0 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.167+0000 7fc543b2c640 1 -- 172.21.15.2:0/2831487595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0cc2e0 msgr2=0x7fc53c0cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.167+0000 7fc543b2c640 1 --2- 172.21.15.2:0/2831487595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0cc2e0 0x7fc53c0cc770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0b32d0 tx=0x7fc53402a750 comp rx=0 tx=0).stop 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.167+0000 7fc543b2c640 1 -- 172.21.15.2:0/2831487595 shutdown_connections 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.167+0000 7fc543b2c640 1 --2- 172.21.15.2:0/2831487595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0cc2e0 0x7fc53c0cc770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.167+0000 7fc543b2c640 1 -- 172.21.15.2:0/2831487595 >> 172.21.15.2:0/2831487595 conn(0x7fc53c06d8d0 msgr2=0x7fc53c06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.167+0000 7fc543b2c640 1 -- 172.21.15.2:0/2831487595 shutdown_connections 2024-09-19T05:43:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.167+0000 7fc543b2c640 1 -- 172.21.15.2:0/2831487595 wait complete. 2024-09-19T05:43:14.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.168+0000 7fc543b2c640 1 Processor -- start 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.168+0000 7fc543b2c640 1 -- start start 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc543b2c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0ca560 0x7fc53c0b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc543b2c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc53c0cc2e0 0x7fc53c150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc543b2c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc53c15e200 0x7fc53c0b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc543b2c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c15ed00 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc543b2c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c15ea20 con 0x7fc53c15e200 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc543b2c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c15efe0 con 0x7fc53c0ca560 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc5410c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc53c0cc2e0 0x7fc53c150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc5410c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc53c0cc2e0 0x7fc53c150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48866/0 (socket says 172.21.15.2:48866) 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc5410c7640 1 -- 172.21.15.2:0/1420536625 learned_addr learned my addr 172.21.15.2:0/1420536625 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0ca560 0x7fc53c0b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.169+0000 7fc5420c9640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc53c15e200 0x7fc53c0b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5410c7640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0ca560 msgr2=0x7fc53c0b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5410c7640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc53c0ca560 0x7fc53c0b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5410c7640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc53c15e200 msgr2=0x7fc53c0b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5410c7640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc53c15e200 0x7fc53c0b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5410c7640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c1558b0 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5410c7640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc53c0cc2e0 0x7fc53c150cd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00c760 tx=0x7fc52c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c017070 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52c005ab0 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c007070 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c1f1890 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.170+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c155e30 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.171+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc53c15ac90 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.173+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc52c01a070 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.173+0000 7fc5327fc640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc52006c2b0 0x7fc52006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.173+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc52c08b000 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.173+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v36) ==== 54+0+16 (secure 0 0 0) 0x7fc52c021070 con 0x7fc53c0cc2e0 2024-09-19T05:43:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.173+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc52006c2b0 0x7fc52006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.174+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc53c0cbe80 0x7fc53c0c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.174+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc53c159c00 con 0x7fc53c0cbe80 2024-09-19T05:43:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.174+0000 7fc5420c9640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc53c0cbe80 0x7fc53c0c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.174+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc52006c2b0 0x7fc52006e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc538007fd0 tx=0x7fc538009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.174+0000 7fc5420c9640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc53c0cbe80 0x7fc53c0c86a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.175+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c159c00 con 0x7fc53c0cbe80 2024-09-19T05:43:14.178 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:14.176+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:14.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.205+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c159920 con 0x7fc53c0cbe80 2024-09-19T05:43:14.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.206+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c159920 con 0x7fc53c0cbe80 2024-09-19T05:43:14.209 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:14.209 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:14.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.207+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc53c0b4f70 0x7fc53c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.207+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc53c14c110 con 0x7fc53c0b4f70 2024-09-19T05:43:14.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.207+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc53c0b4f70 0x7fc53c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.207+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc53c0b4f70 0x7fc53c14bbd0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.208+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c14c110 con 0x7fc53c0b4f70 2024-09-19T05:43:14.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.223+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c0cbba0 con 0x7fc53c0b4f70 2024-09-19T05:43:14.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.224+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c0cbba0 con 0x7fc53c0b4f70 2024-09-19T05:43:14.227 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.225+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc53c1f46f0 0x7fc53c1f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.225+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc53c0d1360 con 0x7fc53c1f46f0 2024-09-19T05:43:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.225+0000 7fc5410c7640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc53c1f46f0 0x7fc53c1f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.225+0000 7fc5410c7640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc53c1f46f0 0x7fc53c1f4b10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.226+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c0d1360 con 0x7fc53c1f46f0 2024-09-19T05:43:14.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.239+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c062880 con 0x7fc53c1f46f0 2024-09-19T05:43:14.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.241+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c062880 con 0x7fc53c1f46f0 2024-09-19T05:43:14.243 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.241+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc53c063360 0x7fc53c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.241+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc53c062880 con 0x7fc53c063360 2024-09-19T05:43:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.241+0000 7fc5420c9640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc53c063360 0x7fc53c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.242+0000 7fc5420c9640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc53c063360 0x7fc53c15bdf0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.243+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c062880 con 0x7fc53c063360 2024-09-19T05:43:14.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.267+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c062880 con 0x7fc53c063360 2024-09-19T05:43:14.270 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:14.271 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:14.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.270+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c062880 con 0x7fc53c063360 2024-09-19T05:43:14.272 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.270+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc53c02c640 0x7fc53c02ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.270+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc53c062880 con 0x7fc53c02c640 2024-09-19T05:43:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.270+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc53c02c640 0x7fc53c02ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.271+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc53c02c640 0x7fc53c02ca40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.271+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c062880 con 0x7fc53c02c640 2024-09-19T05:43:14.279 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:14.277+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:14.280 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:14.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.288+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c062880 con 0x7fc53c02c640 2024-09-19T05:43:14.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.290+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c062880 con 0x7fc53c02c640 2024-09-19T05:43:14.292 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.290+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc53c02d490 0x7fc53c207570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.290+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc53c062880 con 0x7fc53c02d490 2024-09-19T05:43:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.290+0000 7fc5410c7640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc53c02d490 0x7fc53c207570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.291+0000 7fc5410c7640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc53c02d490 0x7fc53c207570 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.292+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c062880 con 0x7fc53c02d490 2024-09-19T05:43:14.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.305+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c062880 con 0x7fc53c02d490 2024-09-19T05:43:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.307+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c062880 con 0x7fc53c02d490 2024-09-19T05:43:14.309 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.307+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc53c02d8d0 0x7fc53c209fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.307+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc53c20cda0 con 0x7fc53c02d8d0 2024-09-19T05:43:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.307+0000 7fc5420c9640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc53c02d8d0 0x7fc53c209fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.308+0000 7fc5420c9640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc53c02d8d0 0x7fc53c209fd0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.309+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c20cda0 con 0x7fc53c02d8d0 2024-09-19T05:43:14.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.326+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c20a510 con 0x7fc53c02d8d0 2024-09-19T05:43:14.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.328+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c20a510 con 0x7fc53c02d8d0 2024-09-19T05:43:14.330 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:14.331 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:14.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.328+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc53c20ab90 0x7fc53c20cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:14.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.328+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc53c20fdd0 con 0x7fc53c20ab90 2024-09-19T05:43:14.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.329+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc53c20ab90 0x7fc53c20cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:14.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.329+0000 7fc5418c8640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc53c20ab90 0x7fc53c20cf80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:14.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.330+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53c20fdd0 con 0x7fc53c20ab90 2024-09-19T05:43:14.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.343+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc53c20d540 con 0x7fc53c20ab90 2024-09-19T05:43:14.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.345+0000 7fc5327fc640 1 -- 172.21.15.2:0/1420536625 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc53c20d540 con 0x7fc53c20ab90 2024-09-19T05:43:14.347 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc53c0cbe80 msgr2=0x7fc53c0c86a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc53c0cbe80 0x7fc53c0c86a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc53c0b4f70 msgr2=0x7fc53c14bbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc53c0b4f70 0x7fc53c14bbd0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc53c1f46f0 msgr2=0x7fc53c1f4b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc53c1f46f0 0x7fc53c1f4b10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc5420c9640 1 -- 172.21.15.2:0/1420536625 reap_dead start 2024-09-19T05:43:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc53c063360 msgr2=0x7fc53c15bdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc53c063360 0x7fc53c15bdf0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc53c02c640 msgr2=0x7fc53c02ca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc53c02c640 0x7fc53c02ca40 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc53c02d490 msgr2=0x7fc53c207570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc53c02d490 0x7fc53c207570 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc53c02d8d0 msgr2=0x7fc53c209fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc53c02d8d0 0x7fc53c209fd0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc53c20ab90 msgr2=0x7fc53c20cf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc53c20ab90 0x7fc53c20cf80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc5420c9640 1 -- 172.21.15.2:0/1420536625 reap_dead start 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc52006c2b0 msgr2=0x7fc52006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc52006c2b0 0x7fc52006e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc538007fd0 tx=0x7fc538009040 comp rx=0 tx=0).stop 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc53c0cc2e0 msgr2=0x7fc53c150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:14.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc53c0cc2e0 0x7fc53c150cd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00c760 tx=0x7fc52c00cc20 comp rx=0 tx=0).stop 2024-09-19T05:43:14.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 shutdown_connections 2024-09-19T05:43:14.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc52006c2b0 0x7fc52006e770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 --2- 172.21.15.2:0/1420536625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc53c0cc2e0 0x7fc53c150cd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:14.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 >> 172.21.15.2:0/1420536625 conn(0x7fc53c06d8d0 msgr2=0x7fc53c1588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:14.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 shutdown_connections 2024-09-19T05:43:14.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:14.346+0000 7fc543b2c640 1 -- 172.21.15.2:0/1420536625 wait complete. 2024-09-19T05:43:14.380 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:14.378+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:14.404 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:14.404 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.427677991000003, 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029411070000000001, 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974719+0000", 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1449e-05 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974732+0000", 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3788000000000001e-05 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974743+0000", 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0831e-05 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974751+0000", 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1659999999999994e-06 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974913+0000", 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016199899999999999 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975022+0000", 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108857 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011757200000000001 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976884+0000", 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017442969999999999 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977555+0000", 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067127799999999998 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977604+0000", 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8068999999999997e-05 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977648+0000", 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4801e-05 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.427611474000003, 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034736569999999998, 2024-09-19T05:43:14.410 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974776+0000", 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974783+0000", 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2940000000000002e-06 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974786+0000", 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6959999999999998e-06 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975441+0000", 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065484099999999997 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975514+0000", 2024-09-19T05:43:14.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2608999999999995e-05 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975640+0000", 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125869 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977680+0000", 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020402739999999999 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978154+0000", 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047386900000000003 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978206+0000", 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2250000000000003e-05 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978247+0000", 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369000000000003e-05 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.427577212000003, 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034986209999999999, 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8999999999999998e-07 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974818+0000", 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6870000000000006e-06 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974826+0000", 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9639999999999999e-06 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974869+0000", 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3380999999999997e-05 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975009+0000", 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013971099999999999 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131235 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976896+0000", 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001756482 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978210+0000", 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013140090000000001 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978275+0000", 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000004e-05 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978307+0000", 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899000000000001e-05 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.427543616000001, 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035825869999999999, 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974846+0000", 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974856+0000", 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0701e-05 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974862+0000", 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0669999999999997e-06 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976006+0000", 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001143205 2024-09-19T05:43:14.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976080+0000", 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4290999999999997e-05 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976213+0000", 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013278500000000001 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977693+0000", 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014800810000000001 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978354+0000", 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000660971 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978385+0000", 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1423000000000001e-05 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978424+0000", 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9131000000000002e-05 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.427513179999998, 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037252449999999999, 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974873+0000", 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2200000000000004e-07 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974876+0000", 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.965e-06 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974882+0000", 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3729999999999998e-06 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039999999999995e-06 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974947+0000", 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6505000000000002e-05 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975044+0000", 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6933000000000002e-05 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975162+0000", 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117879 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977622+0000", 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024597519999999999 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978546+0000", 2024-09-19T05:43:14.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092433599999999999 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978574+0000", 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7923e-05 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978597+0000", 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3553e-05 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:30 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.426804799000003, 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002124711, 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:14.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975581+0000", 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0470000000000001e-06 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975592+0000", 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.097e-05 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975603+0000", 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0891e-05 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975611+0000", 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4569999999999999e-06 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975662+0000", 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1424000000000001e-05 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975687+0000", 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4913999999999999e-05 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977661+0000", 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019738020000000002 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977705+0000", 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4206000000000003e-05 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:24 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.426022537000001, 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012912539999999999, 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2799999999999995e-07 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976370+0000", 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8369999999999999e-06 2024-09-19T05:43:14.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976381+0000", 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0744000000000001e-05 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976388+0000", 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2320000000000004e-06 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976435+0000", 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6726e-05 2024-09-19T05:43:14.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976452+0000", 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6629999999999998e-05 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977623+0000", 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001170894 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977654+0000", 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1362999999999998e-05 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.425833171000001, 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001404389, 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976553+0000", 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-07 2024-09-19T05:43:14.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976558+0000", 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976568+0000", 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976575+0000", 2024-09-19T05:43:14.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976621+0000", 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6136e-05 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976640+0000", 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8604e-05 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977917+0000", 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00127725 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977956+0000", 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9249000000000003e-05 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:14.446 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.425782890999997, 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099816800000000002, 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976603+0000", 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976610+0000", 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0210000000000002e-06 2024-09-19T05:43:14.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5128e-05 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976638+0000", 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2995e-05 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976709+0000", 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0659000000000002e-05 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977601+0000", 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089194000000000003 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:25 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:14.450 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.425558625000001, 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001150365, 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976836+0000", 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3049999999999998e-06 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976845+0000", 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4020000000000002e-06 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976850+0000", 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6890000000000001e-06 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5437000000000004e-05 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976933+0000", 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640999999999999e-05 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977941+0000", 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001008175 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6264999999999999e-05 2024-09-19T05:43:14.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:16 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.424994421999997, 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000264987, 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977392+0000", 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3600000000000002e-07 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977396+0000", 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6190000000000002e-06 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977412+0000", 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5919000000000001e-05 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977420+0000", 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-19T05:43:14.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977543+0000", 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012284899999999999 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977550+0000", 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977656+0000", 2024-09-19T05:43:14.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010567499999999999 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:17 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.424313716, 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183666, 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978076+0000", 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.019e-06 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978089+0000", 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2828e-05 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:14.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4389999999999997e-06 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978141+0000", 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5649999999999998e-05 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978149+0000", 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5820000000000004e-06 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978255+0000", 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010655299999999999 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:20 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.424250061999999, 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176971, 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:14.463 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978136+0000", 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2300000000000001e-07 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978139+0000", 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978148+0000", 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978155+0000", 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0169999999999999e-06 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978198+0000", 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3251999999999999e-05 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978205+0000", 2024-09-19T05:43:14.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2250000000000003e-06 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978312+0000", 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010694800000000001 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:21 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.424215744000001, 2024-09-19T05:43:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000262632, 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:14.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978173+0000", 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8169999999999999e-06 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978180+0000", 2024-09-19T05:43:14.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4839999999999998e-06 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978185+0000", 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978347+0000", 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016165400000000001 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978351+0000", 2024-09-19T05:43:14.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1389999999999997e-06 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978432+0000", 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260999999999993e-05 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.423932790000002, 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000148286, 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978453+0000", 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978458+0000", 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978472+0000", 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3733e-05 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978479+0000", 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8040000000000004e-06 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978539+0000", 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0038999999999999e-05 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978543+0000", 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2699999999999998e-06 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978601+0000", 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7881999999999999e-05 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.407556454000002, 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001029049, 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994831+0000", 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2299999999999998e-06 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994844+0000", 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.291e-05 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994875+0000", 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0898999999999997e-05 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994899+0000", 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3972999999999999e-05 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994994+0000", 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5068000000000005e-05 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995015+0000", 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1166e-05 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995800+0000", 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078543299999999999 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995858+0000", 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000001e-05 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.479 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:14.481 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:14.481 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:14.479+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:14.582 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:14.580+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:14.599 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:14.599 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:14.599 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:14.609 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:14.683 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:14.682+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:14.765 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:14.766 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:14.775 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:14.819 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:14.817+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:14.886 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:14.884+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:14.896 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.013907369999998, 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:14.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.374822561999999, 2024-09-19T05:43:14.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:43:14.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.374780033, 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.914 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.374662473999997, 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.915 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:43:14.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:43:14.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.345405139, 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:43:14.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:43:14.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:14.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.345348201, 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:14.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.191288469, 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:43:14.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:43:14.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.191216138000001, 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:43:14.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:43:14.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.191202589, 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:43:14.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.786035687999998, 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:43:14.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.785957091, 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:43:14.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:43:14.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:43:14.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.785945153, 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.962 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:14.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:43:14.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.785934073, 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.973 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.785859508000001, 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:43:14.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:43:14.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.774432967999999, 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:43:14.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.774370826000002, 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:43:14.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:43:14.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:43:14.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.297147404, 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:14.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:43:14.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:43:14.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:14.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.297126040000002, 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:14.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.297108338000001, 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:43:15.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:43:15.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 50.297054953, 2024-09-19T05:43:15.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:43:15.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:43:15.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:43:15.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.014 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:15.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:15.016 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:15.014+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:15.116 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:15.115+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:15.163 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:15.164 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:15.164 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:15.164 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:15.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:15.258 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:15.256+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:15.319 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:15.317+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:15.323 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:15.323 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:15.323 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:15.323 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:15.323 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:15.332 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:15.401 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:43:15.419 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:15.418+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:15.485 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.507575351, 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.486 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:43:15.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:43:15.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.507517788000001, 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:43:15.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:15.497 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.507445312000002, 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.507408054000003, 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:43:15.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.507368251999999, 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:43:15.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.506539091, 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:43:15.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:43:15.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:43:15.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.506114142000001, 2024-09-19T05:43:15.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:43:15.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.501952140999997, 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013154900000000001, 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3099999999999997e-07 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980460+0000", 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6990000000000001e-06 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980470+0000", 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9420000000000007e-06 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980476+0000", 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0419999999999999e-06 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980510+0000", 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4498000000000001e-05 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980515+0000", 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6539999999999998e-06 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980587+0000", 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2082999999999994e-05 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.501921154000001, 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:43:15.525 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:43:15.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:1 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.501647783000003, 2024-09-19T05:43:15.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103356, 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980763+0000", 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6249999999999999e-06 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980770+0000", 2024-09-19T05:43:15.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2880000000000004e-06 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980774+0000", 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6780000000000002e-06 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980803+0000", 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9512000000000001e-05 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980807+0000", 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3610000000000001e-06 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980863+0000", 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6535e-05 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:15.533 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.501341021000002, 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010034, 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:15.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981070+0000", 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3730000000000001e-06 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981077+0000", 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6370000000000002e-06 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981080+0000", 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3419999999999999e-06 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981107+0000", 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6806000000000001e-05 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981111+0000", 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981167+0000", 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6246999999999999e-05 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.499339241999998, 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:43:15.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.498927481999999, 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:15.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:38 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.494274656999998, 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186071, 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988134+0000", 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0999999999999998e-07 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988137+0000", 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3919999999999999e-06 2024-09-19T05:43:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988147+0000", 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659999999999996e-06 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988151+0000", 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.797e-06 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988194+0000", 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2911000000000003e-05 2024-09-19T05:43:15.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988202+0000", 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988319+0000", 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117355 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.928544071000001, 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:43:15.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:43:15.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:43:15.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:40 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.926805754, 2024-09-19T05:43:15.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178677, 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555603+0000", 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0699999999999996e-07 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555608+0000", 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2240000000000003e-06 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555620+0000", 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.151e-05 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555626+0000", 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8699999999999997e-06 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555665+0000", 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9356000000000001e-05 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555673+0000", 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4230000000000005e-06 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555781+0000", 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010738699999999999 2024-09-19T05:43:15.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.611877116999999, 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:43:15.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:43:15.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:15.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.609791706999999, 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:43:15.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:43:15.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 18'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@18'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0050695559999996, 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084064700000000003, 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:15.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477339+0000", 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000002e-07 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477650+0000", 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031168299999999999 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477668+0000", 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7745e-05 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477689+0000", 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0644000000000001e-05 2024-09-19T05:43:15.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477763+0000", 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3919999999999997e-05 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477772+0000", 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2609999999999999e-06 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.478179+0000", 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00040692799999999998 2024-09-19T05:43:15.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 18'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@18'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0027375799999998, 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010886400000000001, 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479671+0000", 2024-09-19T05:43:15.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7300000000000005e-07 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480001+0000", 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033000999999999999 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480025+0000", 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4053000000000002e-05 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480044+0000", 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.925e-05 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480142+0000", 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7955000000000007e-05 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480156+0000", 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3553999999999999e-05 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480759+0000", 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060294500000000002 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.580 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:15.582 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.520+0000 7fd5bac65640 1 Processor -- start 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.520+0000 7fd5bac65640 1 -- start start 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5bac65640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd5b4154c20 0x7fd5b40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5bac65640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4155000 0x7fd5b4073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5bac65640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b40ccbc0 con 0x7fd5b40caf40 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5bac65640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd5b40cc600 con 0x7fd5b4154c20 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5bac65640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b40cc8e0 con 0x7fd5b4155000 2024-09-19T05:43:15.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4155000 0x7fd5b4073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b8a01640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd5b4154c20 0x7fd5b40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b3fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4155000 0x7fd5b4073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:33538/0 (socket says 172.21.15.2:33538) 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b9202640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd5b40caf40 0x7fd5b415aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:43112/0 (socket says 172.21.15.2:43112) 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b3fff640 1 -- 172.21.15.2:0/2521659555 learned_addr learned my addr 172.21.15.2:0/2521659555 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b3fff640 1 -- 172.21.15.2:0/2521659555 >> v2:172.21.15.123:3300/0 conn(0x7fd5b4154c20 msgr2=0x7fd5b40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b3fff640 1 --2- 172.21.15.2:0/2521659555 >> v2:172.21.15.123:3300/0 conn(0x7fd5b4154c20 0x7fd5b40cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b3fff640 1 -- 172.21.15.2:0/2521659555 >> v1:172.21.15.123:6789/0 conn(0x7fd5b40caf40 legacy=0x7fd5b415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b3fff640 1 -- 172.21.15.2:0/2521659555 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b40cbc60 con 0x7fd5b4155000 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.521+0000 7fd5b8a01640 1 --2- 172.21.15.2:0/2521659555 >> v2:172.21.15.123:3300/0 conn(0x7fd5b4154c20 0x7fd5b40cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.522+0000 7fd5b3fff640 1 --2- 172.21.15.2:0/2521659555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4155000 0x7fd5b4073340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009850 tx=0x7fd5a002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338268d49f57c30f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.522+0000 7fd5b37fe640 1 -- 172.21.15.2:0/2521659555 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0004030 con 0x7fd5b4155000 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.522+0000 7fd5b37fe640 1 -- 172.21.15.2:0/2521659555 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a00041d0 con 0x7fd5b4155000 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.522+0000 7fd5b37fe640 1 -- 172.21.15.2:0/2521659555 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0003610 con 0x7fd5b4155000 2024-09-19T05:43:15.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 -- 172.21.15.2:0/2521659555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4155000 msgr2=0x7fd5b4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 --2- 172.21.15.2:0/2521659555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4155000 0x7fd5b4073340 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009850 tx=0x7fd5a002a230 comp rx=0 tx=0).stop 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 -- 172.21.15.2:0/2521659555 shutdown_connections 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 --2- 172.21.15.2:0/2521659555 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4155000 0x7fd5b4073340 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 --2- 172.21.15.2:0/2521659555 >> v2:172.21.15.123:3300/0 conn(0x7fd5b4154c20 0x7fd5b40cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 -- 172.21.15.2:0/2521659555 >> 172.21.15.2:0/2521659555 conn(0x7fd5b406d0e0 msgr2=0x7fd5b406d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 -- 172.21.15.2:0/2521659555 shutdown_connections 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.523+0000 7fd5bac65640 1 -- 172.21.15.2:0/2521659555 wait complete. 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 Processor -- start 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 -- start start 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5b40caf40 0x7fd5b415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4154c20 0x7fd5b40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4155000 0x7fd5b40cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b415b1f0 con 0x7fd5b40caf40 2024-09-19T05:43:15.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b40cb430 con 0x7fd5b4155000 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.525+0000 7fd5bac65640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b415b370 con 0x7fd5b4154c20 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5b40caf40 0x7fd5b415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5b40caf40 0x7fd5b415a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48874/0 (socket says 172.21.15.2:48874) 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 -- 172.21.15.2:0/451617039 learned_addr learned my addr 172.21.15.2:0/451617039 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b3fff640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4154c20 0x7fd5b40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b9202640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4155000 0x7fd5b40cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 -- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4154c20 msgr2=0x7fd5b40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4154c20 0x7fd5b40cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 -- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4155000 msgr2=0x7fd5b40cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4155000 0x7fd5b40cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 -- 172.21.15.2:0/451617039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b415ae10 con 0x7fd5b40caf40 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b8a01640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5b40caf40 0x7fd5b415a8d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800ebe0 tx=0x7fd5a800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b17fa640 1 -- 172.21.15.2:0/451617039 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8017070 con 0x7fd5b40caf40 2024-09-19T05:43:15.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b17fa640 1 -- 172.21.15.2:0/451617039 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a800cc30 con 0x7fd5b40caf40 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b4159630 con 0x7fd5b40caf40 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5b17fa640 1 -- 172.21.15.2:0/451617039 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8013470 con 0x7fd5b40caf40 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.526+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b4159bf0 con 0x7fd5b40caf40 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.528+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b40c5770 con 0x7fd5b40caf40 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.528+0000 7fd5b17fa640 1 -- 172.21.15.2:0/451617039 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd5a801a070 con 0x7fd5b40caf40 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.529+0000 7fd5b17fa640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd59806c2b0 0x7fd59806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.529+0000 7fd5b17fa640 1 -- 172.21.15.2:0/451617039 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd5a808b250 con 0x7fd5b40caf40 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.529+0000 7fd5b3fff640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd59806c2b0 0x7fd59806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.529+0000 7fd5b3fff640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd59806c2b0 0x7fd59806e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0033000 tx=0x7fd5a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:15.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.533+0000 7fd5b17fa640 1 -- 172.21.15.2:0/451617039 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd5a8059400 con 0x7fd5b40caf40 2024-09-19T05:43:15.591 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:15.580+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:15.682 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:15.680+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:15.705 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:15.706 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:15.706 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:15.706 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:15.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:15.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:15.736+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fd5b40b32c0 con 0x7fd5b40caf40 2024-09-19T05:43:15.783 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:15.782+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:15.850 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:15.851 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:15.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:15.884 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:15.882+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.992447132000002, 2024-09-19T05:43:15.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002950785, 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975069+0000", 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975082+0000", 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3139000000000001e-05 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975096+0000", 2024-09-19T05:43:15.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4250000000000001e-05 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975106+0000", 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4949999999999997e-06 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975244+0000", 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013790100000000001 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122292 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975481+0000", 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011483499999999999 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977502+0000", 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020215849999999998 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977932+0000", 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000429877 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5309000000000003e-05 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978013+0000", 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5796000000000001e-05 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.99236878, 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033652420000000001, 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-07 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975145+0000", 2024-09-19T05:43:15.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975154+0000", 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0219999999999993e-06 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975161+0000", 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9870000000000002e-06 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975720+0000", 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055893000000000002 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975804+0000", 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4025999999999998e-05 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975919+0000", 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115009 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978071+0000", 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002151563 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978433+0000", 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036205599999999998 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7867999999999997e-05 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978506+0000", 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5397e-05 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:28 2.b 2:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.992337956999997, 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030203629999999999, 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2099999999999997e-07 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975175+0000", 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1109999999999999e-06 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975182+0000", 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975188+0000", 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4090000000000002e-06 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5392000000000001e-05 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:15.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112596 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975488+0000", 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013297499999999999 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977490+0000", 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020015580000000001 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:15.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978100+0000", 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060999199999999996 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978150+0000", 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0287000000000002e-05 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978192+0000", 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1635999999999999e-05 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:34 2.b 2:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.992077322, 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028916179999999999, 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:15.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975433+0000", 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9599999999999999e-07 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975438+0000", 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975448+0000", 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0145e-05 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975455+0000", 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9149999999999999e-06 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975979+0000", 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052412100000000001 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976090+0000", 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111307 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978324+0000", 2024-09-19T05:43:15.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022336130000000002 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.992016681999999, 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026148299999999998, 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975496+0000", 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975504+0000", 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975510+0000", 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975620+0000", 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109602 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975717+0000", 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7676999999999993e-05 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978108+0000", 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023904070000000002 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:4 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.991719932999999, 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022744829999999999, 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:15.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:15.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6600000000000005e-07 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975797+0000", 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3790000000000003e-06 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975805+0000", 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975812+0000", 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.029e-06 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975861+0000", 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8266999999999997e-05 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975882+0000", 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896000000000001e-05 2024-09-19T05:43:15.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978029+0000", 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021471490000000001 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978064+0000", 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:15.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:5 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.990448387000001, 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001037481, 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977066+0000", 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2250000000000002e-06 2024-09-19T05:43:16.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977087+0000", 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977095+0000", 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7889999999999995e-06 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977159+0000", 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-19T05:43:16.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977190+0000", 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1526999999999998e-05 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000881224 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978099+0000", 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6962999999999999e-05 2024-09-19T05:43:16.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.989866810000002, 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046917399999999999, 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2399999999999997e-07 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977647+0000", 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977671+0000", 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3926000000000001e-05 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977679+0000", 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.593e-06 2024-09-19T05:43:16.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977789+0000", 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109969 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978112+0000", 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032325799999999999 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.007 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.98978846, 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029909400000000002, 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977722+0000", 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977725+0000", 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3830000000000001e-06 2024-09-19T05:43:16.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977737+0000", 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2194999999999999e-05 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977920+0000", 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017706699999999999 2024-09-19T05:43:16.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977927+0000", 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978020+0000", 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3358999999999998e-05 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:28 2.b e22/19 ondisk, result = 0)", 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.989555049000003, 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024615999999999999, 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977960+0000", 2024-09-19T05:43:16.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3630000000000004e-06 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977970+0000", 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977978+0000", 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0209999999999999e-06 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978088+0000", 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109578 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9630000000000001e-06 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978201+0000", 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106018 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.b epoch 22/19 rep_tid 4 lcod 22'2)", 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.989518644999997, 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033492, 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:16.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977999+0000", 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8690000000000004e-06 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978007+0000", 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3929999999999999e-06 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978223+0000", 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000211051 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978326+0000", 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102874 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:25 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:16.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.989327649000003, 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033417400000000002, 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:16.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978186+0000", 2024-09-19T05:43:16.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6349999999999999e-06 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978195+0000", 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7870000000000007e-06 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978367+0000", 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017261800000000001 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978420+0000", 2024-09-19T05:43:16.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2339999999999997e-05 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978427+0000", 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.25e-06 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978516+0000", 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9038999999999999e-05 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:26 2.1f e22/19 2:fe07bba1:::100.00000000.inode:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.988765915000002, 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047866199999999998, 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:16.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978750+0000", 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4809999999999997e-06 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978759+0000", 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1300000000000007e-06 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978763+0000", 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3290000000000004e-06 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978801+0000", 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8127999999999998e-05 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978815+0000", 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3523000000000001e-05 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979197+0000", 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038223899999999998 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979222+0000", 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5219999999999999e-05 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1f epoch 22/19 rep_tid 9 entries 22'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.988323491000003, 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033723899999999997, 2024-09-19T05:43:16.028 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979187+0000", 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4270000000000002e-06 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979206+0000", 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313000000000001e-05 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979214+0000", 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420000000000001e-06 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979253+0000", 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.892e-05 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979523+0000", 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000270547 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:16.032 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:16.032 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:16.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:16.034 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:16.031+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:16.134 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:16.132+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:16.154 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:16.154 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:16.154 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:16.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:16.236 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:16.234+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:16.270 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:16.270 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:16.271 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:16.271 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:16.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:16.336 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:16.334+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:16.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.390+0000 7fd5b17fa640 1 -- 172.21.15.2:0/451617039 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v37) ==== 97+0+0 (secure 0 0 0) 0x7fd5a805d700 con 0x7fd5b40caf40 2024-09-19T05:43:16.393 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:43:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd59806c2b0 msgr2=0x7fd59806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd59806c2b0 0x7fd59806e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0033000 tx=0x7fd5a002e000 comp rx=0 tx=0).stop 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5b40caf40 msgr2=0x7fd5b415a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5b40caf40 0x7fd5b415a8d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd5a800ebe0 tx=0x7fd5a800c520 comp rx=0 tx=0).stop 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 shutdown_connections 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd59806c2b0 0x7fd59806e770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5b4155000 0x7fd5b40cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5b4154c20 0x7fd5b40cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 --2- 172.21.15.2:0/451617039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5b40caf40 0x7fd5b415a8d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 >> 172.21.15.2:0/451617039 conn(0x7fd5b406d0e0 msgr2=0x7fd5b4067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 shutdown_connections 2024-09-19T05:43:16.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.394+0000 7fd5bac65640 1 -- 172.21.15.2:0/451617039 wait complete. 2024-09-19T05:43:16.427 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.449769430000003, 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:43:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:43:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:43:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.448893773000002, 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:43:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:43:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.448269592999999, 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:43:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.448148854999999, 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:43:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.448109565999999, 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:43:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:43:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:43:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.448045065999999, 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:43:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447981648000003, 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:43:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:43:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447931992999997, 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:43:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447875824, 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447848833000002, 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:43:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:43:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447770386000002, 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:43:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:43:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447736704999997, 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:43:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447713694000001, 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:43:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:43:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447680298999998, 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:43:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:43:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447637620000002, 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:43:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:43:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447598454999998, 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:43:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447566428000002, 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:43:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:43:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447367894999999, 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:43:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:43:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:43:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.447296717, 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 38.441080559, 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:43:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:43:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:43:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:43:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:16.531 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:16.531 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:16.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:16.534 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:16.531+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:16.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.474+0000 7f8ffe45e640 1 Processor -- start 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.474+0000 7f8ffe45e640 1 -- start start 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.474+0000 7f8ffe45e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ff815aa90 0x7f8ff815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.474+0000 7f8ffe45e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8ff80cd8c0 0x7f8ff815e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.474+0000 7f8ffe45e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff815ebe0 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.474+0000 7f8ffe45e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff815e9b0 con 0x7f8ff80cd8c0 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.474+0000 7f8ffe45e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff815eec0 con 0x7f8ff815aa90 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ffc9fb640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8ff80cd8c0 0x7f8ff815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ff7fff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8ff80cd4e0 0x7f8ff80b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:57890/0 (socket says 172.21.15.2:57890) 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ff7fff640 1 -- 172.21.15.2:0/358359953 learned_addr learned my addr 172.21.15.2:0/358359953 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/358359953 >> v2:172.21.15.123:3300/0 conn(0x7f8ff815aa90 0x7f8ff815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ffc9fb640 1 -- 172.21.15.2:0/358359953 >> v2:172.21.15.123:3300/0 conn(0x7f8ff815aa90 msgr2=0x7f8ff815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/358359953 >> v2:172.21.15.123:3300/0 conn(0x7f8ff815aa90 0x7f8ff815df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ffc9fb640 1 -- 172.21.15.2:0/358359953 >> v1:172.21.15.2:6789/0 conn(0x7f8ff80cd4e0 legacy=0x7f8ff80b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ffc9fb640 1 -- 172.21.15.2:0/358359953 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff80b2e00 con 0x7f8ff80cd8c0 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.475+0000 7f8ff6ffd640 1 -- 172.21.15.2:0/358359953 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2910809289 0 0) 0x7f8ff815ebe0 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.476+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/358359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd8c0 0x7f8ff815e470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ff815a940 tx=0x7f8ff002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f033f5c406f0b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.476+0000 7f8ff6ffd640 1 -- 172.21.15.2:0/358359953 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0007740 con 0x7f8ff80cd8c0 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.476+0000 7f8ff6ffd640 1 -- 172.21.15.2:0/358359953 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff00078e0 con 0x7f8ff80cd8c0 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.476+0000 7f8ff6ffd640 1 -- 172.21.15.2:0/358359953 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0007cb0 con 0x7f8ff80cd8c0 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 -- 172.21.15.2:0/358359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd8c0 msgr2=0x7f8ff815e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/358359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd8c0 0x7f8ff815e470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ff815a940 tx=0x7f8ff002a680 comp rx=0 tx=0).stop 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 -- 172.21.15.2:0/358359953 shutdown_connections 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/358359953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd8c0 0x7f8ff815e470 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/358359953 >> v2:172.21.15.123:3300/0 conn(0x7f8ff815aa90 0x7f8ff815df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 -- 172.21.15.2:0/358359953 >> 172.21.15.2:0/358359953 conn(0x7f8ff8067b30 msgr2=0x7f8ff8067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 -- 172.21.15.2:0/358359953 shutdown_connections 2024-09-19T05:43:16.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.477+0000 7f8ffe45e640 1 -- 172.21.15.2:0/358359953 wait complete. 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.478+0000 7f8ffe45e640 1 Processor -- start 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.478+0000 7f8ffe45e640 1 -- start start 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ffe45e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd4e0 0x7f8ff81e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ffe45e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ff80cd8c0 0x7f8ff81597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ffe45e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff815aa90 0x7f8ff81f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ffe45e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80b36a0 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ffe45e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff80b33c0 con 0x7f8ff815aa90 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ffe45e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff80b42d0 con 0x7f8ff80cd8c0 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd4e0 0x7f8ff81e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd4e0 0x7f8ff81e68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48894/0 (socket says 172.21.15.2:48894) 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 -- 172.21.15.2:0/461129240 learned_addr learned my addr 172.21.15.2:0/461129240 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff815aa90 0x7f8ff81f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ff80cd8c0 0x7f8ff81597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ff80cd8c0 msgr2=0x7f8ff81597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8ff80cd8c0 0x7f8ff81597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff815aa90 msgr2=0x7f8ff81f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff815aa90 0x7f8ff81f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.479+0000 7f8ff7fff640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff81e6df0 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.480+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd4e0 0x7f8ff81e68b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c500 tx=0x7f8fe800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.480+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8018070 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.480+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe8007580 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.480+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff80c8040 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.480+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8004a20 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.480+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff80cf340 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.481+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8ff80b3bc0 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.481+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8fe801b070 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.482+0000 7f8ff4ff9640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8fe406c2e0 0x7f8fe406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.482+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe808a690 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.482+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8fe406c2e0 0x7f8fe406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.482+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7f8ff80c8040 con 0x7f8ff80cd4e0 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.482+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8ff815db20 0x7f8ff81f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.482+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ff81f01a0 con 0x7f8ff815db20 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.483+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8ff815db20 0x7f8ff81f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.483+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8fe406c2e0 0x7f8fe406e7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80cebb0 tx=0x7f8fec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.483+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8ff815db20 0x7f8ff81f3bf0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.484+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff81f01a0 con 0x7f8ff815db20 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.498+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff80cf620 con 0x7f8ff815db20 2024-09-19T05:43:16.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.499+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff80cf620 con 0x7f8ff815db20 2024-09-19T05:43:16.543 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:16.543 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.500+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8ff81f0df0 0x7f8ff8156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.500+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8ff81f0960 con 0x7f8ff81f0df0 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.500+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8ff81f0df0 0x7f8ff8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.500+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8ff81f0df0 0x7f8ff8156910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.501+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff81f0960 con 0x7f8ff81f0df0 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.517+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff81f0530 con 0x7f8ff81f0df0 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.518+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff81f0530 con 0x7f8ff81f0df0 2024-09-19T05:43:16.544 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.519+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8ff8157360 0x7f8ff8062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.519+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8ff815a110 con 0x7f8ff8157360 2024-09-19T05:43:16.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.519+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8ff8157360 0x7f8ff8062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.519+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8ff8157360 0x7f8ff8062630 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.520+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff815a110 con 0x7f8ff8157360 2024-09-19T05:43:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.534+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff8062dc0 con 0x7f8ff8157360 2024-09-19T05:43:16.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.536+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff8062dc0 con 0x7f8ff8157360 2024-09-19T05:43:16.546 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.536+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8ff8063390 0x7f8ff81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.536+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8ff8158e30 con 0x7f8ff8063390 2024-09-19T05:43:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.536+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8ff8063390 0x7f8ff81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.537+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8ff8063390 0x7f8ff81588f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.537+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff8158e30 con 0x7f8ff8063390 2024-09-19T05:43:16.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.551+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff81592c0 con 0x7f8ff8063390 2024-09-19T05:43:16.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.553+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff81592c0 con 0x7f8ff8063390 2024-09-19T05:43:16.555 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:16.556 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:16.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.553+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8ff802cb50 0x7f8ff802cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.553+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8ff81f06e0 con 0x7f8ff802cb50 2024-09-19T05:43:16.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.553+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8ff802cb50 0x7f8ff802cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.554+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8ff802cb50 0x7f8ff802cf70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.554+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff81f06e0 con 0x7f8ff802cb50 2024-09-19T05:43:16.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.568+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff81f06e0 con 0x7f8ff802cb50 2024-09-19T05:43:16.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.570+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff81f06e0 con 0x7f8ff802cb50 2024-09-19T05:43:16.572 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:16.573 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:16.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.570+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8ff8204c40 0x7f8ff8207030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.570+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8ff81f06e0 con 0x7f8ff8204c40 2024-09-19T05:43:16.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.571+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8ff8204c40 0x7f8ff8207030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.571+0000 7f8ff77fe640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8ff8204c40 0x7f8ff8207030 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.572+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff81f06e0 con 0x7f8ff8204c40 2024-09-19T05:43:16.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.585+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff81f06e0 con 0x7f8ff8204c40 2024-09-19T05:43:16.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.587+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff81f06e0 con 0x7f8ff8204c40 2024-09-19T05:43:16.589 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.587+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8ff8207a80 0x7f8ff8209e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.587+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8ff802da70 con 0x7f8ff8207a80 2024-09-19T05:43:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.588+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8ff8207a80 0x7f8ff8209e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.588+0000 7f8ffc9fb640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8ff8207a80 0x7f8ff8209e70 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.589+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff802da70 con 0x7f8ff8207a80 2024-09-19T05:43:16.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.603+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff802da40 con 0x7f8ff8207a80 2024-09-19T05:43:16.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.605+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff802da40 con 0x7f8ff8207a80 2024-09-19T05:43:16.607 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:16.607 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:16.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.605+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8ff820a8c0 0x7f8ff820ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:16.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.605+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8ff820fe50 con 0x7f8ff820a8c0 2024-09-19T05:43:16.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.605+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8ff820a8c0 0x7f8ff820ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:16.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.605+0000 7f8ff7fff640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8ff820a8c0 0x7f8ff820ad00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:16.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.606+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff820fe50 con 0x7f8ff820a8c0 2024-09-19T05:43:16.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.620+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ff820b350 con 0x7f8ff820a8c0 2024-09-19T05:43:16.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ff4ff9640 1 -- 172.21.15.2:0/461129240 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ff820b350 con 0x7f8ff820a8c0 2024-09-19T05:43:16.624 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8ff815db20 msgr2=0x7f8ff81f3bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8ff815db20 0x7f8ff81f3bf0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8ff81f0df0 msgr2=0x7f8ff8156910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8ff81f0df0 0x7f8ff8156910 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8ff8157360 msgr2=0x7f8ff8062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8ff8157360 0x7f8ff8062630 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8ff8063390 msgr2=0x7f8ff81588f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffc9fb640 1 -- 172.21.15.2:0/461129240 reap_dead start 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8ff8063390 0x7f8ff81588f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8ff802cb50 msgr2=0x7f8ff802cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8ff802cb50 0x7f8ff802cf70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8ff8204c40 msgr2=0x7f8ff8207030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8ff8204c40 0x7f8ff8207030 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8ff8207a80 msgr2=0x7f8ff8209e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8ff8207a80 0x7f8ff8209e70 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8ff820a8c0 msgr2=0x7f8ff820ad00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8ff820a8c0 0x7f8ff820ad00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffc9fb640 1 -- 172.21.15.2:0/461129240 reap_dead start 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8fe406c2e0 msgr2=0x7f8fe406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8fe406c2e0 0x7f8fe406e7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ff80cebb0 tx=0x7f8fec008040 comp rx=0 tx=0).stop 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd4e0 msgr2=0x7f8ff81e68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd4e0 0x7f8ff81e68b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800c500 tx=0x7f8fe800c9c0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 shutdown_connections 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8fe406c2e0 0x7f8fe406e7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 --2- 172.21.15.2:0/461129240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ff80cd4e0 0x7f8ff81e68b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 >> 172.21.15.2:0/461129240 conn(0x7f8ff8067b30 msgr2=0x7f8ff80ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.622+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 shutdown_connections 2024-09-19T05:43:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:16.623+0000 7f8ffe45e640 1 -- 172.21.15.2:0/461129240 wait complete. 2024-09-19T05:43:16.633 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:16.631+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:16.677 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:16.678 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:16.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:16.734 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:16.733+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:16.789 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:16.789 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:16.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:16.839 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:16.837+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:16.936 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:16.934+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:16.940 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:16.940 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:16.948 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:17.037 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:17.035+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:17.051 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.051 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:17.138 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:17.137+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:17.163 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.163 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.171 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:17.239 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:17.237+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:17.315 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.315 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.324 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:17.340 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:17.338+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:17.428 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.429 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:17.481 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:17.479+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:17.541 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.541 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.542 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:17.540+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:17.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:17.643 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:17.642+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:17.693 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.693 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:17.746 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:17.744+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:17.814 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.814 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:17.845 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:17.844+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:17.946 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:17.944+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:17.972 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:17.973 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:17.981 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:18.047 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:18.045+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:18.134 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:18.134 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:18.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:18.147 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:18.145+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:18.248 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:18.246+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:18.260 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:18.261 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:18.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:18.349 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:18.347+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:18.380 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:18.381 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:18.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:18.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:43:18.479 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:18.477+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:18.513 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:18.514 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:18.514 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-19T05:43:18.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-19T05:43:18.515 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:18.515 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:18.524 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:18.565 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:18.563+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.594+0000 7f551b9f6640 1 Processor -- start 2024-09-19T05:43:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.595+0000 7f551b9f6640 1 -- start start 2024-09-19T05:43:18.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.596+0000 7f5518f91640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f55140c74f0 0x7f55140c8850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:57902/0 (socket says 172.21.15.2:57902) 2024-09-19T05:43:18.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.596+0000 7f5518f91640 1 -- 172.21.15.2:0/1674220681 learned_addr learned my addr 172.21.15.2:0/1674220681 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:18.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.596+0000 7f551b9f6640 1 --2- 172.21.15.2:0/1674220681 >> v2:172.21.15.123:3300/0 conn(0x7f5514071030 0x7f55141549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.596+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55140c8090 con 0x7f55140c74f0 2024-09-19T05:43:18.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.596+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55140c8f60 con 0x7f5514070c50 2024-09-19T05:43:18.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.596+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f551415b480 con 0x7f5514071030 2024-09-19T05:43:18.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.596+0000 7f5519f93640 1 --2- 172.21.15.2:0/1674220681 >> v2:172.21.15.123:3300/0 conn(0x7f5514071030 0x7f55141549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.597+0000 7f550bfff640 1 -- 172.21.15.2:0/1674220681 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234561488 0 0) 0x7f55140c8f60 con 0x7f5514070c50 2024-09-19T05:43:18.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.597+0000 7f5519f93640 1 -- 172.21.15.2:0/1674220681 >> v1:172.21.15.123:6789/0 conn(0x7f5514070c50 legacy=0x7f551415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.597+0000 7f5519f93640 1 -- 172.21.15.2:0/1674220681 >> v1:172.21.15.2:6789/0 conn(0x7f55140c74f0 legacy=0x7f55140c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.597+0000 7f5519f93640 1 -- 172.21.15.2:0/1674220681 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55140714e0 con 0x7f5514071030 2024-09-19T05:43:18.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.598+0000 7f5519f93640 1 --2- 172.21.15.2:0/1674220681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f55141549a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f550c009d90 tx=0x7f550c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8161207c19ebc74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.598+0000 7f550bfff640 1 -- 172.21.15.2:0/1674220681 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c007740 con 0x7f5514071030 2024-09-19T05:43:18.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.598+0000 7f550bfff640 1 -- 172.21.15.2:0/1674220681 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550c0078e0 con 0x7f5514071030 2024-09-19T05:43:18.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.598+0000 7f550bfff640 1 -- 172.21.15.2:0/1674220681 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c007cb0 con 0x7f5514071030 2024-09-19T05:43:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.599+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 msgr2=0x7f55141549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.599+0000 7f551b9f6640 1 --2- 172.21.15.2:0/1674220681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f55141549a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f550c009d90 tx=0x7f550c02a680 comp rx=0 tx=0).stop 2024-09-19T05:43:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.599+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 shutdown_connections 2024-09-19T05:43:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.599+0000 7f551b9f6640 1 --2- 172.21.15.2:0/1674220681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f55141549a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.599+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 >> 172.21.15.2:0/1674220681 conn(0x7f551406cd20 msgr2=0x7f551406d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.600+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 shutdown_connections 2024-09-19T05:43:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.600+0000 7f551b9f6640 1 -- 172.21.15.2:0/1674220681 wait complete. 2024-09-19T05:43:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.601+0000 7f551b9f6640 1 Processor -- start 2024-09-19T05:43:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.602+0000 7f551b9f6640 1 -- start start 2024-09-19T05:43:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.603+0000 7f551b9f6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5514070c50 0x7f5514149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.603+0000 7f551b9f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f551414a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.603+0000 7f551b9f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55140c74f0 0x7f55141508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.603+0000 7f5519792640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5514070c50 0x7f5514149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.603+0000 7f5518f91640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f551414a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.604+0000 7f5518f91640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f551414a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:35804/0 (socket says 172.21.15.2:35804) 2024-09-19T05:43:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.604+0000 7f5519792640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5514070c50 0x7f5514149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:33568/0 (socket says 172.21.15.2:33568) 2024-09-19T05:43:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.604+0000 7f5518f91640 1 -- 172.21.15.2:0/4236124234 learned_addr learned my addr 172.21.15.2:0/4236124234 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.604+0000 7f5519f93640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55140c74f0 0x7f55141508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.604+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f55140c94c0 con 0x7f55140c74f0 2024-09-19T05:43:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.604+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f551415ab70 con 0x7f5514071030 2024-09-19T05:43:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.604+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f551415ae30 con 0x7f5514070c50 2024-09-19T05:43:18.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.605+0000 7f5518f91640 1 -- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5514070c50 msgr2=0x7f5514149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.605+0000 7f5518f91640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5514070c50 0x7f5514149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.605+0000 7f5518f91640 1 -- 172.21.15.2:0/4236124234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55140c74f0 msgr2=0x7f55141508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.605+0000 7f5518f91640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55140c74f0 0x7f55141508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.605+0000 7f5518f91640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55141f0b30 con 0x7f5514071030 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.605+0000 7f5519f93640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55140c74f0 0x7f55141508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.605+0000 7f5518f91640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f551414a1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f550400c780 tx=0x7f550400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.606+0000 7f5509ffb640 1 -- 172.21.15.2:0/4236124234 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504018070 con 0x7f5514071030 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.606+0000 7f5509ffb640 1 -- 172.21.15.2:0/4236124234 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55040072d0 con 0x7f5514071030 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.606+0000 7f5509ffb640 1 -- 172.21.15.2:0/4236124234 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504007680 con 0x7f5514071030 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.606+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55140ccfb0 con 0x7f5514071030 2024-09-19T05:43:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.606+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55140cb3e0 con 0x7f5514071030 2024-09-19T05:43:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.608+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55140c70c0 con 0x7f5514071030 2024-09-19T05:43:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.608+0000 7f5509ffb640 1 -- 172.21.15.2:0/4236124234 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f550401b070 con 0x7f5514071030 2024-09-19T05:43:18.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.609+0000 7f5509ffb640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f54f806c450 0x7f54f806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.609+0000 7f5509ffb640 1 -- 172.21.15.2:0/4236124234 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5709+0+0 (secure 0 0 0) 0x7f550408a940 con 0x7f5514071030 2024-09-19T05:43:18.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.609+0000 7f5519792640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f54f806c450 0x7f54f806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.610+0000 7f5519792640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f54f806c450 0x7f54f806e910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5510001120 tx=0x7f5510009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.614+0000 7f5509ffb640 1 -- 172.21.15.2:0/4236124234 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f55040589c0 con 0x7f5514071030 2024-09-19T05:43:18.636 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:18.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.646+0000 7f977d21e640 1 Processor -- start 2024-09-19T05:43:18.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.646+0000 7f977d21e640 1 -- start start 2024-09-19T05:43:18.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f977d21e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f97780c6760 0x7f9778073e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f977d21e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97780c6b40 0x7f97780c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f9776d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f97780c6760 0x7f9778073e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f9776d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f97780c6760 0x7f9778073e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48912/0 (socket says 172.21.15.2:48912) 2024-09-19T05:43:18.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f9776d9d640 1 -- 172.21.15.2:0/684561720 learned_addr learned my addr 172.21.15.2:0/684561720 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9778072410 con 0x7f97780b2bb0 2024-09-19T05:43:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f97780c9760 con 0x7f97780c6760 2024-09-19T05:43:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.648+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f97780b2720 con 0x7f97780c6b40 2024-09-19T05:43:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.649+0000 7f977659c640 1 --2- 172.21.15.2:0/684561720 >> v2:172.21.15.123:3300/0 conn(0x7f97780c6b40 0x7f97780c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.649+0000 7f9775d9b640 1 -- 172.21.15.2:0/684561720 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 43142066 0 0) 0x7f9778072410 con 0x7f97780b2bb0 2024-09-19T05:43:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.649+0000 7f9775d9b640 1 -- 172.21.15.2:0/684561720 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9760003520 con 0x7f97780b2bb0 2024-09-19T05:43:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.649+0000 7f9776d9d640 1 -- 172.21.15.2:0/684561720 >> v2:172.21.15.123:3300/0 conn(0x7f97780c6b40 msgr2=0x7f97780c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.649+0000 7f9776d9d640 1 --2- 172.21.15.2:0/684561720 >> v2:172.21.15.123:3300/0 conn(0x7f97780c6b40 0x7f97780c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.649+0000 7f9776d9d640 1 -- 172.21.15.2:0/684561720 >> v1:172.21.15.2:6789/0 conn(0x7f97780b2bb0 legacy=0x7f977815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.649+0000 7f9776d9d640 1 -- 172.21.15.2:0/684561720 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9778072680 con 0x7f97780c6760 2024-09-19T05:43:18.653 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:18.650+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:18.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.650+0000 7f9776d9d640 1 --2- 172.21.15.2:0/684561720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780c6760 0x7f9778073e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f97780cac90 tx=0x7f976802a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f259ee261570d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.650+0000 7f9775d9b640 1 -- 172.21.15.2:0/684561720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768037070 con 0x7f97780c6760 2024-09-19T05:43:18.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.650+0000 7f9775d9b640 1 -- 172.21.15.2:0/684561720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9768032640 con 0x7f97780c6760 2024-09-19T05:43:18.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.651+0000 7f9775d9b640 1 -- 172.21.15.2:0/684561720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768037070 con 0x7f97780c6760 2024-09-19T05:43:18.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.651+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780c6760 msgr2=0x7f9778073e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.651+0000 7f977d21e640 1 --2- 172.21.15.2:0/684561720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780c6760 0x7f9778073e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f97780cac90 tx=0x7f976802a310 comp rx=0 tx=0).stop 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.651+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 shutdown_connections 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.651+0000 7f977d21e640 1 --2- 172.21.15.2:0/684561720 >> v2:172.21.15.123:3300/0 conn(0x7f97780c6b40 0x7f97780c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.651+0000 7f977d21e640 1 --2- 172.21.15.2:0/684561720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780c6760 0x7f9778073e70 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f97780cac90 tx=0x7f976802a310 comp rx=0 tx=0).stop 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.651+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 >> 172.21.15.2:0/684561720 conn(0x7f97780cd470 msgr2=0x7f97780c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.652+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 shutdown_connections 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.652+0000 7f977d21e640 1 -- 172.21.15.2:0/684561720 wait complete. 2024-09-19T05:43:18.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.653+0000 7f977d21e640 1 Processor -- start 2024-09-19T05:43:18.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.654+0000 7f977d21e640 1 -- start start 2024-09-19T05:43:18.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.654+0000 7f977d21e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780b2bb0 0x7f97781ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.654+0000 7f977d21e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97780c6b40 0x7f97781edca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f977d21e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9778156b50 0x7f97780ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f977d21e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f97780ca600 con 0x7f97780b2bb0 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f977d21e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9778072960 con 0x7f97780c6b40 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f977d21e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9778158fc0 con 0x7f9778156b50 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780b2bb0 0x7f97781ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780b2bb0 0x7f97781ef7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48926/0 (socket says 172.21.15.2:48926) 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 -- 172.21.15.2:0/531246231 learned_addr learned my addr 172.21.15.2:0/531246231 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f977659c640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97780c6b40 0x7f97781edca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f977759e640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9778156b50 0x7f97780ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 -- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9778156b50 msgr2=0x7f97780ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9778156b50 0x7f97780ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 -- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97780c6b40 msgr2=0x7f97781edca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97780c6b40 0x7f97781edca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.655+0000 7f9776d9d640 1 -- 172.21.15.2:0/531246231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97781f2480 con 0x7f97780b2bb0 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.656+0000 7f9776d9d640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780b2bb0 0x7f97781ef7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f97680090b0 tx=0x7f97680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.656+0000 7f975f7fe640 1 -- 172.21.15.2:0/531246231 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768037040 con 0x7f97780b2bb0 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.656+0000 7f975f7fe640 1 -- 172.21.15.2:0/531246231 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9768003610 con 0x7f97780b2bb0 2024-09-19T05:43:18.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.656+0000 7f975f7fe640 1 -- 172.21.15.2:0/531246231 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976802ae60 con 0x7f97780b2bb0 2024-09-19T05:43:18.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.656+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97781ecee0 con 0x7f97780b2bb0 2024-09-19T05:43:18.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.657+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97781ed4a0 con 0x7f97780b2bb0 2024-09-19T05:43:18.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.658+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97780c76e0 con 0x7f97780b2bb0 2024-09-19T05:43:18.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.659+0000 7f975f7fe640 1 -- 172.21.15.2:0/531246231 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f97680039e0 con 0x7f97780b2bb0 2024-09-19T05:43:18.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.659+0000 7f975f7fe640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f976406c2e0 0x7f976406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.660+0000 7f975f7fe640 1 -- 172.21.15.2:0/531246231 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5709+0+0 (secure 0 0 0) 0x7f97680abb70 con 0x7f97780b2bb0 2024-09-19T05:43:18.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.660+0000 7f977659c640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f976406c2e0 0x7f976406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.661+0000 7f977659c640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f976406c2e0 0x7f976406e7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97781539d0 tx=0x7f976c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.668+0000 7f975f7fe640 1 -- 172.21.15.2:0/531246231 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f976807a2e0 con 0x7f97780b2bb0 2024-09-19T05:43:18.723 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:18.724 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:18.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:18.778 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:18.776+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.797+0000 7f479589b640 1 Processor -- start 2024-09-19T05:43:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.797+0000 7f479589b640 1 -- start start 2024-09-19T05:43:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.798+0000 7f479589b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f479015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.798+0000 7f479589b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f47900c3930 0x7f479015f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.798+0000 7f479589b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4790070890 con 0x7f4790159260 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.798+0000 7f479589b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f47900c8aa0 con 0x7f47900c3930 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.798+0000 7f479589b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4790067c60 con 0x7f47900c33e0 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478effd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4790159260 0x7f47900cb8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:57930/0 (socket says 172.21.15.2:57930) 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478effd640 1 -- 172.21.15.2:0/3346614541 learned_addr learned my addr 172.21.15.2:0/3346614541 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3346614541 >> v2:172.21.15.123:3300/0 conn(0x7f47900c3930 0x7f479015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f47867fc640 1 --2- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f479015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478e7fc640 1 -- 172.21.15.2:0/3346614541 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3775471354 0 0) 0x7f4790070890 con 0x7f4790159260 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478e7fc640 1 -- 172.21.15.2:0/3346614541 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f477c003520 con 0x7f4790159260 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478f7fe640 1 -- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 msgr2=0x7f479015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f479015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478f7fe640 1 -- 172.21.15.2:0/3346614541 >> v1:172.21.15.2:6789/0 conn(0x7f4790159260 legacy=0x7f47900cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.799+0000 7f478f7fe640 1 -- 172.21.15.2:0/3346614541 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47900679f0 con 0x7f47900c3930 2024-09-19T05:43:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.801+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47900c3930 0x7f479015f490 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47880097a0 tx=0x7f478802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984d27d701b2f0ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.801+0000 7f478e7fc640 1 -- 172.21.15.2:0/3346614541 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788037070 con 0x7f47900c3930 2024-09-19T05:43:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.801+0000 7f478e7fc640 1 -- 172.21.15.2:0/3346614541 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4788032440 con 0x7f47900c3930 2024-09-19T05:43:18.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 -- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47900c3930 msgr2=0x7f479015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 --2- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47900c3930 0x7f479015f490 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47880097a0 tx=0x7f478802a250 comp rx=0 tx=0).stop 2024-09-19T05:43:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 -- 172.21.15.2:0/3346614541 shutdown_connections 2024-09-19T05:43:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 --2- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f479015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 --2- 172.21.15.2:0/3346614541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47900c3930 0x7f479015f490 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 -- 172.21.15.2:0/3346614541 >> 172.21.15.2:0/3346614541 conn(0x7f479006fcd0 msgr2=0x7f47900700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 -- 172.21.15.2:0/3346614541 shutdown_connections 2024-09-19T05:43:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.802+0000 7f479589b640 1 -- 172.21.15.2:0/3346614541 wait complete. 2024-09-19T05:43:18.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.804+0000 7f479589b640 1 Processor -- start 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.804+0000 7f479589b640 1 -- start start 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.805+0000 7f479589b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f47900c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.805+0000 7f479589b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47900c3930 0x7f4790157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f479589b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790159260 0x7f479015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f479589b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4790070b00 con 0x7f47900c3930 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f47867fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47900c3930 0x7f4790157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f47867fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47900c3930 0x7f4790157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:48934/0 (socket says 172.21.15.2:48934) 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f47867fc640 1 -- 172.21.15.2:0/3933170243 learned_addr learned my addr 172.21.15.2:0/3933170243 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f47900c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f479589b640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47900c8d60 con 0x7f4790159260 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790159260 0x7f479015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.806+0000 7f479589b640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f47900c9110 con 0x7f47900c33e0 2024-09-19T05:43:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.807+0000 7f478f7fe640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 msgr2=0x7f47900c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.807+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f47900c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.808+0000 7f478f7fe640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47900c3930 msgr2=0x7f4790157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.808+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47900c3930 0x7f4790157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.808+0000 7f478f7fe640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4790073a20 con 0x7f4790159260 2024-09-19T05:43:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.808+0000 7f47867fc640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47900c3930 0x7f4790157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.808+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f47900c33e0 0x7f47900c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.809+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790159260 0x7f479015ecd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4788031000 tx=0x7f47880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.809+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788037040 con 0x7f4790159260 2024-09-19T05:43:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.809+0000 7f479589b640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47900c6c80 con 0x7f4790159260 2024-09-19T05:43:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.809+0000 7f479589b640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4790072250 con 0x7f4790159260 2024-09-19T05:43:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.809+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4788042070 con 0x7f4790159260 2024-09-19T05:43:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.809+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788037040 con 0x7f4790159260 2024-09-19T05:43:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.811+0000 7f479589b640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4774005270 con 0x7f4790159260 2024-09-19T05:43:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.813+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4788047460 con 0x7f4790159260 2024-09-19T05:43:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.813+0000 7f478cff9640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f477006c6b0 0x7f477006eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.814+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5709+0+0 (secure 0 0 0) 0x7f4788003c10 con 0x7f4790159260 2024-09-19T05:43:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.814+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f477006c6b0 0x7f477006eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.814+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7f47880ac220 con 0x7f4790159260 2024-09-19T05:43:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.814+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4774001ca0 0x7f47740040d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.814+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4774005550 con 0x7f4774001ca0 2024-09-19T05:43:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.814+0000 7f47867fc640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4774001ca0 0x7f47740040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.815+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f477006c6b0 0x7f477006eb70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4780001560 tx=0x7f4780001a90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.815+0000 7f47867fc640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4774001ca0 0x7f47740040d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.816+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4774005550 con 0x7f4774001ca0 2024-09-19T05:43:18.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.849+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4774004610 con 0x7f4774001ca0 2024-09-19T05:43:18.854 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:18.852+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:18.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.852+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4774004610 con 0x7f4774001ca0 2024-09-19T05:43:18.855 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:18.855 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:18.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.852+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4774007c70 0x7f477400a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.853+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f477400cee0 con 0x7f4774007c70 2024-09-19T05:43:18.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.853+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4774007c70 0x7f477400a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.853+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4774007c70 0x7f477400a060 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.854+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f477400cee0 con 0x7f4774007c70 2024-09-19T05:43:18.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.887+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f477400a930 con 0x7f4774007c70 2024-09-19T05:43:18.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.890+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f477400a930 con 0x7f4774007c70 2024-09-19T05:43:18.893 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:18.894 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:18.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.891+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f477400aae0 0x7f477400af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:18.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.891+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f477400b4d0 con 0x7f477400aae0 2024-09-19T05:43:18.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.892+0000 7f47867fc640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f477400aae0 0x7f477400af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:18.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.892+0000 7f47867fc640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f477400aae0 0x7f477400af90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:18.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.893+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f477400b4d0 con 0x7f477400aae0 2024-09-19T05:43:18.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.926+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f477400ba10 con 0x7f477400aae0 2024-09-19T05:43:19.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.928+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f477400ba10 con 0x7f477400aae0 2024-09-19T05:43:19.402 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.929+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f477400f130 0x7f4774011520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.929+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4774014400 con 0x7f477400f130 2024-09-19T05:43:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.929+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f477400f130 0x7f4774011520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.929+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f477400f130 0x7f4774011520 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:19.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.931+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4774014400 con 0x7f477400f130 2024-09-19T05:43:19.404 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:19.404 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:19.404 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:19.404 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:19.404 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:19.405 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:19.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.960+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4774011d30 con 0x7f477400f130 2024-09-19T05:43:19.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.962+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4774011d30 con 0x7f477400f130 2024-09-19T05:43:19.406 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:19.407 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:19.400+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:19.407 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:19.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.962+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4774012450 0x7f4774012890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:19.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.963+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4774014020 con 0x7f4774012450 2024-09-19T05:43:19.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.963+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4774012450 0x7f4774012890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:19.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.964+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4774012450 0x7f4774012890 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:19.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.965+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4774014020 con 0x7f4774012450 2024-09-19T05:43:19.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.992+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4774013fa0 con 0x7f4774012450 2024-09-19T05:43:19.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.994+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4774013fa0 con 0x7f4774012450 2024-09-19T05:43:19.408 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:19.409 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:19.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.994+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f47740132e0 0x7f4774013740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:19.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.994+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f477401aef0 con 0x7f47740132e0 2024-09-19T05:43:19.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.995+0000 7f47867fc640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f47740132e0 0x7f4774013740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:19.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.995+0000 7f47867fc640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f47740132e0 0x7f4774013740 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:19.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.996+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f477401aef0 con 0x7f47740132e0 2024-09-19T05:43:19.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:18.998+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f551414a730 con 0x7f5514071030 2024-09-19T05:43:19.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.023+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4774018660 con 0x7f47740132e0 2024-09-19T05:43:19.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.025+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4774018660 con 0x7f47740132e0 2024-09-19T05:43:19.411 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:19.411 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:19.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.026+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4774018ce0 0x7f477401b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:19.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.026+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f477401e040 con 0x7f4774018ce0 2024-09-19T05:43:19.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.026+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4774018ce0 0x7f477401b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:19.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.026+0000 7f478f7fe640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4774018ce0 0x7f477401b0d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:19.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.027+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f477401e040 con 0x7f4774018ce0 2024-09-19T05:43:19.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.037+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd in", "ids": ["1"]} v 0) -- 0x7f9778157a40 con 0x7f97780b2bb0 2024-09-19T05:43:19.412 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-19T05:43:19.037+0000 7fe7b68ac640 -1 mon.a@0(leader).osd e37 definitely_dead 0 2024-09-19T05:43:19.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.054+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f477401b610 con 0x7f4774018ce0 2024-09-19T05:43:19.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.056+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f477401b610 con 0x7f4774018ce0 2024-09-19T05:43:19.413 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:19.414 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:19.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.057+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f477401bd30 0x7f477401c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:19.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.057+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4774021490 con 0x7f477401bd30 2024-09-19T05:43:19.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.057+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f477401bd30 0x7f477401c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:19.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.058+0000 7f478effd640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f477401bd30 0x7f477401c150 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:19.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.058+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4774021490 con 0x7f477401bd30 2024-09-19T05:43:19.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.086+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f477401c690 con 0x7f477401bd30 2024-09-19T05:43:19.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f478cff9640 1 -- 172.21.15.2:0/3933170243 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f477401c690 con 0x7f477401bd30 2024-09-19T05:43:19.415 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4774001ca0 msgr2=0x7f47740040d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4774001ca0 0x7f47740040d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4774007c70 msgr2=0x7f477400a060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4774007c70 0x7f477400a060 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f477400aae0 msgr2=0x7f477400af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f477400aae0 0x7f477400af90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f477400f130 msgr2=0x7f4774011520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f477400f130 0x7f4774011520 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f478f7fe640 1 -- 172.21.15.2:0/3933170243 reap_dead start 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4774012450 msgr2=0x7f4774012890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4774012450 0x7f4774012890 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f47740132e0 msgr2=0x7f4774013740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f478f7fe640 1 -- 172.21.15.2:0/3933170243 reap_dead start 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f47740132e0 0x7f4774013740 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4774018ce0 msgr2=0x7f477401b0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4774018ce0 0x7f477401b0d0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f477401bd30 msgr2=0x7f477401c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f477401bd30 0x7f477401c150 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f477006c6b0 msgr2=0x7f477006eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f477006c6b0 0x7f477006eb70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4780001560 tx=0x7f4780001a90 comp rx=0 tx=0).stop 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790159260 msgr2=0x7f479015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f4785ffb640 1 --2- 172.21.15.2:0/3933170243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4790159260 0x7f479015ecd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4788031000 tx=0x7f47880046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.088+0000 7f478f7fe640 1 -- 172.21.15.2:0/3933170243 reap_dead start 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.089+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 shutdown_connections 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.089+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 >> 172.21.15.2:0/3933170243 conn(0x7f479006fcd0 msgr2=0x7f47900c7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.089+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 shutdown_connections 2024-09-19T05:43:19.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.089+0000 7f4785ffb640 1 -- 172.21.15.2:0/3933170243 wait complete. 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.401+0000 7f975f7fe640 1 -- 172.21.15.2:0/531246231 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["1"]}]=0 marked in osd.1. v38) ==== 85+0+0 (secure 0 0 0) 0x7f9768079be0 con 0x7f97780b2bb0 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:marked in osd.1. 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f976406c2e0 msgr2=0x7f976406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f976406c2e0 0x7f976406e7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97781539d0 tx=0x7f976c009190 comp rx=0 tx=0).stop 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780b2bb0 msgr2=0x7f97781ef7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780b2bb0 0x7f97781ef7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f97680090b0 tx=0x7f97680046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 shutdown_connections 2024-09-19T05:43:19.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f976406c2e0 0x7f976406e7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97780c6b40 0x7f97781edca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9778156b50 0x7f97780ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 --2- 172.21.15.2:0/531246231 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97780b2bb0 0x7f97781ef7c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 >> 172.21.15.2:0/531246231 conn(0x7f97780cd470 msgr2=0x7f9778159350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:19.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 shutdown_connections 2024-09-19T05:43:19.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.405+0000 7f977d21e640 1 -- 172.21.15.2:0/531246231 wait complete. 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.409+0000 7f5509ffb640 1 -- 172.21.15.2:0/4236124234 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v38) ==== 91+0+0 (secure 0 0 0) 0x7f550405ccc0 con 0x7f5514071030 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.414+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f54f806c450 msgr2=0x7f54f806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.414+0000 7f551b9f6640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f54f806c450 0x7f54f806e910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5510001120 tx=0x7f5510009040 comp rx=0 tx=0).stop 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.414+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 msgr2=0x7f551414a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.414+0000 7f551b9f6640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f551414a1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f550400c780 tx=0x7f550400cc40 comp rx=0 tx=0).stop 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 shutdown_connections 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f54f806c450 0x7f54f806e910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5514071030 0x7f551414a1f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55140c74f0 0x7f55141508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 --2- 172.21.15.2:0/4236124234 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5514070c50 0x7f5514149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:19.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 >> 172.21.15.2:0/4236124234 conn(0x7f551406cd20 msgr2=0x7f5514159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:19.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 shutdown_connections 2024-09-19T05:43:19.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:19.415+0000 7f551b9f6640 1 -- 172.21.15.2:0/4236124234 wait complete. 2024-09-19T05:43:19.426 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-19T05:43:19.503 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:19.501+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:19.532 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:19.532 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:19.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:19.532 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:19.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.532 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.554763055000002, 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:43:19.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:43:19.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.554705491999997, 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:43:19.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:43:19.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:19.545 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.554633015999997, 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:43:19.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:43:19.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.551 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.554595757999998, 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:19.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:43:19.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.554555956000002, 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:19.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:43:19.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:43:19.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:19.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.553726795000003, 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:43:19.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.553301845999997, 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:19.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:43:19.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.573 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.549139844999999, 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013154900000000001, 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980456+0000", 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3099999999999997e-07 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980460+0000", 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6990000000000001e-06 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980470+0000", 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9420000000000007e-06 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980476+0000", 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0419999999999999e-06 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980510+0000", 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4498000000000001e-05 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980515+0000", 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6539999999999998e-06 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980587+0000", 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2082999999999994e-05 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.578 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.549108857999997, 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:43:19.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:43:19.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:1 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:19.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.548835486999998, 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103356, 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980760+0000", 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980763+0000", 2024-09-19T05:43:19.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6249999999999999e-06 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980770+0000", 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2880000000000004e-06 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980774+0000", 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6780000000000002e-06 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980803+0000", 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9512000000000001e-05 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980807+0000", 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3610000000000001e-06 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980863+0000", 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6535e-05 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.548528724999997, 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010034, 2024-09-19T05:43:19.588 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981067+0000", 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981070+0000", 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3730000000000001e-06 2024-09-19T05:43:19.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981077+0000", 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6370000000000002e-06 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981080+0000", 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3419999999999999e-06 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981107+0000", 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6806000000000001e-05 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981111+0000", 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981167+0000", 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6246999999999999e-05 2024-09-19T05:43:19.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.546526946, 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:43:19.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.546115186000002, 2024-09-19T05:43:19.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:43:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:43:19.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:43:19.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:38 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 41.541462361000001, 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186071, 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988133+0000", 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988134+0000", 2024-09-19T05:43:19.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0999999999999998e-07 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988137+0000", 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3919999999999999e-06 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988147+0000", 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659999999999996e-06 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988151+0000", 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.797e-06 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988194+0000", 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2911000000000003e-05 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988202+0000", 2024-09-19T05:43:19.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4399999999999999e-06 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988319+0000", 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117355 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.608 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.975731775, 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:43:19.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:43:19.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:40 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.973993458000002, 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178677, 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555602+0000", 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555603+0000", 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0699999999999996e-07 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555608+0000", 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2240000000000003e-06 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555620+0000", 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.151e-05 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555626+0000", 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8699999999999997e-06 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555665+0000", 2024-09-19T05:43:19.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9356000000000001e-05 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555673+0000", 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4230000000000005e-06 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555781+0000", 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010738699999999999 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.619 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.659064821000001, 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:19.620 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:43:19.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:43:19.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.656979411000002, 2024-09-19T05:43:19.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:19.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:43:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 18'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@18'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.0522572599999993, 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084064700000000003, 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477339+0000", 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000002e-07 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477650+0000", 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031168299999999999 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477668+0000", 2024-09-19T05:43:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7745e-05 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477689+0000", 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0644000000000001e-05 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477763+0000", 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3919999999999997e-05 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477772+0000", 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2609999999999999e-06 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.478179+0000", 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00040692799999999998 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 18'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@18'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.0499252840000004, 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010886400000000001, 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479671+0000", 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7300000000000005e-07 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480001+0000", 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033000999999999999 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480025+0000", 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4053000000000002e-05 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480044+0000", 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.925e-05 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480142+0000", 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7955000000000007e-05 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480156+0000", 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3553999999999999e-05 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480759+0000", 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060294500000000002 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:19.640 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:19.642 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:19.642 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:19.639+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:19.742 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:19.740+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:19.791 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:19.792 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:19.792 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:19.792 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:19.801 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:19.843 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:19.841+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:19.944 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:19.942+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:19.950 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:19.950 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:19.951 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:19.951 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:19.951 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:19.960 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:20.045 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:20.043+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:20.109 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.133037350999999, 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002950785, 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.110 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975069+0000", 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975082+0000", 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3139000000000001e-05 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975096+0000", 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4250000000000001e-05 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975106+0000", 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4949999999999997e-06 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975244+0000", 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013790100000000001 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122292 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975481+0000", 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011483499999999999 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977502+0000", 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020215849999999998 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977932+0000", 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000429877 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5309000000000003e-05 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978013+0000", 2024-09-19T05:43:20.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5796000000000001e-05 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.132958999000003, 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033652420000000001, 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.116 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:20.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-07 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975145+0000", 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975154+0000", 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0219999999999993e-06 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975161+0000", 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9870000000000002e-06 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975720+0000", 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055893000000000002 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975804+0000", 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4025999999999998e-05 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975919+0000", 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115009 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978071+0000", 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002151563 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978433+0000", 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036205599999999998 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7867999999999997e-05 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978506+0000", 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5397e-05 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.121 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:28 2.b 2:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.132928176, 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030203629999999999, 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:20.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2099999999999997e-07 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975175+0000", 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1109999999999999e-06 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975182+0000", 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-19T05:43:20.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975188+0000", 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4090000000000002e-06 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5392000000000001e-05 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112596 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975488+0000", 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013297499999999999 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977490+0000", 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020015580000000001 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978100+0000", 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060999199999999996 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:20.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978150+0000", 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0287000000000002e-05 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978192+0000", 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1635999999999999e-05 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:34 2.b 2:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.132667541000004, 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028916179999999999, 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975433+0000", 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9599999999999999e-07 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975438+0000", 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975448+0000", 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0145e-05 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975455+0000", 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9149999999999999e-06 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975979+0000", 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052412100000000001 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976090+0000", 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111307 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978324+0000", 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022336130000000002 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.132606901000003, 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026148299999999998, 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975496+0000", 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975504+0000", 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-19T05:43:20.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975510+0000", 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975620+0000", 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109602 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975717+0000", 2024-09-19T05:43:20.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7676999999999993e-05 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978108+0000", 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023904070000000002 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:4 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.132310152000002, 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022744829999999999, 2024-09-19T05:43:20.138 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6600000000000005e-07 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975797+0000", 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3790000000000003e-06 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975805+0000", 2024-09-19T05:43:20.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:43:20.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975812+0000", 2024-09-19T05:43:20.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.029e-06 2024-09-19T05:43:20.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975861+0000", 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8266999999999997e-05 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975882+0000", 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896000000000001e-05 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978029+0000", 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021471490000000001 2024-09-19T05:43:20.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978064+0000", 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:5 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.131038605999997, 2024-09-19T05:43:20.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001037481, 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:20.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977066+0000", 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2250000000000002e-06 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977087+0000", 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977095+0000", 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7889999999999995e-06 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977159+0000", 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977190+0000", 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1526999999999998e-05 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000881224 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978099+0000", 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6962999999999999e-05 2024-09-19T05:43:20.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.130457028999999, 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046917399999999999, 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2399999999999997e-07 2024-09-19T05:43:20.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977647+0000", 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977671+0000", 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3926000000000001e-05 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977679+0000", 2024-09-19T05:43:20.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.593e-06 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977789+0000", 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109969 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978112+0000", 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032325799999999999 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.130378679000003, 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029909400000000002, 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:20.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977722+0000", 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977725+0000", 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3830000000000001e-06 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977737+0000", 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2194999999999999e-05 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977920+0000", 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017706699999999999 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977927+0000", 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978020+0000", 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3358999999999998e-05 2024-09-19T05:43:20.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:28 2.b e22/19 ondisk, result = 0)", 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.130145268, 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024615999999999999, 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-19T05:43:20.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977960+0000", 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3630000000000004e-06 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977970+0000", 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977978+0000", 2024-09-19T05:43:20.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0209999999999999e-06 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978088+0000", 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109578 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9630000000000001e-06 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978201+0000", 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106018 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.b epoch 22/19 rep_tid 4 lcod 22'2)", 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.130108864, 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033492, 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:20.161 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:20.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977999+0000", 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8690000000000004e-06 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978007+0000", 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:20.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3929999999999999e-06 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978223+0000", 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000211051 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978326+0000", 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102874 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:25 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.129917868, 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033417400000000002, 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978186+0000", 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6349999999999999e-06 2024-09-19T05:43:20.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978195+0000", 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7870000000000007e-06 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978367+0000", 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017261800000000001 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978420+0000", 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2339999999999997e-05 2024-09-19T05:43:20.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978427+0000", 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.25e-06 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978516+0000", 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9038999999999999e-05 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:26 2.1f e22/19 2:fe07bba1:::100.00000000.inode:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.129356133999998, 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047866199999999998, 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978750+0000", 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4809999999999997e-06 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978759+0000", 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1300000000000007e-06 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978763+0000", 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3290000000000004e-06 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978801+0000", 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8127999999999998e-05 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978815+0000", 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3523000000000001e-05 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979197+0000", 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038223899999999998 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979222+0000", 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5219999999999999e-05 2024-09-19T05:43:20.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1f epoch 22/19 rep_tid 9 entries 22'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.128913709999999, 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033723899999999997, 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979187+0000", 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4270000000000002e-06 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979206+0000", 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313000000000001e-05 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979214+0000", 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420000000000001e-06 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979253+0000", 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.892e-05 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979523+0000", 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000270547 2024-09-19T05:43:20.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:20.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:20.177 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:20.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:20.179 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:20.177+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:20.280 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:20.278+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:20.293 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:20.293 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:20.293 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:20.293 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:20.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:20.381 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:20.379+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:20.447 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:20.448 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:20.448 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:20.448 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:20.448 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:20.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:20.482 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:20.480+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:20.563 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:20.563 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:20.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:20.563 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:20.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.563 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.563 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.585505984000001, 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:43:20.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:43:20.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:20.569 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.584630326999999, 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:43:20.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.584006146999997, 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:43:20.575 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:43:20.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:43:20.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:43:20.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:43:20.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:43:20.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583885408999997, 2024-09-19T05:43:20.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:20.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:43:20.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:43:20.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583846119999997, 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:43:20.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:43:20.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:20.589 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583781620000003, 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:20.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:43:20.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:43:20.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583718202, 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.595 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:43:20.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:43:20.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:43:20.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583668547000002, 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:43:20.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.606 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583612377999998, 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:43:20.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:20.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583585386999999, 2024-09-19T05:43:20.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:43:20.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:43:20.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583506939999999, 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.618 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:43:20.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.623 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583473259000002, 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:43:20.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583450247999998, 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:43:20.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:43:20.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583416853000003, 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:43:20.635 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:43:20.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:43:20.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583374173999999, 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:20.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:43:20.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.646 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583335009000002, 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:43:20.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:43:20.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583302981999999, 2024-09-19T05:43:20.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:43:20.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583104448999997, 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:20.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:43:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.583033270999998, 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:43:20.661 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:20.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 42.576817112999997, 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:20.665 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:43:20.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.669 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.670 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.670 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:20.670 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:20.670 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:20.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:20.672 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:20.669+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:20.772 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:20.770+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:20.816 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:20.816 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:20.816 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:20.816 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:20.825 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:20.873 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:20.871+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:20.927 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:20.928 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:20.928 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:20.928 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:20.928 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:20.936 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:20.974 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:20.972+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:21.075 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:21.073+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:21.080 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:21.080 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.559223111999998, 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:43:21.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:43:21.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.558865144000002, 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:43:21.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:43:21.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.558364752999999, 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:43:21.090 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:43:21.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.094 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.55137852, 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:43:21.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:43:21.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:21.099 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.550920712, 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:43:21.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:43:21.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.528207092000002, 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.104 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.108 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 57.527712147000003, 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:21.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.965909191000001, 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:43:21.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.965151415000001, 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:43:21.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:43:21.121 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.964570367999997, 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:43:21.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:43:21.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.963935321999998, 2024-09-19T05:43:21.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:43:21.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.130 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.958128815999999, 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:21.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.957472643999999, 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.480923969999999, 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:43:21.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:43:21.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:43:21.143 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.480555656, 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:43:21.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:43:21.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.480176745999998, 2024-09-19T05:43:21.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:43:21.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.152 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.479479892999997, 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:21.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 56.473695278999998, 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:43:21.157 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:43:21.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.102580265, 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:21.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.102537181999999, 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:21.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.171 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.171 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.171 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:21.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:21.216 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:21.214+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:21.276 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:21.276 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:21.276 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:21.276 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:21.276 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:21.274+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:21.284 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:21.377 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:21.375+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:21.420 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:21.427 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:21.427 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:21.428 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:21.433 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:43:21.436 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:21.478 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:21.476+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:21.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.526+0000 7f1d03277640 1 Processor -- start 2024-09-19T05:43:21.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.526+0000 7f1d03277640 1 -- start start 2024-09-19T05:43:21.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d03277640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1cfc0cd4e0 0x7f1cfc0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d03277640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cfc15aa90 0x7f1cfc0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d03277640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd8c0 0x7f1cfc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d03277640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f1cfc15eec0 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d03277640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1cfc15ebe0 con 0x7f1cfc15aa90 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d03277640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1cfc0b2e00 con 0x7f1cfc0cd8c0 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1cfc0cd4e0 0x7f1cfc0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1cfc0cd4e0 0x7f1cfc0b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59134/0 (socket says 172.21.15.2:59134) 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 -- 172.21.15.2:0/2716911586 learned_addr learned my addr 172.21.15.2:0/2716911586 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01814640 1 --2- 172.21.15.2:0/2716911586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd8c0 0x7f1cfc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d00812640 1 --2- 172.21.15.2:0/2716911586 >> v2:172.21.15.123:3300/0 conn(0x7f1cfc15aa90 0x7f1cfc0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 -- 172.21.15.2:0/2716911586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd8c0 msgr2=0x7f1cfc15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 --2- 172.21.15.2:0/2716911586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd8c0 0x7f1cfc15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 -- 172.21.15.2:0/2716911586 >> v2:172.21.15.123:3300/0 conn(0x7f1cfc15aa90 msgr2=0x7f1cfc0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:21.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 --2- 172.21.15.2:0/2716911586 >> v2:172.21.15.123:3300/0 conn(0x7f1cfc15aa90 0x7f1cfc0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.528+0000 7f1d01013640 1 -- 172.21.15.2:0/2716911586 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc0b30e0 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.530+0000 7f1d01013640 1 --2- 172.21.15.2:0/2716911586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc0cd4e0 0x7f1cfc0b3bc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc15edf0 tx=0x7f1cf802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73388cbc3e5ee26c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.530+0000 7f1cf3fff640 1 -- 172.21.15.2:0/2716911586 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8037070 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.530+0000 7f1cf3fff640 1 -- 172.21.15.2:0/2716911586 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf8032440 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.530+0000 7f1cf3fff640 1 -- 172.21.15.2:0/2716911586 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8032810 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 -- 172.21.15.2:0/2716911586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc0cd4e0 msgr2=0x7f1cfc0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 --2- 172.21.15.2:0/2716911586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc0cd4e0 0x7f1cfc0b3bc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc15edf0 tx=0x7f1cf802a1c0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 -- 172.21.15.2:0/2716911586 shutdown_connections 2024-09-19T05:43:21.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 --2- 172.21.15.2:0/2716911586 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd8c0 0x7f1cfc15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 --2- 172.21.15.2:0/2716911586 >> v2:172.21.15.123:3300/0 conn(0x7f1cfc15aa90 0x7f1cfc0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 --2- 172.21.15.2:0/2716911586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc0cd4e0 0x7f1cfc0b3bc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 -- 172.21.15.2:0/2716911586 >> 172.21.15.2:0/2716911586 conn(0x7f1cfc067b30 msgr2=0x7f1cfc067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:21.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 -- 172.21.15.2:0/2716911586 shutdown_connections 2024-09-19T05:43:21.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.531+0000 7f1d03277640 1 -- 172.21.15.2:0/2716911586 wait complete. 2024-09-19T05:43:21.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.533+0000 7f1d03277640 1 Processor -- start 2024-09-19T05:43:21.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.533+0000 7f1d03277640 1 -- start start 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d03277640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd4e0 0x7f1cfc15c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d03277640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cfc0cd8c0 0x7f1cfc0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d03277640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc15aa90 0x7f1cfc14b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d01814640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc15aa90 0x7f1cfc14b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d01013640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd4e0 0x7f1cfc15c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d01013640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd4e0 0x7f1cfc15c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:36460/0 (socket says 172.21.15.2:36460) 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d01814640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc15aa90 0x7f1cfc14b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59140/0 (socket says 172.21.15.2:59140) 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d00812640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cfc0cd8c0 0x7f1cfc0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d00812640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cfc0cd8c0 0x7f1cfc0c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42364/0 (socket says 172.21.15.2:42364) 2024-09-19T05:43:21.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d01013640 1 -- 172.21.15.2:0/7065615 learned_addr learned my addr 172.21.15.2:0/7065615 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d03277640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc15e470 con 0x7f1cfc15aa90 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d03277640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1cfc0b36a0 con 0x7f1cfc0cd8c0 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.534+0000 7f1d03277640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1cfc15e5f0 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d01013640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cfc0cd8c0 msgr2=0x7f1cfc0c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cfc0cd8c0 0x7f1cfc0c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d01013640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc15aa90 msgr2=0x7f1cfc14b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc15aa90 0x7f1cfc14b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d01013640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cfc1f0320 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d01814640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cfc15aa90 0x7f1cfc14b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd4e0 0x7f1cfc15c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1cf802a200 tx=0x7f1cf80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8037040 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf8042070 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8030880 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.535+0000 7f1d03277640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cfc1f3a30 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.536+0000 7f1d03277640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cfc1f08c0 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.537+0000 7f1d03277640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f1cfc15df30 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.539+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1cf802ad10 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.540+0000 7f1cf1ffb640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ce006c380 0x7f1ce006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.540+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 5738+0+0 (secure 0 0 0) 0x7f1cf80aba20 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.541+0000 7f1d00812640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ce006c380 0x7f1ce006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.542+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v41) ==== 54+0+16 (secure 0 0 0) 0x7f1cf8079cb0 con 0x7f1cfc0cd4e0 2024-09-19T05:43:21.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.542+0000 7f1d00812640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ce006c380 0x7f1ce006e840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0b5fc0 tx=0x7f1cec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.543+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f1cd8001650 0x7f1cd8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.543+0000 7f1d01814640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f1cd8001650 0x7f1cd8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.543+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cd8006bb0 con 0x7f1cd8001650 2024-09-19T05:43:21.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.544+0000 7f1d01814640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f1cd8001650 0x7f1cd8003b10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.545+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd8006bb0 con 0x7f1cd8001650 2024-09-19T05:43:21.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.561+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd8005c60 con 0x7f1cd8001650 2024-09-19T05:43:21.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.564+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd8005c60 con 0x7f1cd8001650 2024-09-19T05:43:21.566 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.564+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f1cd8004190 0x7f1cd8004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.564+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1cd8004c00 con 0x7f1cd8004190 2024-09-19T05:43:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.564+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f1cd8004190 0x7f1cd8004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.565+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f1cd8004190 0x7f1cd8004640 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.566+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd8004c00 con 0x7f1cd8004190 2024-09-19T05:43:21.578 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:21.576+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:21.582 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:21.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 43.604943104999997, 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:21.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:21.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:21.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:21.587 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:21.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.580+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd8005140 con 0x7f1cd8004190 2024-09-19T05:43:21.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.583+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd8005140 con 0x7f1cd8004190 2024-09-19T05:43:21.588 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.583+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f1cd8009370 0x7f1cd800b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.583+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f1cd800e6c0 con 0x7f1cd8009370 2024-09-19T05:43:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.584+0000 7f1d00812640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f1cd8009370 0x7f1cd800b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.584+0000 7f1d00812640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f1cd8009370 0x7f1cd800b760 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.585+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd800e6c0 con 0x7f1cd8009370 2024-09-19T05:43:21.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:21.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.599+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd800d6b0 con 0x7f1cd8009370 2024-09-19T05:43:21.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.600+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd800d6b0 con 0x7f1cd8009370 2024-09-19T05:43:21.603 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:21.604 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:21.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.601+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f1cd800bca0 0x7f1cd800c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.601+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f1cd800c690 con 0x7f1cd800bca0 2024-09-19T05:43:21.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.601+0000 7f1d01814640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f1cd800bca0 0x7f1cd800c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.601+0000 7f1d01814640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f1cd800bca0 0x7f1cd800c150 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.602+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd800c690 con 0x7f1cd800bca0 2024-09-19T05:43:21.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.605+0000 7f33b9cb7640 1 Processor -- start 2024-09-19T05:43:21.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.605+0000 7f33b9cb7640 1 -- start start 2024-09-19T05:43:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.606+0000 7f33b9cb7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4bd0 0x7f33b40c9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.606+0000 7f33b9cb7640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f33b40c4fb0 0x7f33b4073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.606+0000 7f33b9cb7640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b41583e0 con 0x7f33b41578c0 2024-09-19T05:43:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.606+0000 7f33b9cb7640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f33b415ae30 con 0x7f33b40c4fb0 2024-09-19T05:43:21.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.606+0000 7f33b9cb7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f33b41586c0 con 0x7f33b40c4bd0 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b37fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4bd0 0x7f33b40c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b2ffd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f33b40c4fb0 0x7f33b4073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b2ffd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f33b40c4fb0 0x7f33b4073290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42372/0 (socket says 172.21.15.2:42372) 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b2ffd640 1 -- 172.21.15.2:0/1125347957 learned_addr learned my addr 172.21.15.2:0/1125347957 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b2ffd640 1 -- 172.21.15.2:0/1125347957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4bd0 msgr2=0x7f33b40c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b2ffd640 1 --2- 172.21.15.2:0/1125347957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4bd0 0x7f33b40c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b2ffd640 1 -- 172.21.15.2:0/1125347957 >> v1:172.21.15.123:6789/0 conn(0x7f33b41578c0 legacy=0x7f33b4151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.607+0000 7f33b2ffd640 1 -- 172.21.15.2:0/1125347957 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b415b080 con 0x7f33b40c4fb0 2024-09-19T05:43:21.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.608+0000 7f33b2ffd640 1 --2- 172.21.15.2:0/1125347957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4fb0 0x7f33b4073290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33a800b920 tx=0x7f33a802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58993696833a2c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.608+0000 7f33b27fc640 1 -- 172.21.15.2:0/1125347957 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a802f710 con 0x7f33b40c4fb0 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.608+0000 7f33b27fc640 1 -- 172.21.15.2:0/1125347957 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a802f8b0 con 0x7f33b40c4fb0 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.608+0000 7f33b27fc640 1 -- 172.21.15.2:0/1125347957 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a802fc80 con 0x7f33b40c4fb0 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 -- 172.21.15.2:0/1125347957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4fb0 msgr2=0x7f33b4073290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/1125347957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4fb0 0x7f33b4073290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33a800b920 tx=0x7f33a802a6f0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 -- 172.21.15.2:0/1125347957 shutdown_connections 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/1125347957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4fb0 0x7f33b4073290 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/1125347957 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4bd0 0x7f33b40c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 -- 172.21.15.2:0/1125347957 >> 172.21.15.2:0/1125347957 conn(0x7f33b406f3d0 msgr2=0x7f33b40707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 -- 172.21.15.2:0/1125347957 shutdown_connections 2024-09-19T05:43:21.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.609+0000 7f33b9cb7640 1 -- 172.21.15.2:0/1125347957 wait complete. 2024-09-19T05:43:21.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.610+0000 7f33b9cb7640 1 Processor -- start 2024-09-19T05:43:21.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.610+0000 7f33b9cb7640 1 -- start start 2024-09-19T05:43:21.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b9cb7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4bd0 0x7f33b40c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b9cb7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4fb0 0x7f33b41ed0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b9cb7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b41578c0 0x7f33b41ed610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b9cb7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f33b415bb60 con 0x7f33b41578c0 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b9cb7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33b40737d0 con 0x7f33b40c4bd0 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b9cb7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f33b415bce0 con 0x7f33b40c4fb0 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b41578c0 0x7f33b41ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b41578c0 0x7f33b41ed610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59154/0 (socket says 172.21.15.2:59154) 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b3fff640 1 -- 172.21.15.2:0/159889127 learned_addr learned my addr 172.21.15.2:0/159889127 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b2ffd640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4fb0 0x7f33b41ed0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.611+0000 7f33b37fe640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4bd0 0x7f33b40c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b2ffd640 1 -- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4bd0 msgr2=0x7f33b40c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b2ffd640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4bd0 0x7f33b40c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b2ffd640 1 -- 172.21.15.2:0/159889127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b41578c0 msgr2=0x7f33b41ed610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b2ffd640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b41578c0 0x7f33b41ed610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b2ffd640 1 -- 172.21.15.2:0/159889127 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b406d500 con 0x7f33b40c4fb0 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b2ffd640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4fb0 0x7f33b41ed0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f33a8031000 tx=0x7f33a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f339bfff640 1 -- 172.21.15.2:0/159889127 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a803f070 con 0x7f33b40c4fb0 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f339bfff640 1 -- 172.21.15.2:0/159889127 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a8031b40 con 0x7f33b40c4fb0 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b414a010 con 0x7f33b40c4fb0 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f339bfff640 1 -- 172.21.15.2:0/159889127 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8043410 con 0x7f33b40c4fb0 2024-09-19T05:43:21.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.612+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b4159bf0 con 0x7f33b40c4fb0 2024-09-19T05:43:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.613+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b4073290 con 0x7f33b40c4fb0 2024-09-19T05:43:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.614+0000 7f339bfff640 1 -- 172.21.15.2:0/159889127 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f33a80435b0 con 0x7f33b40c4fb0 2024-09-19T05:43:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.615+0000 7f339bfff640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33a006c2b0 0x7f33a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.615+0000 7f339bfff640 1 -- 172.21.15.2:0/159889127 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(41..41 src has 1..41) ==== 5709+0+0 (secure 0 0 0) 0x7f33a80ac5a0 con 0x7f33b40c4fb0 2024-09-19T05:43:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.615+0000 7f33b37fe640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33a006c2b0 0x7f33a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.615+0000 7f33b37fe640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33a006c2b0 0x7f33a006e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33a4015230 tx=0x7f33a400f040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.619+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd800c690 con 0x7f1cd800bca0 2024-09-19T05:43:21.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.619+0000 7f339bfff640 1 -- 172.21.15.2:0/159889127 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f33a807a620 con 0x7f33b40c4fb0 2024-09-19T05:43:21.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.622+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd800c690 con 0x7f1cd800bca0 2024-09-19T05:43:21.624 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:21.625 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:21.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.622+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f1cd8010910 0x7f1cd8012d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.622+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f1cd800d1a0 con 0x7f1cd8010910 2024-09-19T05:43:21.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.622+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f1cd8010910 0x7f1cd8012d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.622+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f1cd8010910 0x7f1cd8012d00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.623+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd800d1a0 con 0x7f1cd8010910 2024-09-19T05:43:21.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.637+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd800d1a0 con 0x7f1cd8010910 2024-09-19T05:43:21.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.638+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd800d1a0 con 0x7f1cd8010910 2024-09-19T05:43:21.641 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:21.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.639+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f1cd80137b0 0x7f1cd8015ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.639+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f1cd8018be0 con 0x7f1cd80137b0 2024-09-19T05:43:21.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.639+0000 7f1d00812640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f1cd80137b0 0x7f1cd8015ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.639+0000 7f1d00812640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f1cd80137b0 0x7f1cd8015ba0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.640+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd8018be0 con 0x7f1cd80137b0 2024-09-19T05:43:21.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.654+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd8000950 con 0x7f1cd80137b0 2024-09-19T05:43:21.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.655+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd8000950 con 0x7f1cd80137b0 2024-09-19T05:43:21.658 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:21.658 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:21.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.656+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f1cd8017e40 0x7f1cd80182a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.656+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f1cd801bfb0 con 0x7f1cd8017e40 2024-09-19T05:43:21.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.656+0000 7f1d01814640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f1cd8017e40 0x7f1cd80182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.656+0000 7f1d01814640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f1cd8017e40 0x7f1cd80182a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.657+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd801bfb0 con 0x7f1cd8017e40 2024-09-19T05:43:21.679 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:21.678+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:21.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.679+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd80161f0 con 0x7f1cd8017e40 2024-09-19T05:43:21.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.681+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd80161f0 con 0x7f1cd8017e40 2024-09-19T05:43:21.683 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:21.684 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:21.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.681+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1cd8016870 0x7f1cd8016c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:21.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.681+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f1cd80173b0 con 0x7f1cd8016870 2024-09-19T05:43:21.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.681+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1cd8016870 0x7f1cd8016c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:21.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.681+0000 7f1d01013640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1cd8016870 0x7f1cd8016c90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:21.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.682+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1cd80173b0 con 0x7f1cd8016870 2024-09-19T05:43:21.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.709+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1cd80173b0 con 0x7f1cd8016870 2024-09-19T05:43:21.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cf1ffb640 1 -- 172.21.15.2:0/7065615 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1cd80173b0 con 0x7f1cd8016870 2024-09-19T05:43:21.713 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:21.714 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:21.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f1cd8001650 msgr2=0x7f1cd8003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f1cd8001650 0x7f1cd8003b10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f1cd8004190 msgr2=0x7f1cd8004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f1cd8004190 0x7f1cd8004640 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f1cd8009370 msgr2=0x7f1cd800b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f1cd8009370 0x7f1cd800b760 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f1cd800bca0 msgr2=0x7f1cd800c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f1cd800bca0 0x7f1cd800c150 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f1cd8010910 msgr2=0x7f1cd8012d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f1cd8010910 0x7f1cd8012d00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f1cd80137b0 msgr2=0x7f1cd8015ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f1cd80137b0 0x7f1cd8015ba0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f1cd8017e40 msgr2=0x7f1cd80182a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1d01814640 1 -- 172.21.15.2:0/7065615 reap_dead start 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f1cd8017e40 0x7f1cd80182a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1d01814640 1 -- 172.21.15.2:0/7065615 reap_dead start 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1cd8016870 msgr2=0x7f1cd8016c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1cd8016870 0x7f1cd8016c90 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1d01814640 1 -- 172.21.15.2:0/7065615 reap_dead start 2024-09-19T05:43:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ce006c380 msgr2=0x7f1ce006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ce006c380 0x7f1ce006e840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0b5fc0 tx=0x7f1cec008040 comp rx=0 tx=0).stop 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd4e0 msgr2=0x7f1cfc15c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.711+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd4e0 0x7f1cfc15c630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1cf802a200 tx=0x7f1cf80046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.712+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 shutdown_connections 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.712+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1ce006c380 0x7f1ce006e840 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.712+0000 7f1cdf7fe640 1 --2- 172.21.15.2:0/7065615 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cfc0cd4e0 0x7f1cfc15c630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.712+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 >> 172.21.15.2:0/7065615 conn(0x7f1cfc067b30 msgr2=0x7f1cfc0b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.712+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 shutdown_connections 2024-09-19T05:43:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.712+0000 7f1cdf7fe640 1 -- 172.21.15.2:0/7065615 wait complete. 2024-09-19T05:43:21.735 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:21.735 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:21.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:21.781 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:21.779+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:21.882 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:21.880+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:21.888 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:21.888 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:21.897 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:21.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:21.895+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f33b414a6d0 con 0x7f33b40c4fb0 2024-09-19T05:43:21.983 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:21.981+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:22.041 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:22.041 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:22.050 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:22.094 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:22.092+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:22.169 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:22.170 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:22.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:22.221 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:22.219+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:22.286 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:22.284+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:22.350 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:22.350 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:22.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:22.386 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:22.385+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:22.487 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:22.485+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:22.528 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:22.528 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:22.538 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.549+0000 7f339bfff640 1 -- 172.21.15.2:0/159889127 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v42) ==== 101+0+0 (secure 0 0 0) 0x7f33a807e920 con 0x7f33b40c4fb0 2024-09-19T05:43:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:43:22.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33a006c2b0 msgr2=0x7f33a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33a006c2b0 0x7f33a006e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f33a4015230 tx=0x7f33a400f040 comp rx=0 tx=0).stop 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4fb0 msgr2=0x7f33b41ed0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4fb0 0x7f33b41ed0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f33a8031000 tx=0x7f33a80046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 shutdown_connections 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33a006c2b0 0x7f33a006e770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f33b41578c0 0x7f33b41ed610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f33b40c4fb0 0x7f33b41ed0d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 --2- 172.21.15.2:0/159889127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33b40c4bd0 0x7f33b40c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 >> 172.21.15.2:0/159889127 conn(0x7f33b406f3d0 msgr2=0x7f33b414aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 shutdown_connections 2024-09-19T05:43:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:22.552+0000 7f33b9cb7640 1 -- 172.21.15.2:0/159889127 wait complete. 2024-09-19T05:43:22.588 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:22.586+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:22.673 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:22.674 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:22.682 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:22.689 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:22.687+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:22.790 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:22.788+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:22.823 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:22.823 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:22.835 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:22.891 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:22.889+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:22.945 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:22.945 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:22.954 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:22.997 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:22.995+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:23.061 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:23.072 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:23.114 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:23.112+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:23.193 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:23.191+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:23.239 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:23.239 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:23.251 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:23.294 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:23.291+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:23.394 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:23.393+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:23.400 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:23.401 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:23.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:23.495 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:23.493+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:23.521 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:23.521 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:23.530 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:23.596 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:23.594+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:23.697 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:23.695+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:23.701 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:23.702 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:23.713 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:23.729 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:23.800 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:23.798+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:23.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.857+0000 7f2bae31f640 1 Processor -- start 2024-09-19T05:43:23.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.857+0000 7f2bae31f640 1 -- start start 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.858+0000 7f2bae31f640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2ba80c6a40 0x7f2ba81630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.858+0000 7f2bae31f640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba81585a0 con 0x7f2ba80c6660 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.858+0000 7f2bae31f640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba80c4860 con 0x7f2ba80cded0 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.858+0000 7f2bae31f640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba80c4b40 con 0x7f2ba80c6a40 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.859+0000 7f2ba7fff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f2ba80c6660 0x7f2ba815b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54742/0 (socket says 172.21.15.2:54742) 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.859+0000 7f2ba7fff640 1 -- 172.21.15.2:0/4257167387 learned_addr learned my addr 172.21.15.2:0/4257167387 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.859+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/4257167387 >> v2:172.21.15.2:3300/0 conn(0x7f2ba80c6a40 0x7f2ba81630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.859+0000 7f2bac8bc640 1 -- 172.21.15.2:0/4257167387 >> v1:172.21.15.123:6789/0 conn(0x7f2ba80cded0 legacy=0x7f2ba8151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:23.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.859+0000 7f2bac8bc640 1 -- 172.21.15.2:0/4257167387 >> v1:172.21.15.2:6789/0 conn(0x7f2ba80c6660 legacy=0x7f2ba815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:23.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.859+0000 7f2ba6ffd640 1 -- 172.21.15.2:0/4257167387 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537068991 0 0) 0x7f2ba80c4860 con 0x7f2ba80cded0 2024-09-19T05:43:23.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.859+0000 7f2bac8bc640 1 -- 172.21.15.2:0/4257167387 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba80c4e20 con 0x7f2ba80c6a40 2024-09-19T05:43:23.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.860+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/4257167387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80c6a40 0x7f2ba81630b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009d90 tx=0x7f2ba002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e652f2ff080f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.860+0000 7f2ba6ffd640 1 -- 172.21.15.2:0/4257167387 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0007740 con 0x7f2ba80c6a40 2024-09-19T05:43:23.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.860+0000 7f2ba6ffd640 1 -- 172.21.15.2:0/4257167387 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba00078e0 con 0x7f2ba80c6a40 2024-09-19T05:43:23.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.860+0000 7f2ba6ffd640 1 -- 172.21.15.2:0/4257167387 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0007cb0 con 0x7f2ba80c6a40 2024-09-19T05:43:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.861+0000 7f2bae31f640 1 -- 172.21.15.2:0/4257167387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80c6a40 msgr2=0x7f2ba81630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.861+0000 7f2bae31f640 1 --2- 172.21.15.2:0/4257167387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80c6a40 0x7f2ba81630b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009d90 tx=0x7f2ba002a560 comp rx=0 tx=0).stop 2024-09-19T05:43:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.861+0000 7f2bae31f640 1 -- 172.21.15.2:0/4257167387 shutdown_connections 2024-09-19T05:43:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.861+0000 7f2bae31f640 1 --2- 172.21.15.2:0/4257167387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80c6a40 0x7f2ba81630b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.861+0000 7f2bae31f640 1 -- 172.21.15.2:0/4257167387 >> 172.21.15.2:0/4257167387 conn(0x7f2ba80cd1e0 msgr2=0x7f2ba80cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.862+0000 7f2bae31f640 1 -- 172.21.15.2:0/4257167387 shutdown_connections 2024-09-19T05:43:23.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.862+0000 7f2bae31f640 1 -- 172.21.15.2:0/4257167387 wait complete. 2024-09-19T05:43:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.863+0000 7f2bae31f640 1 Processor -- start 2024-09-19T05:43:23.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.863+0000 7f2bae31f640 1 -- start start 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bae31f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ba80c6660 0x7f2ba814ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bae31f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba80c6a40 0x7f2ba814a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bae31f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80cded0 0x7f2ba8150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bae31f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba815bbf0 con 0x7f2ba80cded0 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bae31f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba815ba70 con 0x7f2ba80c6a40 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bae31f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba8151b90 con 0x7f2ba80c6660 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bac8bc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80cded0 0x7f2ba8150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2ba77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba80c6a40 0x7f2ba814a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bac8bc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80cded0 0x7f2ba8150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59168/0 (socket says 172.21.15.2:59168) 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2ba77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba80c6a40 0x7f2ba814a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42392/0 (socket says 172.21.15.2:42392) 2024-09-19T05:43:23.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2bac8bc640 1 -- 172.21.15.2:0/150112161 learned_addr learned my addr 172.21.15.2:0/150112161 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.864+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ba80c6660 0x7f2ba814ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bac8bc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ba80c6660 msgr2=0x7f2ba814ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2ba80c6660 0x7f2ba814ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bac8bc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba80c6a40 msgr2=0x7f2ba814a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba80c6a40 0x7f2ba814a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bac8bc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba814a5e0 con 0x7f2ba80cded0 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2ba77fe640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ba80c6a40 0x7f2ba814a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80cded0 0x7f2ba8150140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009d90 tx=0x7f2ba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0040070 con 0x7f2ba80cded0 2024-09-19T05:43:23.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba0031c80 con 0x7f2ba80cded0 2024-09-19T05:43:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba003c640 con 0x7f2ba80cded0 2024-09-19T05:43:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bae31f640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba8156620 con 0x7f2ba80cded0 2024-09-19T05:43:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.865+0000 7f2bae31f640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba81568c0 con 0x7f2ba80cded0 2024-09-19T05:43:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.867+0000 7f2bae31f640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f2ba814fd80 con 0x7f2ba80cded0 2024-09-19T05:43:23.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.867+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2ba0039040 con 0x7f2ba80cded0 2024-09-19T05:43:23.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.868+0000 7f2ba4ff9640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2b9406c2b0 0x7f2b9406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.872 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:23.872 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.868+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f2ba00ac530 con 0x7f2ba80cded0 2024-09-19T05:43:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.868+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2b9406c2b0 0x7f2b9406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.869+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v42) ==== 54+0+16 (secure 0 0 0) 0x7f2ba00aebb0 con 0x7f2ba80cded0 2024-09-19T05:43:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.869+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2b80001650 0x7f2b80003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.869+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b80006bb0 con 0x7f2b80001650 2024-09-19T05:43:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.869+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2b9406c2b0 0x7f2b9406e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b98007fd0 tx=0x7f2b98007e60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.869+0000 7f2ba77fe640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2b80001650 0x7f2b80003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.870+0000 7f2ba77fe640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2b80001650 0x7f2b80003b10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.871+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b80006bb0 con 0x7f2b80001650 2024-09-19T05:43:23.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:23.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.899+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b80005c60 con 0x7f2b80001650 2024-09-19T05:43:23.902 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:23.899+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:23.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.901+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b80005c60 con 0x7f2b80001650 2024-09-19T05:43:23.903 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.901+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2b80004190 0x7f2b80004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.901+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2b80004c00 con 0x7f2b80004190 2024-09-19T05:43:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.901+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2b80004190 0x7f2b80004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.902+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2b80004190 0x7f2b80004640 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.902+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b80004c00 con 0x7f2b80004190 2024-09-19T05:43:23.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.917+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b80005110 con 0x7f2b80004190 2024-09-19T05:43:23.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.918+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b80005110 con 0x7f2b80004190 2024-09-19T05:43:23.920 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:23.921 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:23.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.918+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2b80009370 0x7f2b8000b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.918+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2b8000e740 con 0x7f2b80009370 2024-09-19T05:43:23.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.919+0000 7f2ba77fe640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2b80009370 0x7f2b8000b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.919+0000 7f2ba77fe640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2b80009370 0x7f2b8000b760 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.920+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b8000e740 con 0x7f2b80009370 2024-09-19T05:43:23.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.933+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b8000d730 con 0x7f2b80009370 2024-09-19T05:43:23.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.935+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b8000d730 con 0x7f2b80009370 2024-09-19T05:43:23.937 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:23.937 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:23.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.935+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2b8000e1b0 0x7f2b8000bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.935+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f2b8000c470 con 0x7f2b8000e1b0 2024-09-19T05:43:23.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.935+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2b8000e1b0 0x7f2b8000bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.935+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2b8000e1b0 0x7f2b8000bd20 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.936+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b8000c470 con 0x7f2b8000e1b0 2024-09-19T05:43:23.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.954+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b8000c730 con 0x7f2b8000e1b0 2024-09-19T05:43:23.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.957+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b8000c730 con 0x7f2b8000e1b0 2024-09-19T05:43:23.959 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.957+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2b8000d040 0x7f2b800129b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.957+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f2b80015a50 con 0x7f2b8000d040 2024-09-19T05:43:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.957+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2b8000d040 0x7f2b800129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.958+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2b8000d040 0x7f2b800129b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.958+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b80015a50 con 0x7f2b8000d040 2024-09-19T05:43:23.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.986+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b8000c710 con 0x7f2b8000d040 2024-09-19T05:43:23.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.987+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b8000c710 con 0x7f2b8000d040 2024-09-19T05:43:23.990 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:23.990 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:23.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.988+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2b80015140 0x7f2b80013030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:23.990 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.988+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f2b80015620 con 0x7f2b80015140 2024-09-19T05:43:23.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.988+0000 7f2ba77fe640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2b80015140 0x7f2b80013030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:23.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.988+0000 7f2ba77fe640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2b80015140 0x7f2b80013030 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:23.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:23.989+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b80015620 con 0x7f2b80015140 2024-09-19T05:43:23.997 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:23.997 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:23.997 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:23.997 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:23.999 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:23.997+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:24.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.003+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b8000c3c0 con 0x7f2b80015140 2024-09-19T05:43:24.006 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:24.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.004+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b8000c3c0 con 0x7f2b80015140 2024-09-19T05:43:24.008 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:24.008 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:24.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.005+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2b80013a80 0x7f2b80013ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.005+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f2b8001c540 con 0x7f2b80013a80 2024-09-19T05:43:24.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.005+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2b80013a80 0x7f2b80013ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.005+0000 7f2bac8bc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2b80013a80 0x7f2b80013ee0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:24.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.006+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b8001c540 con 0x7f2b80013a80 2024-09-19T05:43:24.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.019+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b80019cb0 con 0x7f2b80013a80 2024-09-19T05:43:24.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.021+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b80019cb0 con 0x7f2b80013a80 2024-09-19T05:43:24.023 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.021+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2b8001a1c0 0x7f2b8001c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.021+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f2b8001f5f0 con 0x7f2b8001a1c0 2024-09-19T05:43:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.022+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2b8001a1c0 0x7f2b8001c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.024 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.022+0000 7f2ba7fff640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2b8001a1c0 0x7f2b8001c5b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:24.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.023+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b8001f5f0 con 0x7f2b8001a1c0 2024-09-19T05:43:24.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.050+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2b80014850 con 0x7f2b8001a1c0 2024-09-19T05:43:24.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2ba4ff9640 1 -- 172.21.15.2:0/150112161 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2b80014850 con 0x7f2b8001a1c0 2024-09-19T05:43:24.054 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2b80001650 msgr2=0x7f2b80003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f2b80001650 0x7f2b80003b10 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2b80004190 msgr2=0x7f2b80004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f2b80004190 0x7f2b80004640 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2b80009370 msgr2=0x7f2b8000b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f2b80009370 0x7f2b8000b760 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2bac8bc640 1 -- 172.21.15.2:0/150112161 reap_dead start 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2b8000e1b0 msgr2=0x7f2b8000bd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f2b8000e1b0 0x7f2b8000bd20 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.052+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2b8000d040 msgr2=0x7f2b800129b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2b8000d040 0x7f2b800129b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2b80015140 msgr2=0x7f2b80013030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2b80015140 0x7f2b80013030 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2b80013a80 msgr2=0x7f2b80013ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f2b80013a80 0x7f2b80013ee0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2b8001a1c0 msgr2=0x7f2b8001c5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f2b8001a1c0 0x7f2b8001c5b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2bac8bc640 1 -- 172.21.15.2:0/150112161 reap_dead start 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2b9406c2b0 msgr2=0x7f2b9406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2b9406c2b0 0x7f2b9406e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b98007fd0 tx=0x7f2b98007e60 comp rx=0 tx=0).stop 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80cded0 msgr2=0x7f2ba8150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80cded0 0x7f2ba8150140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009d90 tx=0x7f2ba0004a40 comp rx=0 tx=0).stop 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 shutdown_connections 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2b9406c2b0 0x7f2b9406e770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 --2- 172.21.15.2:0/150112161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba80cded0 0x7f2ba8150140 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 >> 172.21.15.2:0/150112161 conn(0x7f2ba80cd1e0 msgr2=0x7f2ba8151250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 shutdown_connections 2024-09-19T05:43:24.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.053+0000 7f2b8e7fc640 1 -- 172.21.15.2:0/150112161 wait complete. 2024-09-19T05:43:24.107 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:24.105+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:24.112 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:24.112 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:24.112 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:24.112 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:24.112 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:24.121 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:24.202 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:24.200+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.247284925999999, 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:43:24.225 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:43:24.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:43:24.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.246409268999997, 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:43:24.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245785089000002, 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.236 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.237 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:43:24.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:43:24.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245664351000002, 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:43:24.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:43:24.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:24.246 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245625062000002, 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:24.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:24.250 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245560562000001, 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:43:24.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:43:24.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.256 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245497143999998, 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:43:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:43:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245447489, 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.262 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:43:24.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:43:24.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:43:24.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:43:24.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245391320000003, 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:43:24.267 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:43:24.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.272 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245364328999997, 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:43:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245285881999997, 2024-09-19T05:43:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:43:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245252201, 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:43:24.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:43:24.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.289 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245229190000003, 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:43:24.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245195795000001, 2024-09-19T05:43:24.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:43:24.300 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245153115999997, 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:24.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:43:24.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:43:24.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:24.306 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245113951, 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:24.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:43:24.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:43:24.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.245081923999997, 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:43:24.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.244883391000002, 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.316 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:43:24.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.244812213000003, 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:43:24.324 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.238596055000002, 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:24.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.329 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.329 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:24.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:24.331 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:24.328+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:24.428 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:24.428 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:24.428 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-19T05:43:24.428 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T05:43:24.428 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T05:43:24.429 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-19T05:43:24.430 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:24.428+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:24.433 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:24.433 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:24.433 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:24.434 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:24.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:24.531 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:24.529+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:24.543 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:24.543 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T05:43:24.543 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:24.569 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:43:24.588 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:24.588 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:24.588 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:24.588 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:24.588 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:24.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:24.632 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:24.630+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:24.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.671+0000 7ff0822b7640 1 Processor -- start 2024-09-19T05:43:24.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.671+0000 7ff0822b7640 1 -- start start 2024-09-19T05:43:24.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff0822b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 0x7ff07c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff07bfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff07c154c20 0x7ff07c073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54750/0 (socket says 172.21.15.2:54750) 2024-09-19T05:43:24.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff07bfff640 1 -- 172.21.15.2:0/4019443840 learned_addr learned my addr 172.21.15.2:0/4019443840 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:24.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff0822b7640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff07c0ccbc0 con 0x7ff07c154c20 2024-09-19T05:43:24.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff073fff640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 0x7ff07c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff0822b7640 1 -- 172.21.15.2:0/4019443840 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff07c0cc600 con 0x7ff07c0caf40 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff0822b7640 1 -- 172.21.15.2:0/4019443840 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c0cc8e0 con 0x7ff07c155000 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396660917 0 0) 0x7ff07c0ccbc0 con 0x7ff07c154c20 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff068003520 con 0x7ff07c154c20 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617789279 0 0) 0x7ff07c0cc600 con 0x7ff07c0caf40 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff07c0ccbc0 con 0x7ff07c0caf40 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.673+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4063406305 0 0) 0x7ff07c0ccbc0 con 0x7ff07c0caf40 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 msgr2=0x7ff07c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 0x7ff07c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 >> v1:172.21.15.2:6789/0 conn(0x7ff07c154c20 legacy=0x7ff07c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c0cbc60 con 0x7ff07c0caf40 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7ff0740019f0 con 0x7ff07c0caf40 2024-09-19T05:43:24.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 >> v1:172.21.15.123:6789/0 conn(0x7ff07c0caf40 legacy=0x7ff07c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff068004ba0 0x7ff068004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 0x7ff0680099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff068003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c0cc8e0 con 0x7ff068004ba0 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff068004370 con 0x7ff0680037a0 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff068004570 con 0x7ff07c155000 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff080854640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff068003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff073fff640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 0x7ff0680099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07bfff640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff068004ba0 0x7ff068004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff080854640 1 -- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 msgr2=0x7ff0680099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff080854640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 0x7ff0680099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff080854640 1 -- 172.21.15.2:0/4019443840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff068004ba0 msgr2=0x7ff068004fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff080854640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff068004ba0 0x7ff068004fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff080854640 1 -- 172.21.15.2:0/4019443840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c0ccbc0 con 0x7ff0680037a0 2024-09-19T05:43:24.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff07bfff640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff068004ba0 0x7ff068004fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:24.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.674+0000 7ff073fff640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c155000 0x7ff0680099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.675+0000 7ff080854640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff068003c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c3e0 tx=0x7ff068005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4204297536a305f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.675+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074033070 con 0x7ff0680037a0 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07402f6a0 con 0x7ff0680037a0 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff07b7fe640 1 -- 172.21.15.2:0/4019443840 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07400c0e0 con 0x7ff0680037a0 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff0822b7640 1 -- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 msgr2=0x7ff068003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff0822b7640 1 --2- 172.21.15.2:0/4019443840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff068003c20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c3e0 tx=0x7ff068005830 comp rx=0 tx=0).stop 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff080854640 1 -- 172.21.15.2:0/4019443840 reap_dead start 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff0822b7640 1 -- 172.21.15.2:0/4019443840 shutdown_connections 2024-09-19T05:43:24.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff0822b7640 1 -- 172.21.15.2:0/4019443840 >> 172.21.15.2:0/4019443840 conn(0x7ff07c06d0e0 msgr2=0x7ff07c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:24.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff0822b7640 1 -- 172.21.15.2:0/4019443840 shutdown_connections 2024-09-19T05:43:24.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.676+0000 7ff0822b7640 1 -- 172.21.15.2:0/4019443840 wait complete. 2024-09-19T05:43:24.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.678+0000 7ff0822b7640 1 Processor -- start 2024-09-19T05:43:24.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.679+0000 7ff0822b7640 1 -- start start 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff0822b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff07c1f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff0822b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c0c99a0 0x7ff07c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff0822b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07c1e0fd0 0x7ff07c14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff07c1f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff073fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c0c99a0 0x7ff07c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff07c1f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42412/0 (socket says 172.21.15.2:42412) 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff073fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c0c99a0 0x7ff07c0b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:36520/0 (socket says 172.21.15.2:36520) 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff07bfff640 1 -- 172.21.15.2:0/3175708127 learned_addr learned my addr 172.21.15.2:0/3175708127 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.680+0000 7ff080854640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07c1e0fd0 0x7ff07c14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c073a50 con 0x7ff07c1e0fd0 2024-09-19T05:43:24.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c0cb430 con 0x7ff0680037a0 2024-09-19T05:43:24.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c073bd0 con 0x7ff07c0c99a0 2024-09-19T05:43:24.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff07bfff640 1 -- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c0c99a0 msgr2=0x7ff07c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff07bfff640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c0c99a0 0x7ff07c0b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff07bfff640 1 -- 172.21.15.2:0/3175708127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07c1e0fd0 msgr2=0x7ff07c14e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:24.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff07bfff640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07c1e0fd0 0x7ff07c14e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff07bfff640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c1f1df0 con 0x7ff0680037a0 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff073fff640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c0c99a0 0x7ff07c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.681+0000 7ff080854640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07c1e0fd0 0x7ff07c14e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.682+0000 7ff07bfff640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff07c1f18b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0016f0 tx=0x7ff06c001af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.682+0000 7ff0797fa640 1 -- 172.21.15.2:0/3175708127 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c019070 con 0x7ff0680037a0 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.682+0000 7ff0797fa640 1 -- 172.21.15.2:0/3175708127 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c0047e0 con 0x7ff0680037a0 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.682+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c1f43f0 con 0x7ff0680037a0 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.682+0000 7ff0797fa640 1 -- 172.21.15.2:0/3175708127 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c004b90 con 0x7ff0680037a0 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.682+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c0ca680 con 0x7ff0680037a0 2024-09-19T05:43:24.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.683+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff07c152210 con 0x7ff0680037a0 2024-09-19T05:43:24.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.684+0000 7ff0797fa640 1 -- 172.21.15.2:0/3175708127 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff06c01c070 con 0x7ff0680037a0 2024-09-19T05:43:24.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.684+0000 7ff0797fa640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff05c06c2e0 0x7ff05c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:24.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.685+0000 7ff0797fa640 1 -- 172.21.15.2:0/3175708127 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7ff06c08aa30 con 0x7ff0680037a0 2024-09-19T05:43:24.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.685+0000 7ff073fff640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff05c06c2e0 0x7ff05c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:24.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.685+0000 7ff073fff640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff05c06c2e0 0x7ff05c06e7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff07c1eeab0 tx=0x7ff064008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.689+0000 7ff0797fa640 1 -- 172.21.15.2:0/3175708127 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff07c152210 con 0x7ff0680037a0 2024-09-19T05:43:24.733 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:24.732+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:24.742 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:24.742 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:24.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:24.742 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:24.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.742 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:43:24.742 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.221417760000001, 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:43:24.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.221059791999998, 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:43:24.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.751 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.220559401000003, 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:43:24.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:43:24.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:24.756 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.213573168000003, 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.213115360000003, 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:24.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:43:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:43:24.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.190401739999999, 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.765 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:43:24.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:43:24.769 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.189906794999999, 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:43:24.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.773 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.628103838999998, 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:24.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.627346062999997, 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.778 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:43:24.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.782 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.626765016, 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:24.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:43:24.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:43:24.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.626129970000001, 2024-09-19T05:43:24.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:24.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.791 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.620323464000002, 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:43:24.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:43:24.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.796 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.619667292000003, 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.143118618000003, 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.801 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:43:24.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.805 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.142750304000003, 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:43:24.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.142371394000001, 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:43:24.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.141674541, 2024-09-19T05:43:24.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:43:24.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:43:24.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 60.135889927000001, 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.764774912999997, 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.823 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:43:24.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:43:24.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 46.764731830000002, 2024-09-19T05:43:24.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:24.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:24.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:24.831 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:24.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:24.834 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:24.832+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:24.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:24.876+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7ff07c0c5770 con 0x7ff0680037a0 2024-09-19T05:43:24.934 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:24.933+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:24.977 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:24.977 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:24.977 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:24.977 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:24.986 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:25.070 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:25.068+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:25.128 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:25.128 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:25.128 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:25.128 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:25.128 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:25.135 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:25.134+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:25.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:25.236 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:25.234+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.302784377999998, 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.280 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.281 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.283 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.284 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.284 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.284 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.284 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:25.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:25.337 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:25.335+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:25.391 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:25.391 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:25.392 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:25.392 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:25.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:25.438 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:25.436+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:25.539 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:25.537+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:25.542 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:25.542 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:25.542 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:25.542 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:25.542 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:25.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:25.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.560+0000 7ff0797fa640 1 -- 172.21.15.2:0/3175708127 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v43) ==== 87+0+0 (secure 0 0 0) 0x7ff06c058a20 con 0x7ff0680037a0 2024-09-19T05:43:25.563 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:43:25.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff05c06c2e0 msgr2=0x7ff05c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:25.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff05c06c2e0 0x7ff05c06e7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff07c1eeab0 tx=0x7ff064008040 comp rx=0 tx=0).stop 2024-09-19T05:43:25.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 msgr2=0x7ff07c1f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:25.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff07c1f18b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0016f0 tx=0x7ff06c001af0 comp rx=0 tx=0).stop 2024-09-19T05:43:25.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 shutdown_connections 2024-09-19T05:43:25.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff05c06c2e0 0x7ff05c06e7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:25.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff07c1e0fd0 0x7ff07c14e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:25.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff07c0c99a0 0x7ff07c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:25.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 --2- 172.21.15.2:0/3175708127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0680037a0 0x7ff07c1f18b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:25.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.563+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 >> 172.21.15.2:0/3175708127 conn(0x7ff07c06d0e0 msgr2=0x7ff07c152440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:25.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.564+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 shutdown_connections 2024-09-19T05:43:25.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:25.564+0000 7ff0822b7640 1 -- 172.21.15.2:0/3175708127 wait complete. 2024-09-19T05:43:25.640 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:25.639+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:25.657 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:25.657 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:25.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:25.657 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:25.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.657 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:25.657 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.680371399999999, 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062186530000000002, 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974705+0000", 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1270000000000002e-06 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974716+0000", 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1306e-05 2024-09-19T05:43:25.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974736+0000", 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9293e-05 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974749+0000", 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3059e-05 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:25.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022932900000000001 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975163+0000", 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018497500000000001 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975314+0000", 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151328 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:25.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979256+0000", 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039412179999999998 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571296 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980877+0000", 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0387999999999999e-05 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980919+0000", 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1334000000000001e-05 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.680248591999998, 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062599709999999996, 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.663 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974824+0000", 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2500000000000004e-07 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974829+0000", 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.66e-06 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974844+0000", 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5160000000000001e-05 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974852+0000", 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9570000000000002e-06 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975748+0000", 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089547100000000001 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975835+0000", 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7190999999999999e-05 2024-09-19T05:43:25.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975997+0000", 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016230800000000001 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979261+0000", 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032635310000000001 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980973+0000", 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017117199999999999 2024-09-19T05:43:25.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981027+0000", 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4732999999999999e-05 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981083+0000", 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5414999999999997e-05 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:30 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.680188061000003, 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040710980000000004, 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:25.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974884+0000", 2024-09-19T05:43:25.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6400000000000001e-07 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974899+0000", 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0584000000000001e-05 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974917+0000", 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9799999999999998e-06 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974994+0000", 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6612000000000004e-05 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975176+0000", 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018201400000000001 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975321+0000", 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014440899999999999 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977845+0000", 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002524409 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978841+0000", 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099560399999999998 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978896+0000", 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5075999999999999e-05 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978954+0000", 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8743999999999999e-05 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.675 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.680120115999998, 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040869110000000004, 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974952+0000", 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974957+0000", 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7260000000000002e-06 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974966+0000", 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974972+0000", 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0970000000000002e-06 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975780+0000", 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080748199999999999 2024-09-19T05:43:25.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975894+0000", 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113867 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976037+0000", 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014333 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978068+0000", 2024-09-19T05:43:25.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020306439999999999 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978965+0000", 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089710699999999996 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979003+0000", 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8797999999999997e-05 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.680 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979038+0000", 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4668000000000001e-05 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.680088398000002, 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062404959999999999, 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.681 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3900000000000005e-07 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974999+0000", 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5817000000000001e-05 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975006+0000", 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5509999999999996e-06 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975012+0000", 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8189999999999997e-06 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976240+0000", 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012283870000000001 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976367+0000", 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126465 2024-09-19T05:43:25.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976487+0000", 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120548 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980396+0000", 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003908702 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981127+0000", 2024-09-19T05:43:25.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073081199999999995 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981182+0000", 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5921999999999998e-05 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981223+0000", 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0933999999999998e-05 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.686 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1e epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:7bf7e7b5:::200.00000006:head by mds.0.4:37 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.678962943999998, 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028842130000000001, 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.687 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976110+0000", 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2869999999999999e-06 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976123+0000", 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3224000000000001e-05 2024-09-19T05:43:25.688 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976140+0000", 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.755e-05 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976157+0000", 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6464e-05 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.689 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976275+0000", 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117602 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978993+0000", 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027180860000000002 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:25.690 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.677756066000001, 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001691115, 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:25.691 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977316+0000", 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209999999999999e-06 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977337+0000", 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0890000000000002e-05 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.692 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977356+0000", 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032999999999999e-05 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977365+0000", 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7649999999999999e-06 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977425+0000", 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0044999999999999e-05 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.693 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977449+0000", 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4006999999999999e-05 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978959+0000", 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015099379999999999 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979006+0000", 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7315999999999999e-05 2024-09-19T05:43:25.694 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:30 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.677335958999997, 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118336, 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.695 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977736+0000", 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.096e-06 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0979999999999999e-06 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977756+0000", 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3891e-05 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977765+0000", 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7579999999999998e-06 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977830+0000", 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5184999999999993e-05 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977839+0000", 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4770000000000003e-06 2024-09-19T05:43:25.698 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977854+0000", 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4831000000000001e-05 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:27 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.677098407999999, 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101439, 2024-09-19T05:43:25.699 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.700 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977974+0000", 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6899999999999996e-07 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977979+0000", 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1399999999999999e-06 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977993+0000", 2024-09-19T05:43:25.701 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4215e-05 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978001+0000", 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978055+0000", 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3711000000000001e-05 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.702 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978062+0000", 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978074+0000", 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2238999999999999e-05 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:25.703 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.675854358999999, 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024064609999999999, 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.704 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979218+0000", 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979227+0000", 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.705 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979239+0000", 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979247+0000", 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8369999999999997e-06 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979307+0000", 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0438e-05 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.706 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979326+0000", 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8938999999999999e-05 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981594+0000", 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022679710000000001 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981623+0000", 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9329999999999999e-05 2024-09-19T05:43:25.707 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.675553678999997, 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021274990000000001, 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.708 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7800000000000001e-07 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.709 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979524+0000", 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8019999999999997e-06 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979534+0000", 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8859999999999999e-06 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8189999999999992e-06 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.710 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979610+0000", 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8433e-05 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979641+0000", 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1233999999999999e-05 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981625+0000", 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001983964 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.711 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981645+0000", 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9783e-05 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.675492706999997, 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020817100000000001, 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.712 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.713 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979598+0000", 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9072e-05 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979609+0000", 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0360000000000001e-05 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.714 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979615+0000", 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8700000000000003e-06 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979866+0000", 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000250454 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979884+0000", 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7725000000000001e-05 2024-09-19T05:43:25.715 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981647+0000", 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017632629999999999 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981660+0000", 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3502999999999999e-05 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.716 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.675006287999999, 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016104190000000001, 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.717 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980066+0000", 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8400000000000005e-07 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980070+0000", 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8010000000000003e-06 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.718 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980077+0000", 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.08e-06 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980082+0000", 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8450000000000002e-06 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980146+0000", 2024-09-19T05:43:25.719 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3589000000000004e-05 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980166+0000", 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0514e-05 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981661+0000", 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014949539999999999 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.720 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981675+0000", 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.674956074000001, 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018022209999999999, 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.721 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.722 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980116+0000", 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980126+0000", 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980143+0000", 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6407999999999999e-05 2024-09-19T05:43:25.723 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980150+0000", 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980385+0000", 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023545400000000001 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.724 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980403+0000", 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7652999999999999e-05 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981869+0000", 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014661679999999999 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981917+0000", 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319000000000003e-05 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.725 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.674357122000004, 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000222196, 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.726 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980715+0000", 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.727 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980721+0000", 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2620000000000004e-06 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980735+0000", 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3756e-05 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980759+0000", 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.728 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980813+0000", 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4249999999999997e-05 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980821+0000", 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7540000000000006e-06 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980936+0000", 2024-09-19T05:43:25.729 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115574 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.67428906, 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030787300000000001, 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.730 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980783+0000", 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.731 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980786+0000", 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5279999999999999e-06 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980795+0000", 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0790000000000007e-06 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980802+0000", 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4910000000000002e-06 2024-09-19T05:43:25.732 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980962+0000", 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015999100000000001 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980968+0000", 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.874e-06 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.733 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981090+0000", 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012244099999999999 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:29 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.674259153999998, 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041785999999999999, 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.734 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1600000000000002e-07 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.735 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980816+0000", 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6660000000000001e-06 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980823+0000", 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6789999999999999e-06 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6480000000000002e-06 2024-09-19T05:43:25.736 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981115+0000", 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028752699999999999 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.737 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981230+0000", 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108546 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.674236522999998, 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011264039999999999, 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.738 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:25.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980840+0000", 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1540000000000001e-06 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980848+0000", 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980853+0000", 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9030000000000003e-06 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980907+0000", 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4224999999999997e-05 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980937+0000", 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9852000000000001e-05 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981931+0000", 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099441099999999995 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981961+0000", 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0144000000000001e-05 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-19T05:43:25.742 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 47.673718280000003, 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013984010000000001, 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:25.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981354+0000", 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0160000000000001e-06 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981361+0000", 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981373+0000", 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1439e-05 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981385+0000", 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2041999999999999e-05 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981446+0000", 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1446999999999996e-05 2024-09-19T05:43:25.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981469+0000", 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982708+0000", 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012396359999999999 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982751+0000", 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3198000000000002e-05 2024-09-19T05:43:25.746 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.747 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.747 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:25.747 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:25.747 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:25.748 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:25.749 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:25.746+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:25.849 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:25.847+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:25.920 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:25.920 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:25.920 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:25.920 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:25.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:25.950 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:25.949+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:26.051 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:26.049+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:26.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:26.090 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:26.091 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:26.100 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:26.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.180+0000 7f14e39fd640 1 Processor -- start 2024-09-19T05:43:26.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.180+0000 7f14e39fd640 1 -- start start 2024-09-19T05:43:26.182 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:26.180+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e39fd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14dc154b30 0x7f14dc06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e39fd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc06dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e39fd640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14dc0c95f0 con 0x7f14dc0cbc00 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e39fd640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f14dc0ca220 con 0x7f14dc154b30 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e39fd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc0ca500 con 0x7f14dc0c9bc0 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e1f9a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc06dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e1799640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14dc154b30 0x7f14dc06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e1799640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14dc154b30 0x7f14dc06d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42414/0 (socket says 172.21.15.2:42414) 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e1f9a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc06dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:36534/0 (socket says 172.21.15.2:36534) 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.181+0000 7f14e1f9a640 1 -- 172.21.15.2:0/3252416970 learned_addr learned my addr 172.21.15.2:0/3252416970 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:26.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.182+0000 7f14e1f9a640 1 -- 172.21.15.2:0/3252416970 >> v2:172.21.15.123:3300/0 conn(0x7f14dc154b30 msgr2=0x7f14dc06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:26.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.182+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/3252416970 >> v2:172.21.15.123:3300/0 conn(0x7f14dc154b30 0x7f14dc06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.182+0000 7f14e1f9a640 1 -- 172.21.15.2:0/3252416970 >> v1:172.21.15.123:6789/0 conn(0x7f14dc0cbc00 legacy=0x7f14dc15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.182+0000 7f14e1f9a640 1 -- 172.21.15.2:0/3252416970 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc0ca7e0 con 0x7f14dc0c9bc0 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.183+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/3252416970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc06dc20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009e50 tx=0x7f14d402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c664c9bb6c609eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.183+0000 7f14d3fff640 1 -- 172.21.15.2:0/3252416970 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4036070 con 0x7f14dc0c9bc0 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.183+0000 7f14d3fff640 1 -- 172.21.15.2:0/3252416970 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d4031600 con 0x7f14dc0c9bc0 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.183+0000 7f14d3fff640 1 -- 172.21.15.2:0/3252416970 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40319d0 con 0x7f14dc0c9bc0 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.183+0000 7f14e39fd640 1 -- 172.21.15.2:0/3252416970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 msgr2=0x7f14dc06dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.183+0000 7f14e39fd640 1 --2- 172.21.15.2:0/3252416970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc06dc20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009e50 tx=0x7f14d402a680 comp rx=0 tx=0).stop 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.184+0000 7f14e39fd640 1 -- 172.21.15.2:0/3252416970 shutdown_connections 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.184+0000 7f14e39fd640 1 --2- 172.21.15.2:0/3252416970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc06dc20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.184+0000 7f14e39fd640 1 --2- 172.21.15.2:0/3252416970 >> v2:172.21.15.123:3300/0 conn(0x7f14dc154b30 0x7f14dc06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.184+0000 7f14e39fd640 1 -- 172.21.15.2:0/3252416970 >> 172.21.15.2:0/3252416970 conn(0x7f14dc0707f0 msgr2=0x7f14dc070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:26.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.184+0000 7f14e39fd640 1 -- 172.21.15.2:0/3252416970 shutdown_connections 2024-09-19T05:43:26.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.184+0000 7f14e39fd640 1 -- 172.21.15.2:0/3252416970 wait complete. 2024-09-19T05:43:26.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.185+0000 7f14e39fd640 1 Processor -- start 2024-09-19T05:43:26.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.185+0000 7f14e39fd640 1 -- start start 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e39fd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc0cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e39fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0cbc00 0x7f14dc0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e39fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc154b30 0x7f14dc153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e39fd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0c9380 con 0x7f14dc0cbc00 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e39fd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0c9200 con 0x7f14dc154b30 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e39fd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc15ea50 con 0x7f14dc0c9bc0 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0cbc00 0x7f14dc0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0cbc00 0x7f14dc0aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59198/0 (socket says 172.21.15.2:59198) 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 -- 172.21.15.2:0/39155233 learned_addr learned my addr 172.21.15.2:0/39155233 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc154b30 0x7f14dc153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc0cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 msgr2=0x7f14dc0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14dc0c9bc0 0x7f14dc0cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc154b30 msgr2=0x7f14dc153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc154b30 0x7f14dc153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.186+0000 7f14e0f98640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc1583c0 con 0x7f14dc0cbc00 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.187+0000 7f14e0f98640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0cbc00 0x7f14dc0aebe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c430 tx=0x7f14cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.187+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc0052d0 con 0x7f14dc0cbc00 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.187+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14cc0054b0 con 0x7f14dc0cbc00 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.187+0000 7f14e39fd640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc1f43a0 con 0x7f14dc0cbc00 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.187+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc006a60 con 0x7f14dc0cbc00 2024-09-19T05:43:26.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.187+0000 7f14e39fd640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1580e0 con 0x7f14dc0cbc00 2024-09-19T05:43:26.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.188+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f14b4005270 con 0x7f14dc0cbc00 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.189+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f14cc006c00 con 0x7f14dc0cbc00 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.189+0000 7f14d1ffb640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14c006c300 0x7f14c006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.190+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14c006c300 0x7f14c006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.190+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f14cc091aa0 con 0x7f14dc0cbc00 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.190+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v43) ==== 54+0+16 (secure 0 0 0) 0x7f14cc00e280 con 0x7f14dc0cbc00 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.190+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14c006c300 0x7f14c006e7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009ad0 tx=0x7f14d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.190+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f14b4000fa0 0x7f14b40045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.190+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f14b4005550 con 0x7f14b4000fa0 2024-09-19T05:43:26.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.191+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f14b4000fa0 0x7f14b40045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.191+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f14b4000fa0 0x7f14b40045e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.192+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b4005550 con 0x7f14b4000fa0 2024-09-19T05:43:26.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.206+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b4004f70 con 0x7f14b4000fa0 2024-09-19T05:43:26.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.208+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b4004f70 con 0x7f14b4000fa0 2024-09-19T05:43:26.210 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:26.211 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:26.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.208+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f14b4001cb0 0x7f14b4002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.208+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f14b4002210 con 0x7f14b4001cb0 2024-09-19T05:43:26.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.208+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f14b4001cb0 0x7f14b4002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.209+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f14b4001cb0 0x7f14b4002990 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.210+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b4002210 con 0x7f14b4001cb0 2024-09-19T05:43:26.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.239+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b40031d0 con 0x7f14b4001cb0 2024-09-19T05:43:26.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.241+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b40031d0 con 0x7f14b4001cb0 2024-09-19T05:43:26.243 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:26.244 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:26.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.242+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f14b4003c50 0x7f14b400bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.242+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f14b400ec80 con 0x7f14b4003c50 2024-09-19T05:43:26.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.242+0000 7f14e0f98640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f14b4003c50 0x7f14b400bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.242+0000 7f14e0f98640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f14b4003c50 0x7f14b400bc90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.243+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b400ec80 con 0x7f14b4003c50 2024-09-19T05:43:26.252 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:26.251+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.272+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b400dcd0 con 0x7f14b4003c50 2024-09-19T05:43:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.274+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b400dcd0 con 0x7f14b4003c50 2024-09-19T05:43:26.276 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.275+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f14b400e5e0 0x7f14b400c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.275+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f14b4012690 con 0x7f14b400e5e0 2024-09-19T05:43:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.275+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f14b400e5e0 0x7f14b400c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.275+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f14b400e5e0 0x7f14b400c2c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.276+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b4012690 con 0x7f14b400e5e0 2024-09-19T05:43:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.303+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b400cb20 con 0x7f14b400e5e0 2024-09-19T05:43:26.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.306+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b400cb20 con 0x7f14b400e5e0 2024-09-19T05:43:26.308 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.306+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f14b400d270 0x7f14b4010e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.306+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f14b400d6f0 con 0x7f14b400d270 2024-09-19T05:43:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.307+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f14b400d270 0x7f14b4010e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.307+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f14b400d270 0x7f14b4010e40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.308+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b400d6f0 con 0x7f14b400d270 2024-09-19T05:43:26.313 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:26.313 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:26.325 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:26.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.334+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b400d6f0 con 0x7f14b400d270 2024-09-19T05:43:26.339 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.336+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b400d6f0 con 0x7f14b400d270 2024-09-19T05:43:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.337+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f14b4011890 0x7f14b4011d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.337+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f14b4019180 con 0x7f14b4011890 2024-09-19T05:43:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.337+0000 7f14e0f98640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f14b4011890 0x7f14b4011d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.338+0000 7f14e0f98640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f14b4011890 0x7f14b4011d20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.338+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b4019180 con 0x7f14b4011890 2024-09-19T05:43:26.353 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:26.352+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:26.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.366+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b40168f0 con 0x7f14b4011890 2024-09-19T05:43:26.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.368+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b40168f0 con 0x7f14b4011890 2024-09-19T05:43:26.370 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:26.371 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:26.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.368+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f14b4016e00 0x7f14b40191f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.369+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f14b401c240 con 0x7f14b4016e00 2024-09-19T05:43:26.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.369+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f14b4016e00 0x7f14b40191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.370+0000 7f14e1f9a640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f14b4016e00 0x7f14b40191f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.370+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b401c240 con 0x7f14b4016e00 2024-09-19T05:43:26.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.384+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b401af60 con 0x7f14b4016e00 2024-09-19T05:43:26.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.386+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b401af60 con 0x7f14b4016e00 2024-09-19T05:43:26.389 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.387+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f14b401b680 0x7f14b401bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.387+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f14b401f610 con 0x7f14b401b680 2024-09-19T05:43:26.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.387+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f14b401b680 0x7f14b401bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:26.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.388+0000 7f14e1799640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f14b401b680 0x7f14b401bac0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:26.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.388+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f14b401f610 con 0x7f14b401b680 2024-09-19T05:43:26.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.402+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f14b401e490 con 0x7f14b401b680 2024-09-19T05:43:26.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.404+0000 7f14d1ffb640 1 -- 172.21.15.2:0/39155233 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f14b401e490 con 0x7f14b401b680 2024-09-19T05:43:26.406 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:26.407 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:26.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.404+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f14b4000fa0 msgr2=0x7f14b40045e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.404+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f14b4000fa0 0x7f14b40045e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.404+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f14b4001cb0 msgr2=0x7f14b4002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.404+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f14b4001cb0 0x7f14b4002990 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f14b4003c50 msgr2=0x7f14b400bc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f14b4003c50 0x7f14b400bc90 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f14b400e5e0 msgr2=0x7f14b400c2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f14b400e5e0 0x7f14b400c2c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f14b400d270 msgr2=0x7f14b4010e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f14b400d270 0x7f14b4010e40 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f14b4011890 msgr2=0x7f14b4011d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f14b4011890 0x7f14b4011d20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f14b4016e00 msgr2=0x7f14b40191f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f14b4016e00 0x7f14b40191f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f14b401b680 msgr2=0x7f14b401bac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f14b401b680 0x7f14b401bac0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14e1f9a640 1 -- 172.21.15.2:0/39155233 reap_dead start 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14c006c300 msgr2=0x7f14c006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14c006c300 0x7f14c006e7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009ad0 tx=0x7f14d8008040 comp rx=0 tx=0).stop 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14e1f9a640 1 -- 172.21.15.2:0/39155233 reap_dead start 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0cbc00 msgr2=0x7f14dc0aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 --2- 172.21.15.2:0/39155233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14dc0cbc00 0x7f14dc0aebe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c430 tx=0x7f14cc00c8f0 comp rx=0 tx=0).stop 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14e1f9a640 1 -- 172.21.15.2:0/39155233 reap_dead start 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14e1f9a640 1 -- 172.21.15.2:0/39155233 reap_dead start 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14e1f9a640 1 -- 172.21.15.2:0/39155233 reap_dead start 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 shutdown_connections 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 >> 172.21.15.2:0/39155233 conn(0x7f14dc0707f0 msgr2=0x7f14dc071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 shutdown_connections 2024-09-19T05:43:26.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:26.405+0000 7f14c77fe640 1 -- 172.21.15.2:0/39155233 wait complete. 2024-09-19T05:43:26.460 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:26.460 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:26.460 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:26.458+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:26.469 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:26.555 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:26.553+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:26.620 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:26.620 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:26.629 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:26.674 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:26.672+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:26.738 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:26.738 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:26.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:26.757 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:26.755+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:26.858 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:26.856+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:26.898 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:26.898 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:26.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:26.959 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:26.958+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:27.019 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:27.019 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:27.028 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:27.072 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:27.070+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:27.136 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:27.136 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:27.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:27.162 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:27.160+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:27.263 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:27.261+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:27.294 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:27.295 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:27.304 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:27.389 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:27.387+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:27.454 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:27.454 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:27.463 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:27.504 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:27.502+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:27.565 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:27.563+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:27.582 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:43:27.631 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:27.631 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:27.640 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:27.685 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:27.683+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.732+0000 7f14f4f0c640 1 Processor -- start 2024-09-19T05:43:27.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.732+0000 7f14f4f0c640 1 -- start start 2024-09-19T05:43:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14f4f0c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f0154c20 0x7f14f00cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14f4f0c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f14f0155000 0x7f14f0073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:27.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14f4f0c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14f00caf40 0x7f14f0073880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14f4f0c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f14f00ccbc0 con 0x7f14f0155000 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14f4f0c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f14f00cc600 con 0x7f14f00caf40 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14f4f0c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f14f00cc8e0 con 0x7f14f0154c20 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14edd9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f14f0155000 0x7f14f0073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14edd9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f14f0155000 0x7f14f0073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59212/0 (socket says 172.21.15.2:59212) 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14edd9b640 1 -- 172.21.15.2:0/1389714220 learned_addr learned my addr 172.21.15.2:0/1389714220 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14ee59c640 1 --2- 172.21.15.2:0/1389714220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f0154c20 0x7f14f00cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.734+0000 7f14eed9d640 1 --2- 172.21.15.2:0/1389714220 >> v2:172.21.15.123:3300/0 conn(0x7f14f00caf40 0x7f14f0073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.735+0000 7f14edd9b640 1 -- 172.21.15.2:0/1389714220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f0154c20 msgr2=0x7f14f00cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.735+0000 7f14edd9b640 1 --2- 172.21.15.2:0/1389714220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f0154c20 0x7f14f00cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.735+0000 7f14edd9b640 1 -- 172.21.15.2:0/1389714220 >> v2:172.21.15.123:3300/0 conn(0x7f14f00caf40 msgr2=0x7f14f0073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.735+0000 7f14edd9b640 1 --2- 172.21.15.2:0/1389714220 >> v2:172.21.15.123:3300/0 conn(0x7f14f00caf40 0x7f14f0073880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.735+0000 7f14edd9b640 1 -- 172.21.15.2:0/1389714220 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f00cbc60 con 0x7f14f0155000 2024-09-19T05:43:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.735+0000 7f14ee59c640 1 --2- 172.21.15.2:0/1389714220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f0154c20 0x7f14f00cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.735+0000 7f14eed9d640 1 --2- 172.21.15.2:0/1389714220 >> v2:172.21.15.123:3300/0 conn(0x7f14f00caf40 0x7f14f0073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.736+0000 7f14edd9b640 1 --2- 172.21.15.2:0/1389714220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0155000 0x7f14f0073340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14f00b61f0 tx=0x7f14e402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c0304090403d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.736+0000 7f14ed59a640 1 -- 172.21.15.2:0/1389714220 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4004030 con 0x7f14f0155000 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.736+0000 7f14ed59a640 1 -- 172.21.15.2:0/1389714220 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e40041d0 con 0x7f14f0155000 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.736+0000 7f14ed59a640 1 -- 172.21.15.2:0/1389714220 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4003610 con 0x7f14f0155000 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 -- 172.21.15.2:0/1389714220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0155000 msgr2=0x7f14f0073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/1389714220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0155000 0x7f14f0073340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14f00b61f0 tx=0x7f14e402a1c0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 -- 172.21.15.2:0/1389714220 shutdown_connections 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/1389714220 >> v2:172.21.15.123:3300/0 conn(0x7f14f00caf40 0x7f14f0073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/1389714220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0155000 0x7f14f0073340 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/1389714220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f0154c20 0x7f14f00cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 -- 172.21.15.2:0/1389714220 >> 172.21.15.2:0/1389714220 conn(0x7f14f006d0e0 msgr2=0x7f14f006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 -- 172.21.15.2:0/1389714220 shutdown_connections 2024-09-19T05:43:27.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.737+0000 7f14f4f0c640 1 -- 172.21.15.2:0/1389714220 wait complete. 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.738+0000 7f14f4f0c640 1 Processor -- start 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14f4f0c640 1 -- start start 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14f4f0c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f00caf40 0x7f14f00b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14f4f0c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0154c20 0x7f14f0155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14f4f0c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f0155000 0x7f14f0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14f4f0c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00c5770 con 0x7f14f0154c20 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14f4f0c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00cb430 con 0x7f14f0155000 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14f4f0c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f14f00c58f0 con 0x7f14f00caf40 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14edd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0154c20 0x7f14f0155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14edd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0154c20 0x7f14f0155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59224/0 (socket says 172.21.15.2:59224) 2024-09-19T05:43:27.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.739+0000 7f14edd9b640 1 -- 172.21.15.2:0/872020249 learned_addr learned my addr 172.21.15.2:0/872020249 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14ee59c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f00caf40 0x7f14f00b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14eed9d640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f0155000 0x7f14f0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14edd9b640 1 -- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f00caf40 msgr2=0x7f14f00b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14edd9b640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f00caf40 0x7f14f00b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14edd9b640 1 -- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f0155000 msgr2=0x7f14f0156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14edd9b640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f0155000 0x7f14f0156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14edd9b640 1 -- 172.21.15.2:0/872020249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f01e1d50 con 0x7f14f0154c20 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14ee59c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f00caf40 0x7f14f00b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14edd9b640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0154c20 0x7f14f0155e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f14e4000c00 tx=0x7f14e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14deffd640 1 -- 172.21.15.2:0/872020249 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4041020 con 0x7f14f0154c20 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14deffd640 1 -- 172.21.15.2:0/872020249 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e4005e10 con 0x7f14f0154c20 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.740+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f01e1ff0 con 0x7f14f0154c20 2024-09-19T05:43:27.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.741+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f01e25b0 con 0x7f14f0154c20 2024-09-19T05:43:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.742+0000 7f14deffd640 1 -- 172.21.15.2:0/872020249 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4004030 con 0x7f14f0154c20 2024-09-19T05:43:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.742+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d4005290 con 0x7f14f0154c20 2024-09-19T05:43:27.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.743+0000 7f14deffd640 1 -- 172.21.15.2:0/872020249 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f14e4044020 con 0x7f14f0154c20 2024-09-19T05:43:27.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.743+0000 7f14deffd640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14d006c2b0 0x7f14d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:27.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.744+0000 7f14deffd640 1 -- 172.21.15.2:0/872020249 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f14e40abd90 con 0x7f14f0154c20 2024-09-19T05:43:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.744+0000 7f14ee59c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14d006c2b0 0x7f14d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:27.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.744+0000 7f14ee59c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14d006c2b0 0x7f14d006e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009560 tx=0x7f14e0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:27.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.752+0000 7f14deffd640 1 -- 172.21.15.2:0/872020249 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f14e404b030 con 0x7f14f0154c20 2024-09-19T05:43:27.764 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:27.764 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:27.766 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:27.765+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:27.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:27.867 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:27.867+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:27.920 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:27.921 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:27.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:27.940+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f14d4005580 con 0x7f14f0154c20 2024-09-19T05:43:27.973 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:27.972+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:28.033 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:28.033 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:28.042 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:28.083 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:28.081+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:28.172 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:28.171+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:28.185 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:28.185 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:28.194 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:28.273 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:28.271+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:28.296 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:28.296 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:28.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:28.389 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:28.387+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:28.423 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:28.448 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:28.448 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:28.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:28.475 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:28.474+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.526+0000 7f9713513640 1 Processor -- start 2024-09-19T05:43:28.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.526+0000 7f9713513640 1 -- start start 2024-09-19T05:43:28.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.527+0000 7f9713513640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f970c154b30 0x7f970c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.527+0000 7f9713513640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970c0c95f0 con 0x7f970c0c9bc0 2024-09-19T05:43:28.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.527+0000 7f9713513640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970c0ca220 con 0x7f970c0cbc00 2024-09-19T05:43:28.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.527+0000 7f9713513640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f970c0ca500 con 0x7f970c154b30 2024-09-19T05:43:28.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.527+0000 7f9711ab0640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f970c0c9bc0 0x7f970c165360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54758/0 (socket says 172.21.15.2:54758) 2024-09-19T05:43:28.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.527+0000 7f9711ab0640 1 -- 172.21.15.2:0/2398249091 learned_addr learned my addr 172.21.15.2:0/2398249091 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:28.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.527+0000 7f97112af640 1 --2- 172.21.15.2:0/2398249091 >> v2:172.21.15.123:3300/0 conn(0x7f970c154b30 0x7f970c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.528+0000 7f9703fff640 1 -- 172.21.15.2:0/2398249091 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806478716 0 0) 0x7f970c0c95f0 con 0x7f970c0c9bc0 2024-09-19T05:43:28.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.528+0000 7f97112af640 1 -- 172.21.15.2:0/2398249091 >> v1:172.21.15.123:6789/0 conn(0x7f970c0cbc00 legacy=0x7f970c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.528+0000 7f97112af640 1 -- 172.21.15.2:0/2398249091 >> v1:172.21.15.2:6789/0 conn(0x7f970c0c9bc0 legacy=0x7f970c165360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.528+0000 7f97112af640 1 -- 172.21.15.2:0/2398249091 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c0ca7e0 con 0x7f970c154b30 2024-09-19T05:43:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.529+0000 7f97112af640 1 --2- 172.21.15.2:0/2398249091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c06d6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9708009720 tx=0x7f970802a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef26dcba044995f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.529+0000 7f9703fff640 1 -- 172.21.15.2:0/2398249091 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708037070 con 0x7f970c154b30 2024-09-19T05:43:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.529+0000 7f9703fff640 1 -- 172.21.15.2:0/2398249091 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970802ad40 con 0x7f970c154b30 2024-09-19T05:43:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.529+0000 7f9703fff640 1 -- 172.21.15.2:0/2398249091 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708032660 con 0x7f970c154b30 2024-09-19T05:43:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.530+0000 7f9713513640 1 -- 172.21.15.2:0/2398249091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 msgr2=0x7f970c06d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.530+0000 7f9713513640 1 --2- 172.21.15.2:0/2398249091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c06d6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9708009720 tx=0x7f970802a030 comp rx=0 tx=0).stop 2024-09-19T05:43:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.530+0000 7f9713513640 1 -- 172.21.15.2:0/2398249091 shutdown_connections 2024-09-19T05:43:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.530+0000 7f9713513640 1 --2- 172.21.15.2:0/2398249091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c06d6e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.530+0000 7f9713513640 1 -- 172.21.15.2:0/2398249091 >> 172.21.15.2:0/2398249091 conn(0x7f970c0707f0 msgr2=0x7f970c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.530+0000 7f9713513640 1 -- 172.21.15.2:0/2398249091 shutdown_connections 2024-09-19T05:43:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.530+0000 7f9713513640 1 -- 172.21.15.2:0/2398249091 wait complete. 2024-09-19T05:43:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.532+0000 7f9713513640 1 Processor -- start 2024-09-19T05:43:28.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.532+0000 7f9713513640 1 -- start start 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9713513640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f970c0c9bc0 0x7f970c1f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9713513640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0cbc00 0x7f970c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9713513640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c0c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9710aae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0cbc00 0x7f970c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9710aae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0cbc00 0x7f970c0c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59230/0 (socket says 172.21.15.2:59230) 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f97112af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f970c0c9bc0 0x7f970c1f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9710aae640 1 -- 172.21.15.2:0/1139296025 learned_addr learned my addr 172.21.15.2:0/1139296025 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9713513640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0c9380 con 0x7f970c0cbc00 2024-09-19T05:43:28.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9713513640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0c9200 con 0x7f970c154b30 2024-09-19T05:43:28.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9713513640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f970c15ea50 con 0x7f970c0c9bc0 2024-09-19T05:43:28.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.533+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c0c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9711ab0640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f970c0c9bc0 msgr2=0x7f970c1f1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f970c0c9bc0 0x7f970c1f1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9711ab0640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0cbc00 msgr2=0x7f970c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0cbc00 0x7f970c0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9711ab0640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c1582d0 con 0x7f970c154b30 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f970c0c9bc0 0x7f970c1f1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c0c5b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f970400c750 tx=0x7f970400cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704018070 con 0x7f970c154b30 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9704004d60 con 0x7f970c154b30 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9713513640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c1f0b30 con 0x7f970c154b30 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970400a880 con 0x7f970c154b30 2024-09-19T05:43:28.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.534+0000 7f9713513640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c0c6a10 con 0x7f970c154b30 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.536+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f96e4005270 con 0x7f970c154b30 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.537+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f970401b070 con 0x7f970c154b30 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.537+0000 7f9701ffb640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96f006c2e0 0x7f96f006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.538+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f970408aa30 con 0x7f970c154b30 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.538+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96f006c2e0 0x7f96f006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.538+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v43) ==== 54+0+16 (secure 0 0 0) 0x7f970408b0a0 con 0x7f970c154b30 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.538+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96e4000fa0 0x7f96e40045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.538+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96e4005550 con 0x7f96e4000fa0 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.538+0000 7f9710aae640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96e4000fa0 0x7f96e40045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.539+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96f006c2e0 0x7f96f006e7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9708034000 tx=0x7f970802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.539+0000 7f9710aae640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96e4000fa0 0x7f96e40045e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.540+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e4005550 con 0x7f96e4000fa0 2024-09-19T05:43:28.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.568+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e4004f70 con 0x7f96e4000fa0 2024-09-19T05:43:28.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.570+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e4004f70 con 0x7f96e4000fa0 2024-09-19T05:43:28.571 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:28.572 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:28.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.570+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96e4001cb0 0x7f96e4002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.570+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f96e4002290 con 0x7f96e4001cb0 2024-09-19T05:43:28.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.570+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96e4001cb0 0x7f96e4002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.571+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96e4001cb0 0x7f96e4002990 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.572+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e4002290 con 0x7f96e4001cb0 2024-09-19T05:43:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.573+0000 7f14deffd640 1 -- 172.21.15.2:0/872020249 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v44) ==== 97+0+0 (secure 0 0 0) 0x7f14e407a460 con 0x7f14f0154c20 2024-09-19T05:43:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:43:28.576 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:28.574+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:28.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14d006c2b0 msgr2=0x7f14d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14d006c2b0 0x7f14d006e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009560 tx=0x7f14e0009190 comp rx=0 tx=0).stop 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0154c20 msgr2=0x7f14f0155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0154c20 0x7f14f0155e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f14e4000c00 tx=0x7f14e40046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 shutdown_connections 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f14d006c2b0 0x7f14d006e770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14f0155000 0x7f14f0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14f0154c20 0x7f14f0155e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 --2- 172.21.15.2:0/872020249 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14f00caf40 0x7f14f00b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 >> 172.21.15.2:0/872020249 conn(0x7f14f006d0e0 msgr2=0x7f14f0073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:28.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 shutdown_connections 2024-09-19T05:43:28.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.577+0000 7f14f4f0c640 1 -- 172.21.15.2:0/872020249 wait complete. 2024-09-19T05:43:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.597+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e40031d0 con 0x7f96e4001cb0 2024-09-19T05:43:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.600+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e40031d0 con 0x7f96e4001cb0 2024-09-19T05:43:28.601 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.600+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96e4003c90 0x7f96e400bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.600+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f96e400ecf0 con 0x7f96e4003c90 2024-09-19T05:43:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.600+0000 7f9710aae640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96e4003c90 0x7f96e400bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.601+0000 7f9710aae640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96e4003c90 0x7f96e400bc90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.603 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:28.603 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.601+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e400ecf0 con 0x7f96e4003c90 2024-09-19T05:43:28.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.615+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e400dd30 con 0x7f96e4003c90 2024-09-19T05:43:28.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.618+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e400dd30 con 0x7f96e4003c90 2024-09-19T05:43:28.619 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.618+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96e400e7b0 0x7f96e400c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.618+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f96e400cab0 con 0x7f96e400e7b0 2024-09-19T05:43:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.618+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96e400e7b0 0x7f96e400c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.619+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96e400e7b0 0x7f96e400c2f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.620+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e400cab0 con 0x7f96e400e7b0 2024-09-19T05:43:28.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.647+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e400cff0 con 0x7f96e400e7b0 2024-09-19T05:43:28.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.650+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e400cff0 con 0x7f96e400e7b0 2024-09-19T05:43:28.651 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:28.652 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:28.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.650+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96e400d530 0x7f96e4012ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.650+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f96e4015ef0 con 0x7f96e400d530 2024-09-19T05:43:28.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.650+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96e400d530 0x7f96e4012ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.651+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96e400d530 0x7f96e4012ec0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.652+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e4015ef0 con 0x7f96e400d530 2024-09-19T05:43:28.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.670+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e4014c30 con 0x7f96e400d530 2024-09-19T05:43:28.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.672+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e4014c30 con 0x7f96e400d530 2024-09-19T05:43:28.673 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:28.673 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:28.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.672+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96e4015380 0x7f96e40157e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.672+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f96e40192c0 con 0x7f96e4015380 2024-09-19T05:43:28.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.672+0000 7f9710aae640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96e4015380 0x7f96e40157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.673+0000 7f9710aae640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96e4015380 0x7f96e40157e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.673+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e40192c0 con 0x7f96e4015380 2024-09-19T05:43:28.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.687+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e40181c0 con 0x7f96e4015380 2024-09-19T05:43:28.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.689+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e40181c0 con 0x7f96e4015380 2024-09-19T05:43:28.690 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:28.691 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:28.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.689+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96e4018910 0x7f96e4018d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.689+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f96e401c690 con 0x7f96e4018910 2024-09-19T05:43:28.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.689+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96e4018910 0x7f96e4018d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.690+0000 7f9711ab0640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96e4018910 0x7f96e4018d90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.691 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.690+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e401c690 con 0x7f96e4018910 2024-09-19T05:43:28.695 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:28.694+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:28.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.704+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e401b510 con 0x7f96e4018910 2024-09-19T05:43:28.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.706+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e401b510 con 0x7f96e4018910 2024-09-19T05:43:28.707 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:28.708 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:28.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.706+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96e401bc60 0x7f96e401c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:28.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.706+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f96e401fa00 con 0x7f96e401bc60 2024-09-19T05:43:28.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.706+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96e401bc60 0x7f96e401c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:28.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.707+0000 7f97112af640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96e401bc60 0x7f96e401c0e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:28.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.707+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96e401fa00 con 0x7f96e401bc60 2024-09-19T05:43:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.721+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f96e40134f0 con 0x7f96e401bc60 2024-09-19T05:43:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f9701ffb640 1 -- 172.21.15.2:0/1139296025 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96e40134f0 con 0x7f96e401bc60 2024-09-19T05:43:28.724 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96e4000fa0 msgr2=0x7f96e40045e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96e4000fa0 0x7f96e40045e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96e4001cb0 msgr2=0x7f96e4002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96e4001cb0 0x7f96e4002990 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96e4003c90 msgr2=0x7f96e400bc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96e4003c90 0x7f96e400bc90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f9711ab0640 1 -- 172.21.15.2:0/1139296025 reap_dead start 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96e400e7b0 msgr2=0x7f96e400c2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.723+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96e400e7b0 0x7f96e400c2f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96e400d530 msgr2=0x7f96e4012ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96e400d530 0x7f96e4012ec0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96e4015380 msgr2=0x7f96e40157e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96e4015380 0x7f96e40157e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96e4018910 msgr2=0x7f96e4018d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96e4018910 0x7f96e4018d90 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96e401bc60 msgr2=0x7f96e401c0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96e401bc60 0x7f96e401c0e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f9711ab0640 1 -- 172.21.15.2:0/1139296025 reap_dead start 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96f006c2e0 msgr2=0x7f96f006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96f006c2e0 0x7f96f006e7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9708034000 tx=0x7f970802e000 comp rx=0 tx=0).stop 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 msgr2=0x7f970c0c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c0c5b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f970400c750 tx=0x7f970400cc10 comp rx=0 tx=0).stop 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 shutdown_connections 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96f006c2e0 0x7f96f006e7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 --2- 172.21.15.2:0/1139296025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f970c154b30 0x7f970c0c5b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 >> 172.21.15.2:0/1139296025 conn(0x7f970c0707f0 msgr2=0x7f970c0c3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 shutdown_connections 2024-09-19T05:43:28.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:28.724+0000 7f96f77fe640 1 -- 172.21.15.2:0/1139296025 wait complete. 2024-09-19T05:43:28.757 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:28.757 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:28.757 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:28.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:28.807 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:28.805+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:28.879 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:28.878+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 50.931515894, 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:28.912 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:28.913 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:28.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:28.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:28.913 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:28.913 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:28.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:28.980 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:28.978+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:29.022 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:29.022 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:29.023 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:29.023 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:29.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:29.080 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:29.080+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:29.173 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:29.173 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:29.173 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:29.173 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:29.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:29.182 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:29.180+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:29.282 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:29.281+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:29.287 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:29.287 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:29.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:29.287 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:29.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.287 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.287 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.310063081000003, 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062186530000000002, 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974705+0000", 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1270000000000002e-06 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974716+0000", 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1306e-05 2024-09-19T05:43:29.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974736+0000", 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9293e-05 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974749+0000", 2024-09-19T05:43:29.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3059e-05 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022932900000000001 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975163+0000", 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018497500000000001 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:29.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975314+0000", 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151328 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979256+0000", 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039412179999999998 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571296 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980877+0000", 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0387999999999999e-05 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980919+0000", 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1334000000000001e-05 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.293 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.309940273000002, 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062599709999999996, 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.294 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974824+0000", 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2500000000000004e-07 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974829+0000", 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.66e-06 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974844+0000", 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5160000000000001e-05 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974852+0000", 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9570000000000002e-06 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975748+0000", 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089547100000000001 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975835+0000", 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7190999999999999e-05 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975997+0000", 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016230800000000001 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979261+0000", 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032635310000000001 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980973+0000", 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017117199999999999 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981027+0000", 2024-09-19T05:43:29.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4732999999999999e-05 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981083+0000", 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5414999999999997e-05 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:30 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.309879742, 2024-09-19T05:43:29.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040710980000000004, 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974884+0000", 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6400000000000001e-07 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974899+0000", 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0584000000000001e-05 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974917+0000", 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9799999999999998e-06 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974994+0000", 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6612000000000004e-05 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975176+0000", 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018201400000000001 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975321+0000", 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014440899999999999 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977845+0000", 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002524409 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978841+0000", 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099560399999999998 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978896+0000", 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5075999999999999e-05 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978954+0000", 2024-09-19T05:43:29.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8743999999999999e-05 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.309811797000002, 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040869110000000004, 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.305 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974952+0000", 2024-09-19T05:43:29.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974957+0000", 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7260000000000002e-06 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974966+0000", 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974972+0000", 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0970000000000002e-06 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975780+0000", 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080748199999999999 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975894+0000", 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113867 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976037+0000", 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014333 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978068+0000", 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020306439999999999 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978965+0000", 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089710699999999996 2024-09-19T05:43:29.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979003+0000", 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8797999999999997e-05 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979038+0000", 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4668000000000001e-05 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.309780078999999, 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062404959999999999, 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3900000000000005e-07 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974999+0000", 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5817000000000001e-05 2024-09-19T05:43:29.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975006+0000", 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5509999999999996e-06 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975012+0000", 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8189999999999997e-06 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976240+0000", 2024-09-19T05:43:29.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012283870000000001 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976367+0000", 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126465 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976487+0000", 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120548 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980396+0000", 2024-09-19T05:43:29.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003908702 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981127+0000", 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073081199999999995 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981182+0000", 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5921999999999998e-05 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981223+0000", 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0933999999999998e-05 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1e epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:7bf7e7b5:::200.00000006:head by mds.0.4:37 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.308654625000003, 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028842130000000001, 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.316 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976110+0000", 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2869999999999999e-06 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976123+0000", 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3224000000000001e-05 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976140+0000", 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.755e-05 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976157+0000", 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6464e-05 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976275+0000", 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117602 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978993+0000", 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027180860000000002 2024-09-19T05:43:29.319 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.307447746999998, 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001691115, 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977316+0000", 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209999999999999e-06 2024-09-19T05:43:29.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977337+0000", 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0890000000000002e-05 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977356+0000", 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032999999999999e-05 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977365+0000", 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7649999999999999e-06 2024-09-19T05:43:29.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977425+0000", 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0044999999999999e-05 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977449+0000", 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4006999999999999e-05 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978959+0000", 2024-09-19T05:43:29.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015099379999999999 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979006+0000", 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7315999999999999e-05 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:30 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.307027640000001, 2024-09-19T05:43:29.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118336, 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:29.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977736+0000", 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.096e-06 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0979999999999999e-06 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977756+0000", 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3891e-05 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977765+0000", 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7579999999999998e-06 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977830+0000", 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5184999999999993e-05 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977839+0000", 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4770000000000003e-06 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977854+0000", 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4831000000000001e-05 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:27 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.328 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.306790089000003, 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101439, 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977974+0000", 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6899999999999996e-07 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977979+0000", 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1399999999999999e-06 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977993+0000", 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4215e-05 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978001+0000", 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978055+0000", 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3711000000000001e-05 2024-09-19T05:43:29.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978062+0000", 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978074+0000", 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2238999999999999e-05 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.332 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.305546040000003, 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024064609999999999, 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979218+0000", 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979227+0000", 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979239+0000", 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979247+0000", 2024-09-19T05:43:29.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8369999999999997e-06 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979307+0000", 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0438e-05 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979326+0000", 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8938999999999999e-05 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981594+0000", 2024-09-19T05:43:29.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022679710000000001 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981623+0000", 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9329999999999999e-05 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.305245360000001, 2024-09-19T05:43:29.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021274990000000001, 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7800000000000001e-07 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979524+0000", 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8019999999999997e-06 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979534+0000", 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8859999999999999e-06 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8189999999999992e-06 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979610+0000", 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8433e-05 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979641+0000", 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1233999999999999e-05 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981625+0000", 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001983964 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981645+0000", 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9783e-05 2024-09-19T05:43:29.341 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.305184388000001, 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020817100000000001, 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979598+0000", 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9072e-05 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979609+0000", 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0360000000000001e-05 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979615+0000", 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8700000000000003e-06 2024-09-19T05:43:29.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979866+0000", 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000250454 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979884+0000", 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7725000000000001e-05 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981647+0000", 2024-09-19T05:43:29.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017632629999999999 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981660+0000", 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3502999999999999e-05 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.304697969000003, 2024-09-19T05:43:29.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016104190000000001, 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980066+0000", 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8400000000000005e-07 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980070+0000", 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8010000000000003e-06 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980077+0000", 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.08e-06 2024-09-19T05:43:29.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980082+0000", 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8450000000000002e-06 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980146+0000", 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3589000000000004e-05 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980166+0000", 2024-09-19T05:43:29.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0514e-05 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981661+0000", 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014949539999999999 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981675+0000", 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.304647754999998, 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018022209999999999, 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980116+0000", 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980126+0000", 2024-09-19T05:43:29.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980143+0000", 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6407999999999999e-05 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980150+0000", 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980385+0000", 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023545400000000001 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980403+0000", 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7652999999999999e-05 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981869+0000", 2024-09-19T05:43:29.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014661679999999999 2024-09-19T05:43:29.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981917+0000", 2024-09-19T05:43:29.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319000000000003e-05 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.304048803000001, 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000222196, 2024-09-19T05:43:29.357 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980715+0000", 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980721+0000", 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2620000000000004e-06 2024-09-19T05:43:29.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980735+0000", 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3756e-05 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980759+0000", 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980813+0000", 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4249999999999997e-05 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980821+0000", 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7540000000000006e-06 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980936+0000", 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115574 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.303980740999997, 2024-09-19T05:43:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030787300000000001, 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980783+0000", 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980786+0000", 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5279999999999999e-06 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980795+0000", 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0790000000000007e-06 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980802+0000", 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4910000000000002e-06 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980962+0000", 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015999100000000001 2024-09-19T05:43:29.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980968+0000", 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.874e-06 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981090+0000", 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012244099999999999 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:29 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.303950835000002, 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041785999999999999, 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1600000000000002e-07 2024-09-19T05:43:29.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980816+0000", 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6660000000000001e-06 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980823+0000", 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6789999999999999e-06 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6480000000000002e-06 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981115+0000", 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028752699999999999 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981230+0000", 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108546 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.303928204000002, 2024-09-19T05:43:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011264039999999999, 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:29.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980840+0000", 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1540000000000001e-06 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980848+0000", 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980853+0000", 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9030000000000003e-06 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980907+0000", 2024-09-19T05:43:29.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4224999999999997e-05 2024-09-19T05:43:29.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980937+0000", 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9852000000000001e-05 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981931+0000", 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099441099999999995 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981961+0000", 2024-09-19T05:43:29.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0144000000000001e-05 2024-09-19T05:43:29.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.378 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.378 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 51.303409961, 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013984010000000001, 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:29.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:29.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:29.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981354+0000", 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0160000000000001e-06 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981361+0000", 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981373+0000", 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1439e-05 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981385+0000", 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2041999999999999e-05 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981446+0000", 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1446999999999996e-05 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981469+0000", 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982708+0000", 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012396359999999999 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:29.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982751+0000", 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3198000000000002e-05 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:29.385 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:29.387 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:29.385+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:29.387 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:29.488 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:29.486+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:29.506 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:29.507 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:29.519 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:29.554 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:29.554 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:29.554 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-19T05:43:29.554 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T05:43:29.554 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T05:43:29.555 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-19T05:43:29.604 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:29.603+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:29.658 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:29.658 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T05:43:29.659 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:29.672 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:29.672 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:29.672 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:29.682 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:29.689 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:29.688+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:29.791 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:29.789+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:29.846 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:29.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.868217854000001, 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029411070000000001, 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974719+0000", 2024-09-19T05:43:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1449e-05 2024-09-19T05:43:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974732+0000", 2024-09-19T05:43:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3788000000000001e-05 2024-09-19T05:43:29.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974743+0000", 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0831e-05 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974751+0000", 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1659999999999994e-06 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974913+0000", 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016199899999999999 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975022+0000", 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108857 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011757200000000001 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976884+0000", 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017442969999999999 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977555+0000", 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067127799999999998 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977604+0000", 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8068999999999997e-05 2024-09-19T05:43:29.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977648+0000", 2024-09-19T05:43:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4801e-05 2024-09-19T05:43:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.868151337, 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034736569999999998, 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:29.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974776+0000", 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974783+0000", 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2940000000000002e-06 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974786+0000", 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6959999999999998e-06 2024-09-19T05:43:29.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975441+0000", 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065484099999999997 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975514+0000", 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2608999999999995e-05 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975640+0000", 2024-09-19T05:43:29.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125869 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977680+0000", 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020402739999999999 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978154+0000", 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047386900000000003 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978206+0000", 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2250000000000003e-05 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978247+0000", 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369000000000003e-05 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.860 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.860 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:29.860 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.868117075000001, 2024-09-19T05:43:29.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034986209999999999, 2024-09-19T05:43:29.860 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8999999999999998e-07 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974818+0000", 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6870000000000006e-06 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974826+0000", 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9639999999999999e-06 2024-09-19T05:43:29.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974869+0000", 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3380999999999997e-05 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975009+0000", 2024-09-19T05:43:29.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013971099999999999 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131235 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976896+0000", 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001756482 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978210+0000", 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013140090000000001 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978275+0000", 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000004e-05 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978307+0000", 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899000000000001e-05 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.868083478999999, 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035825869999999999, 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974846+0000", 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974856+0000", 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0701e-05 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974862+0000", 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0669999999999997e-06 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976006+0000", 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001143205 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976080+0000", 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4290999999999997e-05 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976213+0000", 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013278500000000001 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977693+0000", 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014800810000000001 2024-09-19T05:43:29.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978354+0000", 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000660971 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978385+0000", 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1423000000000001e-05 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978424+0000", 2024-09-19T05:43:29.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9131000000000002e-05 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.868053043000003, 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037252449999999999, 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:29.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974873+0000", 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2200000000000004e-07 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974876+0000", 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.965e-06 2024-09-19T05:43:29.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974882+0000", 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3729999999999998e-06 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039999999999995e-06 2024-09-19T05:43:29.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974947+0000", 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6505000000000002e-05 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975044+0000", 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6933000000000002e-05 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975162+0000", 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117879 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977622+0000", 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024597519999999999 2024-09-19T05:43:29.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978546+0000", 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092433599999999999 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978574+0000", 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7923e-05 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978597+0000", 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3553e-05 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:30 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.867344662000001, 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002124711, 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975581+0000", 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0470000000000001e-06 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975592+0000", 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.097e-05 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975603+0000", 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0891e-05 2024-09-19T05:43:29.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975611+0000", 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4569999999999999e-06 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975662+0000", 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1424000000000001e-05 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975687+0000", 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4913999999999999e-05 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977661+0000", 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019738020000000002 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977705+0000", 2024-09-19T05:43:29.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4206000000000003e-05 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:24 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.866562399999999, 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012912539999999999, 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2799999999999995e-07 2024-09-19T05:43:29.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976370+0000", 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8369999999999999e-06 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976381+0000", 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0744000000000001e-05 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976388+0000", 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2320000000000004e-06 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976435+0000", 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6726e-05 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976452+0000", 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6629999999999998e-05 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977623+0000", 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001170894 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977654+0000", 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1362999999999998e-05 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.891 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.866373033999999, 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001404389, 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976553+0000", 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-07 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976558+0000", 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-19T05:43:29.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976568+0000", 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976575+0000", 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976621+0000", 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6136e-05 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976640+0000", 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8604e-05 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977917+0000", 2024-09-19T05:43:29.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00127725 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977956+0000", 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9249000000000003e-05 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.866322754000002, 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099816800000000002, 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:29.897 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976603+0000", 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976610+0000", 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0210000000000002e-06 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5128e-05 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976638+0000", 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2995e-05 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976709+0000", 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0659000000000002e-05 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977601+0000", 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089194000000000003 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.902 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:25 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.866098487999999, 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001150365, 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976836+0000", 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3049999999999998e-06 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976845+0000", 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4020000000000002e-06 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976850+0000", 2024-09-19T05:43:29.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6890000000000001e-06 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5437000000000004e-05 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976933+0000", 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640999999999999e-05 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977941+0000", 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001008175 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6264999999999999e-05 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:16 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.865534285000003, 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000264987, 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977392+0000", 2024-09-19T05:43:29.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3600000000000002e-07 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977396+0000", 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6190000000000002e-06 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977412+0000", 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5919000000000001e-05 2024-09-19T05:43:29.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977420+0000", 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977543+0000", 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012284899999999999 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977550+0000", 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977656+0000", 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010567499999999999 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:17 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.913 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.864853578999998, 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183666, 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978076+0000", 2024-09-19T05:43:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.019e-06 2024-09-19T05:43:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978089+0000", 2024-09-19T05:43:29.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2828e-05 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4389999999999997e-06 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978141+0000", 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5649999999999998e-05 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978149+0000", 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5820000000000004e-06 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978255+0000", 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010655299999999999 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:20 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:29.918 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.864789924999997, 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176971, 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978136+0000", 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2300000000000001e-07 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978139+0000", 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978148+0000", 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978155+0000", 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0169999999999999e-06 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978198+0000", 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3251999999999999e-05 2024-09-19T05:43:29.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978205+0000", 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2250000000000003e-06 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978312+0000", 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010694800000000001 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.922 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:21 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.864755606999999, 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000262632, 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:29.923 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:29.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978173+0000", 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8169999999999999e-06 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978180+0000", 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4839999999999998e-06 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978185+0000", 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978347+0000", 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016165400000000001 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978351+0000", 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1389999999999997e-06 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978432+0000", 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260999999999993e-05 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 51.864472653, 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000148286, 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:29.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978453+0000", 2024-09-19T05:43:29.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978458+0000", 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978472+0000", 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3733e-05 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978479+0000", 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8040000000000004e-06 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978539+0000", 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0038999999999999e-05 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978543+0000", 2024-09-19T05:43:29.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2699999999999998e-06 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978601+0000", 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7881999999999999e-05 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 45.848096317, 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001029049, 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994831+0000", 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2299999999999998e-06 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994844+0000", 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.291e-05 2024-09-19T05:43:29.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994875+0000", 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0898999999999997e-05 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994899+0000", 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3972999999999999e-05 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994994+0000", 2024-09-19T05:43:29.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5068000000000005e-05 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995015+0000", 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1166e-05 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995800+0000", 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078543299999999999 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995858+0000", 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000001e-05 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(2.1d 39/38 [PushReplyOp(2:b871830b:::605.00000000:head)])", 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.284788624000001, 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019826500000000001, 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:29.936 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558137+0000", 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558140+0000", 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4690000000000002e-06 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558149+0000", 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5329999999999994e-06 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558157+0000", 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5639999999999997e-06 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558207+0000", 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9462000000000002e-05 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558335+0000", 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012793599999999999 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(2.1c 39/38 [PushReplyOp(2:3ed09add:::607.00000000:head)])", 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.283008812, 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027980799999999999, 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559917+0000", 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8299999999999998e-07 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559920+0000", 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559929+0000", 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6829999999999997e-06 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559935+0000", 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559989+0000", 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3659999999999998e-05 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.560196+0000", 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207071 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:45 2.6 e42/19 2:654134d2:::mds0_openfiles.0:head v 42'3, mlcod=22'1)", 2024-09-19T05:43:29.943 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8430735670000002, 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084542299999999999, 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:29.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999853+0000", 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.857e-06 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999869+0000", 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5271e-05 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999890+0000", 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1373999999999999e-05 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999902+0000", 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1603999999999999e-05 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999981+0000", 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8935000000000002e-05 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000009+0000", 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8702999999999999e-05 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000650+0000", 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064042099999999996 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000697+0000", 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7258000000000001e-05 2024-09-19T05:43:29.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:29.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:29.948 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:29.949 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:29.950 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:29.947+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:30.050 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:30.048+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:30.065 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:30.065 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:30.065 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:30.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:30.150 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:30.149+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:30.184 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:30.184 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:30.184 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:30.193 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:30.251 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:30.250+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:30.306 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.306 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 67.423506449000001, 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:30.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:30.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:30.314 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.784421640999994, 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:43:30.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:43:30.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.784379111999996, 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:43:30.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:43:30.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.784261552999993, 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:43:30.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:43:30.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.333 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.755004217999996, 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:43:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:43:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:43:30.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.754947279999996, 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:43:30.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:43:30.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.600887548000003, 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:43:30.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:30.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:43:30.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:30.354 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.600815217000005, 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:43:30.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:43:30.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.600801668000003, 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:43:30.360 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:43:30.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:43:30.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.195634767000001, 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:43:30.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:43:30.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.375 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.195556170000003, 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:43:30.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.381 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.195544232000003, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:43:30.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:43:30.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.195533151999996, 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:30.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:43:30.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:43:30.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.195458587000005, 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:30.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:43:30.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.184032047000002, 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:43:30.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 66.183969904999998, 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:30.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:43:30.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:43:30.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:43:30.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:43:30.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:43:30.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:43:30.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.706746483000003, 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:43:30.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:43:30.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.706725118999998, 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.706707417000004, 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:30.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:30.430 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.706654032000003, 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:43:30.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:43:30.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:43:30.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.437 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:30.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:30.439 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:30.437+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:30.539 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:30.537+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:30.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:43:30.617 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:30.617 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:30.617 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:30.617 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:30.628 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:30.640 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:30.638+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.710+0000 7f0e79386640 1 Processor -- start 2024-09-19T05:43:30.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.710+0000 7f0e79386640 1 -- start start 2024-09-19T05:43:30.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.711+0000 7f0e79386640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e741592a0 0x7f0e7415f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.711+0000 7f0e79386640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e740c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.711+0000 7f0e79386640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e7406cea0 con 0x7f0e74158ec0 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.711+0000 7f0e79386640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0e7406d180 con 0x7f0e741592a0 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.711+0000 7f0e79386640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7415eeb0 con 0x7f0e7406ea80 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e72ffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f0e74158ec0 0x7f0e740c4ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54770/0 (socket says 172.21.15.2:54770) 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e72ffd640 1 -- 172.21.15.2:0/2600530905 learned_addr learned my addr 172.21.15.2:0/2600530905 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e727fc640 1 --2- 172.21.15.2:0/2600530905 >> v2:172.21.15.123:3300/0 conn(0x7f0e741592a0 0x7f0e7415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e737fe640 1 --2- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e740c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e71ffb640 1 -- 172.21.15.2:0/2600530905 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1065392842 0 0) 0x7f0e7406cea0 con 0x7f0e74158ec0 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e71ffb640 1 -- 172.21.15.2:0/2600530905 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e5c003520 con 0x7f0e74158ec0 2024-09-19T05:43:30.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e727fc640 1 -- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 msgr2=0x7f0e740c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e727fc640 1 --2- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e740c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e727fc640 1 -- 172.21.15.2:0/2600530905 >> v1:172.21.15.2:6789/0 conn(0x7f0e74158ec0 legacy=0x7f0e740c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.712+0000 7f0e727fc640 1 -- 172.21.15.2:0/2600530905 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7415f190 con 0x7f0e741592a0 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.713+0000 7f0e727fc640 1 --2- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e741592a0 0x7f0e7415f990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097d0 tx=0x7f0e6802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c5b6450f6c1d7bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.713+0000 7f0e71ffb640 1 -- 172.21.15.2:0/2600530905 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68037070 con 0x7f0e741592a0 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.713+0000 7f0e71ffb640 1 -- 172.21.15.2:0/2600530905 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e68032440 con 0x7f0e741592a0 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.713+0000 7f0e71ffb640 1 -- 172.21.15.2:0/2600530905 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680327f0 con 0x7f0e741592a0 2024-09-19T05:43:30.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 -- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e741592a0 msgr2=0x7f0e7415f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 --2- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e741592a0 0x7f0e7415f990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097d0 tx=0x7f0e6802a250 comp rx=0 tx=0).stop 2024-09-19T05:43:30.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 -- 172.21.15.2:0/2600530905 shutdown_connections 2024-09-19T05:43:30.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 --2- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e740c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 --2- 172.21.15.2:0/2600530905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e741592a0 0x7f0e7415f990 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097d0 tx=0x7f0e6802a250 comp rx=0 tx=0).stop 2024-09-19T05:43:30.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 -- 172.21.15.2:0/2600530905 >> 172.21.15.2:0/2600530905 conn(0x7f0e7406c2d0 msgr2=0x7f0e74067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 -- 172.21.15.2:0/2600530905 shutdown_connections 2024-09-19T05:43:30.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.714+0000 7f0e79386640 1 -- 172.21.15.2:0/2600530905 wait complete. 2024-09-19T05:43:30.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.716+0000 7f0e79386640 1 Processor -- start 2024-09-19T05:43:30.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.716+0000 7f0e79386640 1 -- start start 2024-09-19T05:43:30.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e79386640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e74160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e79386640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e74158ec0 0x7f0e74153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e79386640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e740cc630 0x7f0e74154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e79386640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0e740c5700 con 0x7f0e740cc630 2024-09-19T05:43:30.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e79386640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7406d460 con 0x7f0e74158ec0 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e727fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e74158ec0 0x7f0e74153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e74160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e72ffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e74160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:36604/0 (socket says 172.21.15.2:36604) 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e72ffd640 1 -- 172.21.15.2:0/2327550970 learned_addr learned my addr 172.21.15.2:0/2327550970 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e737fe640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e740cc630 0x7f0e74154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.717+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0e740c5880 con 0x7f0e7406ea80 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e72ffd640 1 -- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e74158ec0 msgr2=0x7f0e74153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e72ffd640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e74158ec0 0x7f0e74153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e72ffd640 1 -- 172.21.15.2:0/2327550970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e740cc630 msgr2=0x7f0e74154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e72ffd640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e740cc630 0x7f0e74154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e72ffd640 1 -- 172.21.15.2:0/2327550970 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e740ccca0 con 0x7f0e7406ea80 2024-09-19T05:43:30.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e727fc640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e74158ec0 0x7f0e74153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e737fe640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e740cc630 0x7f0e74154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e72ffd640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e74160eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c430 tx=0x7f0e6400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e5b7fe640 1 -- 172.21.15.2:0/2327550970 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e64017070 con 0x7f0e7406ea80 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e5b7fe640 1 -- 172.21.15.2:0/2327550970 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e64005ce0 con 0x7f0e7406ea80 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.718+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e740ccf60 con 0x7f0e7406ea80 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.719+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e741603e0 con 0x7f0e7406ea80 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.719+0000 7f0e5b7fe640 1 -- 172.21.15.2:0/2327550970 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e640072f0 con 0x7f0e7406ea80 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.721+0000 7f0e5b7fe640 1 -- 172.21.15.2:0/2327550970 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0e6401a070 con 0x7f0e7406ea80 2024-09-19T05:43:30.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.721+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e741e9c30 con 0x7f0e7406ea80 2024-09-19T05:43:30.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.721+0000 7f0e5b7fe640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f0e6006c2e0 0x7f0e6006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.721+0000 7f0e5b7fe640 1 -- 172.21.15.2:0/2327550970 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f0e6408a7e0 con 0x7f0e7406ea80 2024-09-19T05:43:30.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.722+0000 7f0e727fc640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f0e6006c2e0 0x7f0e6006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.722+0000 7f0e727fc640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f0e6006c2e0 0x7f0e6006e7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e68034000 tx=0x7f0e6802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.729+0000 7f0e5b7fe640 1 -- 172.21.15.2:0/2327550970 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0e640588f0 con 0x7f0e7406ea80 2024-09-19T05:43:30.737 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:30.741 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:30.740+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:30.795 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:30.796 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:30.796 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:30.796 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:30.796 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:30.805 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:30.850 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:30.849+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.904+0000 7f7ef7065640 1 Processor -- start 2024-09-19T05:43:30.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.904+0000 7f7ef7065640 1 -- start start 2024-09-19T05:43:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef7065640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef01528e0 0x7f7ef00c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef7065640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ef00c34d0 0x7f7ef00c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef7065640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7ef0152cc0 0x7f7ef0158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef7065640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef015cc30 con 0x7f7ef0152cc0 2024-09-19T05:43:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef7065640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef015bdd0 con 0x7f7ef00c34d0 2024-09-19T05:43:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef7065640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef015c020 con 0x7f7ef01528e0 2024-09-19T05:43:30.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7eeffff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ef00c34d0 0x7f7ef00c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef5602640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7ef0152cc0 0x7f7ef0158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef5602640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7ef0152cc0 0x7f7ef0158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59260/0 (socket says 172.21.15.2:59260) 2024-09-19T05:43:30.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.906+0000 7f7ef5602640 1 -- 172.21.15.2:0/2261689218 learned_addr learned my addr 172.21.15.2:0/2261689218 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:30.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.907+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2261689218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef01528e0 0x7f7ef00c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.907+0000 7f7ef5602640 1 -- 172.21.15.2:0/2261689218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef01528e0 msgr2=0x7f7ef00c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.907+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2261689218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef01528e0 0x7f7ef00c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.907+0000 7f7ef5602640 1 -- 172.21.15.2:0/2261689218 >> v2:172.21.15.123:3300/0 conn(0x7f7ef00c34d0 msgr2=0x7f7ef00c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.907+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2261689218 >> v2:172.21.15.123:3300/0 conn(0x7f7ef00c34d0 0x7f7ef00c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.907+0000 7f7ef5602640 1 -- 172.21.15.2:0/2261689218 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef015c300 con 0x7f7ef0152cc0 2024-09-19T05:43:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.908+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2261689218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef0158a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00721e0 tx=0x7f7ee802ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5871aea2bd3b4570 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.909+0000 7f7eef7fe640 1 -- 172.21.15.2:0/2261689218 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8037020 con 0x7f7ef0152cc0 2024-09-19T05:43:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.909+0000 7f7eef7fe640 1 -- 172.21.15.2:0/2261689218 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8033680 con 0x7f7ef0152cc0 2024-09-19T05:43:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.909+0000 7f7eef7fe640 1 -- 172.21.15.2:0/2261689218 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8033ab0 con 0x7f7ef0152cc0 2024-09-19T05:43:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 -- 172.21.15.2:0/2261689218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 msgr2=0x7f7ef0158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2261689218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef0158a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00721e0 tx=0x7f7ee802ea50 comp rx=0 tx=0).stop 2024-09-19T05:43:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 -- 172.21.15.2:0/2261689218 shutdown_connections 2024-09-19T05:43:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2261689218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef0158a00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2261689218 >> v2:172.21.15.123:3300/0 conn(0x7f7ef00c34d0 0x7f7ef00c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2261689218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef01528e0 0x7f7ef00c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 -- 172.21.15.2:0/2261689218 >> 172.21.15.2:0/2261689218 conn(0x7f7ef00707f0 msgr2=0x7f7ef0070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:30.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 -- 172.21.15.2:0/2261689218 shutdown_connections 2024-09-19T05:43:30.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.910+0000 7f7ef7065640 1 -- 172.21.15.2:0/2261689218 wait complete. 2024-09-19T05:43:30.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.912+0000 7f7ef7065640 1 Processor -- start 2024-09-19T05:43:30.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.912+0000 7f7ef7065640 1 -- start start 2024-09-19T05:43:30.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef7065640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef00c34d0 0x7f7ef01ec310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef7065640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ef01528e0 0x7f7ef00cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef7065640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef7065640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef00c8f20 con 0x7f7ef0152cc0 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef7065640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef015c8c0 con 0x7f7ef01528e0 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef7065640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef00cbc00 con 0x7f7ef00c34d0 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef5602640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef5602640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef006cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59272/0 (socket says 172.21.15.2:59272) 2024-09-19T05:43:30.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 learned_addr learned my addr 172.21.15.2:0/2062151675 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7eeffff640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ef01528e0 0x7f7ef00cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.913+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef00c34d0 0x7f7ef01ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef00c34d0 msgr2=0x7f7ef01ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ef00c34d0 0x7f7ef01ec310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ef01528e0 msgr2=0x7f7ef00cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ef01528e0 0x7f7ef00cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef01efd40 con 0x7f7ef0152cc0 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef006cf50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009f40 tx=0x7f7ee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8037040 con 0x7f7ef0152cc0 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8003e10 con 0x7f7ef0152cc0 2024-09-19T05:43:30.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.914+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8004260 con 0x7f7ef0152cc0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.915+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef01ec850 con 0x7f7ef0152cc0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.915+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01eef80 con 0x7f7ef0152cc0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.917+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7ef00c9200 con 0x7f7ef0152cc0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.917+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7ee803a020 con 0x7f7ef0152cc0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.918+0000 7f7eed7fa640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7ed406c2b0 0x7f7ed406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.918+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f7ee80b0440 con 0x7f7ef0152cc0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.918+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v44) ==== 54+0+16 (secure 0 0 0) 0x7f7ee80b2a20 con 0x7f7ef0152cc0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.918+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7ed406c2b0 0x7f7ed406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.918+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ef006d490 0x7f7ef00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.918+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ef01ef690 con 0x7f7ef006d490 2024-09-19T05:43:30.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.919+0000 7f7eeffff640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ef006d490 0x7f7ef00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.919+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7ed406c2b0 0x7f7ed406e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009b40 tx=0x7f7ee4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.919+0000 7f7eeffff640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ef006d490 0x7f7ef00cea70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.920+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef01ef690 con 0x7f7ef006d490 2024-09-19T05:43:30.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.935+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef01ef8b0 con 0x7f7ef006d490 2024-09-19T05:43:30.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.937+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef01ef8b0 con 0x7f7ef006d490 2024-09-19T05:43:30.939 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:30.939 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:30.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.938+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ef015b840 0x7f7ef01e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:30.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.938+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7ef015bc20 con 0x7f7ef015b840 2024-09-19T05:43:30.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.938+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ef015b840 0x7f7ef01e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:30.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.938+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ef015b840 0x7f7ef01e2cb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:30.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.940+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef015bc20 con 0x7f7ef015b840 2024-09-19T05:43:30.943 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:30.942+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:30.945 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:30.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.967347873000001, 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.946 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:43:30.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.967290310000003, 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:30.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:43:30.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.967217834000003, 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:43:30.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:43:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:43:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.967180575999997, 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:43:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:43:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.967140774000001, 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:43:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.966311613000002, 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:43:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:43:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:43:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.965886664000003, 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:43:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.961693676000003, 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:43:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:43:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:43:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.959111763999999, 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:43:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:43:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.958700004000001, 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:43:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 44.388316592999999, 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.071649639, 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:43:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:43:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.069564229000001, 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:43:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:43:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:43:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 18'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@18'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.464842078, 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084064700000000003, 2024-09-19T05:43:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477339+0000", 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000002e-07 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477650+0000", 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031168299999999999 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477668+0000", 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7745e-05 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477689+0000", 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0644000000000001e-05 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477763+0000", 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3919999999999997e-05 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477772+0000", 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2609999999999999e-06 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.478179+0000", 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00040692799999999998 2024-09-19T05:43:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 18'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@18'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.462510102, 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010886400000000001, 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479671+0000", 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7300000000000005e-07 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480001+0000", 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033000999999999999 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480025+0000", 2024-09-19T05:43:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4053000000000002e-05 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480044+0000", 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.925e-05 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480142+0000", 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7955000000000007e-05 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480156+0000", 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3553999999999999e-05 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480759+0000", 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060294500000000002 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(1.0 40/39 [PushReplyOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head)])", 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.21500601, 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023152599999999999, 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727175+0000", 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5000000000000003e-07 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727179+0000", 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0620000000000002e-06 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727190+0000", 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.116e-05 2024-09-19T05:43:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727196+0000", 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.404e-06 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727258+0000", 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1362e-05 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727406+0000", 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014798799999999999 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(1.0 40/39 [PushReplyOp(1:30306672:devicehealth::main.db.0000000000000000:head)])", 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.213036006999999, 2024-09-19T05:43:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204537, 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729145+0000", 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729148+0000", 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.495e-06 2024-09-19T05:43:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729156+0000", 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3590000000000008e-06 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729165+0000", 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0390000000000005e-06 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729208+0000", 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.316e-05 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729349+0000", 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014112700000000001 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:43 2.14 2:2e2fa760:::200.00000001:head [write 4260~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0603370249999999, 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015875220000000001, 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881846+0000", 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0019999999999998e-06 2024-09-19T05:43:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881862+0000", 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5673999999999999e-05 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881887+0000", 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4825999999999999e-05 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881901+0000", 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3866000000000001e-05 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881991+0000", 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0592000000000003e-05 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882055+0000", 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3559000000000006e-05 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882147+0000", 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1955999999999998e-05 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882985+0000", 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083855599999999996 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883339+0000", 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035373799999999998 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883390+0000", 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883431+0000", 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0593000000000003e-05 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:44 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0602476149999998, 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001741942, 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881937+0000", 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9990000000000002e-06 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881948+0000", 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0711e-05 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881954+0000", 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2260000000000002e-06 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882539+0000", 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000584675 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882625+0000", 2024-09-19T05:43:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6278000000000003e-05 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882741+0000", 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115909 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883444+0000", 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070269799999999997 2024-09-19T05:43:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883601+0000", 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000157825 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883640+0000", 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8702000000000003e-05 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883675+0000", 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4445e-05 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:43 2.14 e42/19 ondisk, result = 0)", 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0589374999999999, 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019390400000000001, 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883244+0000", 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883248+0000", 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2100000000000003e-06 2024-09-19T05:43:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883259+0000", 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0824e-05 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883275+0000", 2024-09-19T05:43:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6415000000000001e-05 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883325+0000", 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883333+0000", 2024-09-19T05:43:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9859999999999994e-06 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883437+0000", 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010347 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:31.063 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:31.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:31.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.969+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef00ce030 con 0x7f7ef015b840 2024-09-19T05:43:31.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.970+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef00ce030 con 0x7f7ef015b840 2024-09-19T05:43:31.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.971+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ef01e3870 0x7f7ef0062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.971+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f7ef0062b70 con 0x7f7ef01e3870 2024-09-19T05:43:31.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.971+0000 7f7eeffff640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ef01e3870 0x7f7ef0062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.971+0000 7f7eeffff640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ef01e3870 0x7f7ef0062630 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:30.973+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef0062b70 con 0x7f7ef01e3870 2024-09-19T05:43:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.000+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef0063080 con 0x7f7ef01e3870 2024-09-19T05:43:31.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.003+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef0063080 con 0x7f7ef01e3870 2024-09-19T05:43:31.068 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.004+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ef00b20f0 0x7f7ef00b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.004+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f7ef00b2a90 con 0x7f7ef00b20f0 2024-09-19T05:43:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.004+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ef00b20f0 0x7f7ef00b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.004+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ef00b20f0 0x7f7ef00b2550 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.005+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef00b2a90 con 0x7f7ef00b20f0 2024-09-19T05:43:31.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.033+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef002c910 con 0x7f7ef00b20f0 2024-09-19T05:43:31.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.036+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef002c910 con 0x7f7ef00b20f0 2024-09-19T05:43:31.071 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.036+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ef002d030 0x7f7ef002d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.036+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f7ef00b2a90 con 0x7f7ef002d030 2024-09-19T05:43:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.037+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ef002d030 0x7f7ef002d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.038+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ef002d030 0x7f7ef002d470 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:31.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.038+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f0e740c4ab0 con 0x7f0e7406ea80 2024-09-19T05:43:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.038+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef00b2a90 con 0x7f7ef002d030 2024-09-19T05:43:31.073 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:31.064+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.066+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef00b2a90 con 0x7f7ef002d030 2024-09-19T05:43:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.068+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef00b2a90 con 0x7f7ef002d030 2024-09-19T05:43:31.074 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.068+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f7ef0200700 0x7f7ef0202af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.068+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f7ef002da80 con 0x7f7ef0200700 2024-09-19T05:43:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.069+0000 7f7eeffff640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f7ef0200700 0x7f7ef0202af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.069+0000 7f7eeffff640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f7ef0200700 0x7f7ef0202af0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.070+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef002da80 con 0x7f7ef0200700 2024-09-19T05:43:31.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.097+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef006e580 con 0x7f7ef0200700 2024-09-19T05:43:31.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.099+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef006e580 con 0x7f7ef0200700 2024-09-19T05:43:31.101 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:31.102 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:31.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.100+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ef0203540 0x7f7ef0205930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:31.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.100+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f7ef02088b0 con 0x7f7ef0203540 2024-09-19T05:43:31.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.100+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ef0203540 0x7f7ef0205930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:31.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.101+0000 7f7ef5602640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ef0203540 0x7f7ef0205930 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:31.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.101+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef02088b0 con 0x7f7ef0203540 2024-09-19T05:43:31.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.129+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef002da80 con 0x7f7ef0203540 2024-09-19T05:43:31.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.131+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef002da80 con 0x7f7ef0203540 2024-09-19T05:43:31.132 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.132+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ef0206420 0x7f7ef0206860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.132+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f7ef020bcd0 con 0x7f7ef0206420 2024-09-19T05:43:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.132+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ef0206420 0x7f7ef0206860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.133+0000 7f7ef4e01640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ef0206420 0x7f7ef0206860 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.133+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ef020bcd0 con 0x7f7ef0206420 2024-09-19T05:43:31.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.162+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7ef0206e10 con 0x7f7ef0206420 2024-09-19T05:43:31.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.164+0000 7f7eed7fa640 1 -- 172.21.15.2:0/2062151675 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ef0206e10 con 0x7f7ef0206420 2024-09-19T05:43:31.165 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:31.168 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ef006d490 msgr2=0x7f7ef00cea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ef006d490 0x7f7ef00cea70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ef015b840 msgr2=0x7f7ef01e2cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ef015b840 0x7f7ef01e2cb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ef01e3870 msgr2=0x7f7ef0062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ef01e3870 0x7f7ef0062630 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ef00b20f0 msgr2=0x7f7ef00b2550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ef00b20f0 0x7f7ef00b2550 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ef002d030 msgr2=0x7f7ef002d470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ef002d030 0x7f7ef002d470 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f7ef0200700 msgr2=0x7f7ef0202af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f7ef0200700 0x7f7ef0202af0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ef0203540 msgr2=0x7f7ef0205930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ef0203540 0x7f7ef0205930 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ef0206420 msgr2=0x7f7ef0206860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ef0206420 0x7f7ef0206860 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 reap_dead start 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 reap_dead start 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7ed406c2b0 msgr2=0x7f7ed406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7ed406c2b0 0x7f7ed406e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009b40 tx=0x7f7ee4009190 comp rx=0 tx=0).stop 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 msgr2=0x7f7ef006cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef7065640 1 --2- 172.21.15.2:0/2062151675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ef0152cc0 0x7f7ef006cf50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009f40 tx=0x7f7ee8004a40 comp rx=0 tx=0).stop 2024-09-19T05:43:31.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 reap_dead start 2024-09-19T05:43:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 reap_dead start 2024-09-19T05:43:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.165+0000 7f7ef5602640 1 -- 172.21.15.2:0/2062151675 reap_dead start 2024-09-19T05:43:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.166+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 shutdown_connections 2024-09-19T05:43:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.166+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 >> 172.21.15.2:0/2062151675 conn(0x7f7ef00707f0 msgr2=0x7f7ef00ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.166+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 shutdown_connections 2024-09-19T05:43:31.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.166+0000 7f7ef7065640 1 -- 172.21.15.2:0/2062151675 wait complete. 2024-09-19T05:43:31.171 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:31.164+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:31.202 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:31.203 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:31.212 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:31.296 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:31.295+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:31.367 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:31.366+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:31.421 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:31.421 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:31.433 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:31.469 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:31.467+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:31.570 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:31.569+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:31.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.584+0000 7f0e5b7fe640 1 -- 172.21.15.2:0/2327550970 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v45) ==== 91+0+0 (secure 0 0 0) 0x7f0e6405cbf0 con 0x7f0e7406ea80 2024-09-19T05:43:31.586 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:43:31.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.587+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f0e6006c2e0 msgr2=0x7f0e6006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.587+0000 7f0e79386640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f0e6006c2e0 0x7f0e6006e7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e68034000 tx=0x7f0e6802e000 comp rx=0 tx=0).stop 2024-09-19T05:43:31.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 msgr2=0x7f0e74160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:31.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e74160eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0e6400c430 tx=0x7f0e6400c8f0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 shutdown_connections 2024-09-19T05:43:31.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f0e6006c2e0 0x7f0e6006e7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e74158ec0 0x7f0e74153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0e740cc630 0x7f0e74154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 --2- 172.21.15.2:0/2327550970 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0e7406ea80 0x7f0e74160eb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:31.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 >> 172.21.15.2:0/2327550970 conn(0x7f0e7406c2d0 msgr2=0x7f0e7415c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:31.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 shutdown_connections 2024-09-19T05:43:31.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:31.588+0000 7f0e79386640 1 -- 172.21.15.2:0/2327550970 wait complete. 2024-09-19T05:43:31.593 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:31.593 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:31.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:31.671 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:31.669+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:31.721 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:31.722 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:31.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:31.772 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:31.770+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:31.873 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:31.871+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:31.875 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:31.875 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:31.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:31.973 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:31.972+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:32.027 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.028 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:32.081 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:32.080+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:32.140 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.140 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:32.175 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:32.174+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:32.277 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:32.276+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:32.291 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.291 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:32.377 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:32.376+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:32.403 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.403 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.411 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:32.497 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:32.495+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:32.556 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.556 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.565 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:32.578 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:32.577+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:32.680 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:32.679+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:32.710 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.710 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:32.780 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:32.779+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:32.822 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.822 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:32.881 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:32.880+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:32.933 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:32.933 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:32.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:32.982 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:32.981+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:33.043 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:33.043 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:33.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:33.095 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:33.094+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:33.155 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:33.155 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:33.164 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:33.184 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:33.208 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:33.207+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:33.284 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:33.283+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:33.290 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:33.291 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:33.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:33.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.359+0000 7f10e46ba640 1 Processor -- start 2024-09-19T05:43:33.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.359+0000 7f10e46ba640 1 -- start start 2024-09-19T05:43:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e46ba640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10dc151b50 0x7f10dc0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e46ba640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e46ba640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f10dc0c7400 0x7f10dc0c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e46ba640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc0c4780 con 0x7f10dc0c7400 2024-09-19T05:43:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e46ba640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc06fc20 con 0x7f10dc151b50 2024-09-19T05:43:33.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e46ba640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc0c4a00 con 0x7f10dc071160 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e1c55640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e2456640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10dc151b50 0x7f10dc0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e2c57640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f10dc0c7400 0x7f10dc0c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e1c55640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:53804/0 (socket says 172.21.15.2:53804) 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.360+0000 7f10e1c55640 1 -- 172.21.15.2:0/2098626465 learned_addr learned my addr 172.21.15.2:0/2098626465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.361+0000 7f10e1c55640 1 -- 172.21.15.2:0/2098626465 >> v2:172.21.15.123:3300/0 conn(0x7f10dc151b50 msgr2=0x7f10dc0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.361+0000 7f10e1c55640 1 --2- 172.21.15.2:0/2098626465 >> v2:172.21.15.123:3300/0 conn(0x7f10dc151b50 0x7f10dc0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.361+0000 7f10e1c55640 1 -- 172.21.15.2:0/2098626465 >> v2:172.21.15.2:3300/0 conn(0x7f10dc0c7400 msgr2=0x7f10dc0c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.361+0000 7f10e1c55640 1 --2- 172.21.15.2:0/2098626465 >> v2:172.21.15.2:3300/0 conn(0x7f10dc0c7400 0x7f10dc0c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.361+0000 7f10e1c55640 1 -- 172.21.15.2:0/2098626465 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc0703f0 con 0x7f10dc071160 2024-09-19T05:43:33.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.361+0000 7f10e2c57640 1 --2- 172.21.15.2:0/2098626465 >> v2:172.21.15.2:3300/0 conn(0x7f10dc0c7400 0x7f10dc0c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.362+0000 7f10e1c55640 1 --2- 172.21.15.2:0/2098626465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc1576e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0cadf0 tx=0x7f10cc02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec17e41ff8a8158f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.362+0000 7f10e1454640 1 -- 172.21.15.2:0/2098626465 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc037070 con 0x7f10dc071160 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.362+0000 7f10e1454640 1 -- 172.21.15.2:0/2098626465 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc032440 con 0x7f10dc071160 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.362+0000 7f10e1454640 1 -- 172.21.15.2:0/2098626465 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc032810 con 0x7f10dc071160 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 -- 172.21.15.2:0/2098626465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 msgr2=0x7f10dc1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 --2- 172.21.15.2:0/2098626465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc1576e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0cadf0 tx=0x7f10cc02a1e0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 -- 172.21.15.2:0/2098626465 shutdown_connections 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 --2- 172.21.15.2:0/2098626465 >> v2:172.21.15.2:3300/0 conn(0x7f10dc0c7400 0x7f10dc0c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 --2- 172.21.15.2:0/2098626465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc1576e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 --2- 172.21.15.2:0/2098626465 >> v2:172.21.15.123:3300/0 conn(0x7f10dc151b50 0x7f10dc0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 -- 172.21.15.2:0/2098626465 >> 172.21.15.2:0/2098626465 conn(0x7f10dc072a40 msgr2=0x7f10dc072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 -- 172.21.15.2:0/2098626465 shutdown_connections 2024-09-19T05:43:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.363+0000 7f10e46ba640 1 -- 172.21.15.2:0/2098626465 wait complete. 2024-09-19T05:43:33.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.365+0000 7f10e46ba640 1 Processor -- start 2024-09-19T05:43:33.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.365+0000 7f10e46ba640 1 -- start start 2024-09-19T05:43:33.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e46ba640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc14a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e46ba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10dc0c7400 0x7f10dc1421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e46ba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10dc151b50 0x7f10dc142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e2456640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc14a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e2456640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc14a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:53812/0 (socket says 172.21.15.2:53812) 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e1c55640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10dc0c7400 0x7f10dc1421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e1c55640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10dc0c7400 0x7f10dc1421f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42620/0 (socket says 172.21.15.2:42620) 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e2c57640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10dc151b50 0x7f10dc142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e2c57640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10dc151b50 0x7f10dc142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36018/0 (socket says 172.21.15.2:36018) 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e2456640 1 -- 172.21.15.2:0/4217897045 learned_addr learned my addr 172.21.15.2:0/4217897045 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e46ba640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0c4ce0 con 0x7f10dc151b50 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e46ba640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0cd3b0 con 0x7f10dc0c7400 2024-09-19T05:43:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.367+0000 7f10e46ba640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc0c7b80 con 0x7f10dc071160 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10e1c55640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 msgr2=0x7f10dc14a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10e1c55640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10dc071160 0x7f10dc14a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10e1c55640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10dc151b50 msgr2=0x7f10dc142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10e1c55640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10dc151b50 0x7f10dc142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10e1c55640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc158a40 con 0x7f10dc0c7400 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10e2c57640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10dc151b50 0x7f10dc142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10e1c55640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10dc0c7400 0x7f10dc1421f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10cc031000 tx=0x7f10cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc041020 con 0x7f10dc0c7400 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.368+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc03d660 con 0x7f10dc0c7400 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.369+0000 7f10e46ba640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc158d20 con 0x7f10dc0c7400 2024-09-19T05:43:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.369+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc037040 con 0x7f10dc0c7400 2024-09-19T05:43:33.372 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.369+0000 7f10e46ba640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1592e0 con 0x7f10dc0c7400 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.370+0000 7f10e46ba640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f10dc0cd8d0 con 0x7f10dc0c7400 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.370+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f10cc044020 con 0x7f10dc0c7400 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.371+0000 7f10d2ffd640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f10c006c2b0 0x7f10c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.371+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f10cc0abb20 con 0x7f10dc0c7400 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.371+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f10c006c2b0 0x7f10c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.371+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v45) ==== 54+0+16 (secure 0 0 0) 0x7f10cc04b030 con 0x7f10dc0c7400 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.372+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f10b8001650 0x7f10b8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.372+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f10c006c2b0 0x7f10c006e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10dc158850 tx=0x7f10d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.372+0000 7f10e2c57640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f10b8001650 0x7f10b8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.372+0000 7f10e2c57640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f10b8001650 0x7f10b8003b10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.372+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10b8006bb0 con 0x7f10b8001650 2024-09-19T05:43:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.374+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b8006bb0 con 0x7f10b8001650 2024-09-19T05:43:33.385 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:33.383+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:33.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.388+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b8005c60 con 0x7f10b8001650 2024-09-19T05:43:33.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.390+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b8005c60 con 0x7f10b8001650 2024-09-19T05:43:33.391 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:33.391 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:33.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.390+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f10b8004190 0x7f10b8004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.390+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f10b8004c00 con 0x7f10b8004190 2024-09-19T05:43:33.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.390+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f10b8004190 0x7f10b8004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.391+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f10b8004190 0x7f10b8004640 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.392+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b8004c00 con 0x7f10b8004190 2024-09-19T05:43:33.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.406+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b80051a0 con 0x7f10b8004190 2024-09-19T05:43:33.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.409+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b80051a0 con 0x7f10b8004190 2024-09-19T05:43:33.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.409+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f10b8009370 0x7f10b800b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.409+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f10b800e720 con 0x7f10b8009370 2024-09-19T05:43:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.409+0000 7f10e1c55640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f10b8009370 0x7f10b800b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.410+0000 7f10e1c55640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f10b8009370 0x7f10b800b760 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.411+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b800e720 con 0x7f10b8009370 2024-09-19T05:43:33.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.425+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b800d730 con 0x7f10b8009370 2024-09-19T05:43:33.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.427+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b800d730 con 0x7f10b8009370 2024-09-19T05:43:33.428 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.427+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f10b800bd20 0x7f10b800c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.427+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f10b800e3f0 con 0x7f10b800bd20 2024-09-19T05:43:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.428+0000 7f10e2c57640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f10b800bd20 0x7f10b800c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.428+0000 7f10e2c57640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f10b800bd20 0x7f10b800c1b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.429+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b800e3f0 con 0x7f10b800bd20 2024-09-19T05:43:33.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.443+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b800c7d0 con 0x7f10b800bd20 2024-09-19T05:43:33.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.445+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b800c7d0 con 0x7f10b800bd20 2024-09-19T05:43:33.447 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.446+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f10b800d070 0x7f10b8012740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.446+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f10b80157e0 con 0x7f10b800d070 2024-09-19T05:43:33.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.446+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f10b800d070 0x7f10b8012740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.446+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f10b800d070 0x7f10b8012740 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.447+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b80157e0 con 0x7f10b800d070 2024-09-19T05:43:33.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.461+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b8014500 con 0x7f10b800d070 2024-09-19T05:43:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.463+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b8014500 con 0x7f10b800d070 2024-09-19T05:43:33.464 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.463+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f10b8014fe0 0x7f10b8012dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.463+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f10b8015440 con 0x7f10b8014fe0 2024-09-19T05:43:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.463+0000 7f10e1c55640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f10b8014fe0 0x7f10b8012dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.464+0000 7f10e1c55640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f10b8014fe0 0x7f10b8012dc0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.464+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b8015440 con 0x7f10b8014fe0 2024-09-19T05:43:33.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.478+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b8015440 con 0x7f10b8014fe0 2024-09-19T05:43:33.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.480+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b8015440 con 0x7f10b8014fe0 2024-09-19T05:43:33.481 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.480+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f10b8013810 0x7f10b8013c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.480+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f10b8014310 con 0x7f10b8013810 2024-09-19T05:43:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.480+0000 7f10e2c57640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f10b8013810 0x7f10b8013c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.481+0000 7f10e2c57640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f10b8013810 0x7f10b8013c90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.482+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10d4025040 con 0x7f10b8013810 2024-09-19T05:43:33.485 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:33.484+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:33.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.495+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b8019ac0 con 0x7f10b8013810 2024-09-19T05:43:33.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.497+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b8019ac0 con 0x7f10b8013810 2024-09-19T05:43:33.498 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:33.499 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:33.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.497+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f10b801bae0 0x7f10b8019a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.497+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f10b801c010 con 0x7f10b801bae0 2024-09-19T05:43:33.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.497+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f10b801bae0 0x7f10b8019a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.498+0000 7f10e2456640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f10b801bae0 0x7f10b8019a40 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.498+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10b801c010 con 0x7f10b801bae0 2024-09-19T05:43:33.505 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:33.506 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:33.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.512+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10b801b1b0 con 0x7f10b801bae0 2024-09-19T05:43:33.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.514+0000 7f10d2ffd640 1 -- 172.21.15.2:0/4217897045 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10b801b1b0 con 0x7f10b801bae0 2024-09-19T05:43:33.515 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:33.516 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:33.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.514+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f10b8001650 msgr2=0x7f10b8003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.514+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f10b8001650 0x7f10b8003b10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.514+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f10b8004190 msgr2=0x7f10b8004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f10b8004190 0x7f10b8004640 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f10b8009370 msgr2=0x7f10b800b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f10b8009370 0x7f10b800b760 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f10b800bd20 msgr2=0x7f10b800c1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f10b800bd20 0x7f10b800c1b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f10b800d070 msgr2=0x7f10b8012740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f10b800d070 0x7f10b8012740 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f10b8014fe0 msgr2=0x7f10b8012dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f10b8014fe0 0x7f10b8012dc0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f10b8013810 msgr2=0x7f10b8013c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f10b8013810 0x7f10b8013c90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f10b801bae0 msgr2=0x7f10b8019a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f10b801bae0 0x7f10b8019a40 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f10c006c2b0 msgr2=0x7f10c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f10c006c2b0 0x7f10c006e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f10dc158850 tx=0x7f10d8008040 comp rx=0 tx=0).stop 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10dc0c7400 msgr2=0x7f10dc1421f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 --2- 172.21.15.2:0/4217897045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10dc0c7400 0x7f10dc1421f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10cc031000 tx=0x7f10cc0046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10e2c57640 1 -- 172.21.15.2:0/4217897045 reap_dead start 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10e2c57640 1 -- 172.21.15.2:0/4217897045 reap_dead start 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10e2c57640 1 -- 172.21.15.2:0/4217897045 reap_dead start 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10e2c57640 1 -- 172.21.15.2:0/4217897045 reap_dead start 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10e2c57640 1 -- 172.21.15.2:0/4217897045 reap_dead start 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 shutdown_connections 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 >> 172.21.15.2:0/4217897045 conn(0x7f10dc072a40 msgr2=0x7f10dc0c8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 shutdown_connections 2024-09-19T05:43:33.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.515+0000 7f10d0ff9640 1 -- 172.21.15.2:0/4217897045 wait complete. 2024-09-19T05:43:33.519 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:33.562 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:43:33.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:43:33.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:43:33.616 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:33.615+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:33.687 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:33.685+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:33.742 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.764411954000003, 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029411070000000001, 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974719+0000", 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1449e-05 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974732+0000", 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3788000000000001e-05 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974743+0000", 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0831e-05 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974751+0000", 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1659999999999994e-06 2024-09-19T05:43:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974913+0000", 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016199899999999999 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975022+0000", 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108857 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011757200000000001 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976884+0000", 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017442969999999999 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977555+0000", 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067127799999999998 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977604+0000", 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8068999999999997e-05 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977648+0000", 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4801e-05 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.764345437000003, 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034736569999999998, 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974776+0000", 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974783+0000", 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2940000000000002e-06 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974786+0000", 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6959999999999998e-06 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975441+0000", 2024-09-19T05:43:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065484099999999997 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975514+0000", 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2608999999999995e-05 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975640+0000", 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125869 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977680+0000", 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020402739999999999 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978154+0000", 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047386900000000003 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978206+0000", 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2250000000000003e-05 2024-09-19T05:43:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978247+0000", 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369000000000003e-05 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.764311175000003, 2024-09-19T05:43:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034986209999999999, 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8999999999999998e-07 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974818+0000", 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6870000000000006e-06 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974826+0000", 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9639999999999999e-06 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974869+0000", 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3380999999999997e-05 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975009+0000", 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013971099999999999 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131235 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976896+0000", 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001756482 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978210+0000", 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013140090000000001 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978275+0000", 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000004e-05 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978307+0000", 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899000000000001e-05 2024-09-19T05:43:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.764277579000002, 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035825869999999999, 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974846+0000", 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974856+0000", 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0701e-05 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974862+0000", 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0669999999999997e-06 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976006+0000", 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001143205 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976080+0000", 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4290999999999997e-05 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976213+0000", 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013278500000000001 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977693+0000", 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014800810000000001 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978354+0000", 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000660971 2024-09-19T05:43:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978385+0000", 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1423000000000001e-05 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978424+0000", 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9131000000000002e-05 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.764247142999999, 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037252449999999999, 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974873+0000", 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2200000000000004e-07 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974876+0000", 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.965e-06 2024-09-19T05:43:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974882+0000", 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3729999999999998e-06 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039999999999995e-06 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974947+0000", 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6505000000000002e-05 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975044+0000", 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6933000000000002e-05 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975162+0000", 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117879 2024-09-19T05:43:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977622+0000", 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024597519999999999 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978546+0000", 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092433599999999999 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978574+0000", 2024-09-19T05:43:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7923e-05 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978597+0000", 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3553e-05 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:30 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.763538762000003, 2024-09-19T05:43:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002124711, 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975581+0000", 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0470000000000001e-06 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975592+0000", 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.097e-05 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975603+0000", 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0891e-05 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975611+0000", 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4569999999999999e-06 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975662+0000", 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1424000000000001e-05 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975687+0000", 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4913999999999999e-05 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977661+0000", 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019738020000000002 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977705+0000", 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4206000000000003e-05 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:24 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.762756500000002, 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012912539999999999, 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2799999999999995e-07 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976370+0000", 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8369999999999999e-06 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976381+0000", 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0744000000000001e-05 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976388+0000", 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2320000000000004e-06 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976435+0000", 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6726e-05 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976452+0000", 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6629999999999998e-05 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977623+0000", 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001170894 2024-09-19T05:43:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977654+0000", 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1362999999999998e-05 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.762567134000001, 2024-09-19T05:43:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001404389, 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976553+0000", 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-07 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976558+0000", 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976568+0000", 2024-09-19T05:43:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976575+0000", 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976621+0000", 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6136e-05 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976640+0000", 2024-09-19T05:43:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8604e-05 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977917+0000", 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00127725 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977956+0000", 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9249000000000003e-05 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.762516853999998, 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099816800000000002, 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976603+0000", 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976610+0000", 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0210000000000002e-06 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5128e-05 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976638+0000", 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2995e-05 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976709+0000", 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0659000000000002e-05 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977601+0000", 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089194000000000003 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:25 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.762292588000001, 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001150365, 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976836+0000", 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3049999999999998e-06 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976845+0000", 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4020000000000002e-06 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976850+0000", 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6890000000000001e-06 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5437000000000004e-05 2024-09-19T05:43:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976933+0000", 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640999999999999e-05 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977941+0000", 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001008175 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6264999999999999e-05 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:16 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.761728384999998, 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000264987, 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977392+0000", 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3600000000000002e-07 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977396+0000", 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6190000000000002e-06 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977412+0000", 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5919000000000001e-05 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977420+0000", 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977543+0000", 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012284899999999999 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977550+0000", 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-19T05:43:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977656+0000", 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010567499999999999 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:17 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.761047679000001, 2024-09-19T05:43:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183666, 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978076+0000", 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.019e-06 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978089+0000", 2024-09-19T05:43:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2828e-05 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4389999999999997e-06 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978141+0000", 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5649999999999998e-05 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978149+0000", 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5820000000000004e-06 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978255+0000", 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010655299999999999 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:20 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.760984024999999, 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176971, 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978136+0000", 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2300000000000001e-07 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978139+0000", 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978148+0000", 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978155+0000", 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0169999999999999e-06 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978198+0000", 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3251999999999999e-05 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978205+0000", 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2250000000000003e-06 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978312+0000", 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010694800000000001 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:21 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.760949707000002, 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000262632, 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978173+0000", 2024-09-19T05:43:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8169999999999999e-06 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978180+0000", 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4839999999999998e-06 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978185+0000", 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978347+0000", 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016165400000000001 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978351+0000", 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1389999999999997e-06 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978432+0000", 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260999999999993e-05 2024-09-19T05:43:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 55.760666753000002, 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000148286, 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978453+0000", 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978458+0000", 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978472+0000", 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3733e-05 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978479+0000", 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8040000000000004e-06 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978539+0000", 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0038999999999999e-05 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978543+0000", 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2699999999999998e-06 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978601+0000", 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7881999999999999e-05 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.744290417000002, 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001029049, 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994831+0000", 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2299999999999998e-06 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994844+0000", 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.291e-05 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994875+0000", 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0898999999999997e-05 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994899+0000", 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3972999999999999e-05 2024-09-19T05:43:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994994+0000", 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5068000000000005e-05 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995015+0000", 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1166e-05 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995800+0000", 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078543299999999999 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995858+0000", 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000001e-05 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(2.1d 39/38 [PushReplyOp(2:b871830b:::605.00000000:head)])", 2024-09-19T05:43:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.180982724, 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019826500000000001, 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558137+0000", 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558140+0000", 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4690000000000002e-06 2024-09-19T05:43:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558149+0000", 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5329999999999994e-06 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558157+0000", 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5639999999999997e-06 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558207+0000", 2024-09-19T05:43:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9462000000000002e-05 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558335+0000", 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012793599999999999 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(2.1c 39/38 [PushReplyOp(2:3ed09add:::607.00000000:head)])", 2024-09-19T05:43:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.179202911999999, 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027980799999999999, 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559917+0000", 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8299999999999998e-07 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559920+0000", 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-19T05:43:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559929+0000", 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6829999999999997e-06 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559935+0000", 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559989+0000", 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3659999999999998e-05 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.560196+0000", 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207071 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:45 2.6 e42/19 2:654134d2:::mds0_openfiles.0:head v 42'3, mlcod=22'1)", 2024-09-19T05:43:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.739267667, 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084542299999999999, 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999853+0000", 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.857e-06 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999869+0000", 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5271e-05 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999890+0000", 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1373999999999999e-05 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999902+0000", 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1603999999999999e-05 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999981+0000", 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8935000000000002e-05 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000009+0000", 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8702999999999999e-05 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000650+0000", 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064042099999999996 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000697+0000", 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7258000000000001e-05 2024-09-19T05:43:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:33.831 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:33.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:33.837 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:33.830+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.771+0000 7fe9b8eed640 1 Processor -- start 2024-09-19T05:43:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.771+0000 7fe9b8eed640 1 -- start start 2024-09-19T05:43:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b8eed640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b8eed640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9b4155000 0x7fe9b4073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b8eed640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b40ccbc0 con 0x7fe9b40caf40 2024-09-19T05:43:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b8eed640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b40cc600 con 0x7fe9b4155000 2024-09-19T05:43:33.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b8eed640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40cc8e0 con 0x7fe9b4154c20 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b1d9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9b4155000 0x7fe9b4073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b1d9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9b4155000 0x7fe9b4073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36022/0 (socket says 172.21.15.2:36022) 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b1d9b640 1 -- 172.21.15.2:0/1105824237 learned_addr learned my addr 172.21.15.2:0/1105824237 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b259c640 1 --2- 172.21.15.2:0/1105824237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b1d9b640 1 -- 172.21.15.2:0/1105824237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 msgr2=0x7fe9b40cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b1d9b640 1 --2- 172.21.15.2:0/1105824237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b40cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b1d9b640 1 -- 172.21.15.2:0/1105824237 >> v1:172.21.15.123:6789/0 conn(0x7fe9b40caf40 legacy=0x7fe9b415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.773+0000 7fe9b1d9b640 1 -- 172.21.15.2:0/1105824237 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b40cbc60 con 0x7fe9b4155000 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.774+0000 7fe9b1d9b640 1 --2- 172.21.15.2:0/1105824237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b4073340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009850 tx=0x7fe9a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3211694363bf4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.775+0000 7fe9b159a640 1 -- 172.21.15.2:0/1105824237 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8004030 con 0x7fe9b4155000 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.775+0000 7fe9b159a640 1 -- 172.21.15.2:0/1105824237 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a80041d0 con 0x7fe9b4155000 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.775+0000 7fe9b159a640 1 -- 172.21.15.2:0/1105824237 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8003610 con 0x7fe9b4155000 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.775+0000 7fe9b8eed640 1 -- 172.21.15.2:0/1105824237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 msgr2=0x7fe9b4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.775+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/1105824237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b4073340 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8009850 tx=0x7fe9a802a230 comp rx=0 tx=0).stop 2024-09-19T05:43:33.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.776+0000 7fe9b8eed640 1 -- 172.21.15.2:0/1105824237 shutdown_connections 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.776+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/1105824237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b4073340 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.776+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/1105824237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b40cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.776+0000 7fe9b8eed640 1 -- 172.21.15.2:0/1105824237 >> 172.21.15.2:0/1105824237 conn(0x7fe9b406d0e0 msgr2=0x7fe9b406d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.776+0000 7fe9b8eed640 1 -- 172.21.15.2:0/1105824237 shutdown_connections 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.776+0000 7fe9b8eed640 1 -- 172.21.15.2:0/1105824237 wait complete. 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.778+0000 7fe9b8eed640 1 Processor -- start 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.778+0000 7fe9b8eed640 1 -- start start 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b8eed640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9b40caf40 0x7fe9b41e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b8eed640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b41e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b8eed640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b41e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b8eed640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b415b1f0 con 0x7fe9b4155000 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b8eed640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40cb430 con 0x7fe9b40caf40 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b8eed640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b415b370 con 0x7fe9b4154c20 2024-09-19T05:43:33.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b1d9b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b41e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9b40caf40 0x7fe9b41e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b1d9b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b41e1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:53824/0 (socket says 172.21.15.2:53824) 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b1d9b640 1 -- 172.21.15.2:0/2386788864 learned_addr learned my addr 172.21.15.2:0/2386788864 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.779+0000 7fe9b2d9d640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b41e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b259c640 1 -- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 msgr2=0x7fe9b41e1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b259c640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b41e1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b259c640 1 -- 172.21.15.2:0/2386788864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 msgr2=0x7fe9b41e20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b259c640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b41e20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b259c640 1 -- 172.21.15.2:0/2386788864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b41e1100 con 0x7fe9b40caf40 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b2d9d640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b41e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b259c640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9b40caf40 0x7fe9b41e5a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e880 tx=0x7fe9a400ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9a2ffd640 1 -- 172.21.15.2:0/2386788864 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4017070 con 0x7fe9b40caf40 2024-09-19T05:43:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9a2ffd640 1 -- 172.21.15.2:0/2386788864 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a400c940 con 0x7fe9b40caf40 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b4159a80 con 0x7fe9b40caf40 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9a2ffd640 1 -- 172.21.15.2:0/2386788864 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400cd30 con 0x7fe9b40caf40 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.780+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b4159d20 con 0x7fe9b40caf40 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.782+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b40c5770 con 0x7fe9b40caf40 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.782+0000 7fe9a2ffd640 1 -- 172.21.15.2:0/2386788864 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe9a401a070 con 0x7fe9b40caf40 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.783+0000 7fe9a2ffd640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe99406c2b0 0x7fe99406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.783+0000 7fe9a2ffd640 1 -- 172.21.15.2:0/2386788864 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe9a408b420 con 0x7fe9b40caf40 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.783+0000 7fe9b1d9b640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe99406c2b0 0x7fe99406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.784+0000 7fe9b1d9b640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe99406c2b0 0x7fe99406e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80090b0 tx=0x7fe9a802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:33.788+0000 7fe9a2ffd640 1 -- 172.21.15.2:0/2386788864 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe9a4059410 con 0x7fe9b40caf40 2024-09-19T05:43:33.933 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:33.932+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:33.953 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:33.953 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:33.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:34.049 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:34.045+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:34.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.084+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fe9b40b32c0 con 0x7fe9b40caf40 2024-09-19T05:43:34.134 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:34.133+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:34.135 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:34.135 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:34.135 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:34.147 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:34.235 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:34.234+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:34.336 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:34.334+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:34.359 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:34.359 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:34.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:34.359 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.476265303000005, 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:34.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:34.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.366 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.837180494999998, 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:43:34.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:43:34.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.837137966, 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.373 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:43:34.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:43:34.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.378 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.837020406999997, 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:34.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:43:34.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:43:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.807763072, 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:43:34.384 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.807706134, 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:34.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.395 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.653646402000007, 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:43:34.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:34.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.653574070999994, 2024-09-19T05:43:34.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:43:34.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.653560522000006, 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:43:34.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:43:34.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.248393621000005, 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:43:34.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.248315023999993, 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:43:34.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:43:34.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:43:34.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.248303086000007, 2024-09-19T05:43:34.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.426 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:34.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.431 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.248292006, 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:43:34.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.248217440999994, 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:34.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:43:34.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.443 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.236790901000006, 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.444 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:34.449 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 70.236728759000002, 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:43:34.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:43:34.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 69.759505336999993, 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:43:34.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:43:34.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 69.759483973000002, 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:43:34.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:43:34.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 69.759466270999994, 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:34.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.472 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 69.759412886000007, 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:43:34.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:43:34.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:34.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.478 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:34.480 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:34.480 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:34.478+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:34.581 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:34.579+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:34.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.603+0000 7fe9a2ffd640 1 -- 172.21.15.2:0/2386788864 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v46) ==== 101+0+0 (secure 0 0 0) 0x7fe9a405d710 con 0x7fe9b40caf40 2024-09-19T05:43:34.605 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:43:34.606 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:34.606 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:34.606 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:34.606 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe99406c2b0 msgr2=0x7fe99406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe99406c2b0 0x7fe99406e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80090b0 tx=0x7fe9a802e000 comp rx=0 tx=0).stop 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9b40caf40 msgr2=0x7fe9b41e5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9b40caf40 0x7fe9b41e5a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400e880 tx=0x7fe9a400ed40 comp rx=0 tx=0).stop 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 shutdown_connections 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe99406c2b0 0x7fe99406e770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b4155000 0x7fe9b41e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9b4154c20 0x7fe9b41e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.606+0000 7fe9b8eed640 1 --2- 172.21.15.2:0/2386788864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9b40caf40 0x7fe9b41e5a50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.607+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 >> 172.21.15.2:0/2386788864 conn(0x7fe9b406d0e0 msgr2=0x7fe9b4073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.607+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 shutdown_connections 2024-09-19T05:43:34.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.607+0000 7fe9b8eed640 1 -- 172.21.15.2:0/2386788864 wait complete. 2024-09-19T05:43:34.617 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:34.668 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:34.669 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:34.669 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-19T05:43:34.669 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T05:43:34.713 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:34.712+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:34.769 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:34.769 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:34.770 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:34.770 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:34.770 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:34.782 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:34.782 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:34.780+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:34.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.808+0000 7f1cc60ba640 1 Processor -- start 2024-09-19T05:43:34.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.808+0000 7f1cc60ba640 1 -- start start 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.809+0000 7f1cc60ba640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1cc00d06e0 0x7f1cc00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.809+0000 7f1cc60ba640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cc015eb50 0x7f1cc00cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.809+0000 7f1cc60ba640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cc00cb400 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.809+0000 7f1cc60ba640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f1cc00cb6e0 con 0x7f1cc00d06e0 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.809+0000 7f1cc60ba640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1cc00cc970 con 0x7f1cc015eb50 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbffff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cc015eb50 0x7f1cc00cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbffff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cc015eb50 0x7f1cc00cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42636/0 (socket says 172.21.15.2:42636) 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbffff640 1 -- 172.21.15.2:0/1219036179 learned_addr learned my addr 172.21.15.2:0/1219036179 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbf7fe640 1 --2- 172.21.15.2:0/1219036179 >> v2:172.21.15.2:3300/0 conn(0x7f1cc00d06e0 0x7f1cc00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:34.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbe7fc640 1 -- 172.21.15.2:0/1219036179 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4287914768 0 0) 0x7f1cc00cb400 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbe7fc640 1 -- 172.21.15.2:0/1219036179 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ca8003520 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbffff640 1 -- 172.21.15.2:0/1219036179 >> v2:172.21.15.2:3300/0 conn(0x7f1cc00d06e0 msgr2=0x7f1cc00cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:34.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbffff640 1 --2- 172.21.15.2:0/1219036179 >> v2:172.21.15.2:3300/0 conn(0x7f1cc00d06e0 0x7f1cc00cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbffff640 1 -- 172.21.15.2:0/1219036179 >> v1:172.21.15.2:6789/0 conn(0x7f1cc00d0ac0 legacy=0x7f1cc0154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:34.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.810+0000 7f1cbffff640 1 -- 172.21.15.2:0/1219036179 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc00ccc50 con 0x7f1cc015eb50 2024-09-19T05:43:34.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.811+0000 7f1cbf7fe640 1 --2- 172.21.15.2:0/1219036179 >> v2:172.21.15.2:3300/0 conn(0x7f1cc00d06e0 0x7f1cc00cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:34.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.811+0000 7f1cbffff640 1 --2- 172.21.15.2:0/1219036179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc015eb50 0x7f1cc00cd450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00ce390 tx=0x7f1cb802a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e53d4dc2a29c498 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:34.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.812+0000 7f1cbe7fc640 1 -- 172.21.15.2:0/1219036179 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8036070 con 0x7f1cc015eb50 2024-09-19T05:43:34.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.812+0000 7f1cbe7fc640 1 -- 172.21.15.2:0/1219036179 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb8031600 con 0x7f1cc015eb50 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.812+0000 7f1cbe7fc640 1 -- 172.21.15.2:0/1219036179 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80319b0 con 0x7f1cc015eb50 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 -- 172.21.15.2:0/1219036179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc015eb50 msgr2=0x7f1cc00cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/1219036179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc015eb50 0x7f1cc00cd450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00ce390 tx=0x7f1cb802a750 comp rx=0 tx=0).stop 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 -- 172.21.15.2:0/1219036179 shutdown_connections 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/1219036179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc015eb50 0x7f1cc00cd450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/1219036179 >> v2:172.21.15.2:3300/0 conn(0x7f1cc00d06e0 0x7f1cc00cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 -- 172.21.15.2:0/1219036179 >> 172.21.15.2:0/1219036179 conn(0x7f1cc006d740 msgr2=0x7f1cc0073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 -- 172.21.15.2:0/1219036179 shutdown_connections 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.813+0000 7f1cc60ba640 1 -- 172.21.15.2:0/1219036179 wait complete. 2024-09-19T05:43:34.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.815+0000 7f1cc60ba640 1 Processor -- start 2024-09-19T05:43:34.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.815+0000 7f1cc60ba640 1 -- start start 2024-09-19T05:43:34.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.816+0000 7f1cc60ba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc00d06e0 0x7f1cc00b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:34.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.816+0000 7f1cc60ba640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cc00d0ac0 0x7f1cc0157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cc60ba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc015eb50 0x7f1cc0158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cbeffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cc00d0ac0 0x7f1cc0157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc00d06e0 0x7f1cc00b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cbf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc00d06e0 0x7f1cc00b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42652/0 (socket says 172.21.15.2:42652) 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cbeffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cc00d0ac0 0x7f1cc0157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:53840/0 (socket says 172.21.15.2:53840) 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc015eb50 0x7f1cc0158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cbffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc015eb50 0x7f1cc0158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36046/0 (socket says 172.21.15.2:36046) 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cbf7fe640 1 -- 172.21.15.2:0/96834068 learned_addr learned my addr 172.21.15.2:0/96834068 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc00c5830 con 0x7f1cc015eb50 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.817+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc015efc0 con 0x7f1cc00d06e0 2024-09-19T05:43:34.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.818+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1cc0155200 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.818+0000 7f1cbeffd640 1 -- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc00d06e0 msgr2=0x7f1cc00b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.818+0000 7f1cbeffd640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc00d06e0 0x7f1cc00b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.818+0000 7f1cbeffd640 1 -- 172.21.15.2:0/96834068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc015eb50 msgr2=0x7f1cc0158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.818+0000 7f1cbeffd640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc015eb50 0x7f1cc0158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.818+0000 7f1cbeffd640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc00ca230 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.818+0000 7f1cbffff640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc015eb50 0x7f1cc0158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.819+0000 7f1cbeffd640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cc00d0ac0 0x7f1cc0157af0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c400 tx=0x7f1cb400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.819+0000 7f1ca7fff640 1 -- 172.21.15.2:0/96834068 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4017070 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.819+0000 7f1ca7fff640 1 -- 172.21.15.2:0/96834068 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb40052d0 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.819+0000 7f1ca7fff640 1 -- 172.21.15.2:0/96834068 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4003830 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.819+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc00ca4d0 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.819+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc01f3e10 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.821+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc00cbb40 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.822+0000 7f1ca7fff640 1 -- 172.21.15.2:0/96834068 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1cb401a070 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.822+0000 7f1ca7fff640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cac06c2b0 0x7f1cac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:34.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.822+0000 7f1ca7fff640 1 -- 172.21.15.2:0/96834068 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f1cb408a680 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.823+0000 7f1cbf7fe640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cac06c2b0 0x7f1cac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:34.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.823+0000 7f1cbf7fe640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cac06c2b0 0x7f1cac06e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009560 tx=0x7f1cb0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:34.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:34.827+0000 7f1ca7fff640 1 -- 172.21.15.2:0/96834068 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1cb4058830 con 0x7f1cc00d0ac0 2024-09-19T05:43:34.882 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:34.881+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.003615142000001, 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.982 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:43:34.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:43:34.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:43:34.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.003557579000002, 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:43:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:43:34.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:43:34.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.003485103000003, 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:34.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:43:34.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:43:34.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:43:34.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:34.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.003447844999997, 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:43:34.999 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:43:35.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:43:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:35.004 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.003408043, 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:35.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:35.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:43:35.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.002578882000002, 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:43:35.008 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:43:35.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.002153933000002, 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:35.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:43:35.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:43:35.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:35.016 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.997960945000003, 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:35.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:43:35.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:43:35.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.995379032999999, 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:43:35.020 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.994967273, 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:43:35.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.424583861999999, 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:43:35.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.107916908, 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:43:35.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:43:35.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:43:35.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.105831498000001, 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:43:35.040 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 18'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@18'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.501109347, 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084064700000000003, 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477339+0000", 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000002e-07 2024-09-19T05:43:35.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477650+0000", 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031168299999999999 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477668+0000", 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7745e-05 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477689+0000", 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0644000000000001e-05 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477763+0000", 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3919999999999997e-05 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477772+0000", 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2609999999999999e-06 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.478179+0000", 2024-09-19T05:43:35.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00040692799999999998 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 18'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@18'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.498777370999999, 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010886400000000001, 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479671+0000", 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7300000000000005e-07 2024-09-19T05:43:35.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480001+0000", 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033000999999999999 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480025+0000", 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4053000000000002e-05 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480044+0000", 2024-09-19T05:43:35.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.925e-05 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480142+0000", 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7955000000000007e-05 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480156+0000", 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3553999999999999e-05 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480759+0000", 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060294500000000002 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(1.0 40/39 [PushReplyOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head)])", 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.251273278999999, 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023152599999999999, 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727175+0000", 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5000000000000003e-07 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727179+0000", 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0620000000000002e-06 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727190+0000", 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.116e-05 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727196+0000", 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.404e-06 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727258+0000", 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1362e-05 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727406+0000", 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014798799999999999 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(1.0 40/39 [PushReplyOp(1:30306672:devicehealth::main.db.0000000000000000:head)])", 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.249303275999999, 2024-09-19T05:43:35.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204537, 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729145+0000", 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729148+0000", 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.495e-06 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729156+0000", 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3590000000000008e-06 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729165+0000", 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0390000000000005e-06 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729208+0000", 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.316e-05 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729349+0000", 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014112700000000001 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:43 2.14 2:2e2fa760:::200.00000001:head [write 4260~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.096604294, 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015875220000000001, 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881846+0000", 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0019999999999998e-06 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881862+0000", 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5673999999999999e-05 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881887+0000", 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4825999999999999e-05 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881901+0000", 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3866000000000001e-05 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881991+0000", 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0592000000000003e-05 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882055+0000", 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3559000000000006e-05 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882147+0000", 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1955999999999998e-05 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882985+0000", 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083855599999999996 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883339+0000", 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035373799999999998 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883390+0000", 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883431+0000", 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0593000000000003e-05 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:44 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.096514883999999, 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001741942, 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881937+0000", 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9990000000000002e-06 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881948+0000", 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0711e-05 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881954+0000", 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2260000000000002e-06 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882539+0000", 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000584675 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882625+0000", 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6278000000000003e-05 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882741+0000", 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115909 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883444+0000", 2024-09-19T05:43:35.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070269799999999997 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883601+0000", 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000157825 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883640+0000", 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8702000000000003e-05 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883675+0000", 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4445e-05 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:43 2.14 e42/19 ondisk, result = 0)", 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.095204769, 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019390400000000001, 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883244+0000", 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883248+0000", 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2100000000000003e-06 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883259+0000", 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0824e-05 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883275+0000", 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6415000000000001e-05 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883325+0000", 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883333+0000", 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9859999999999994e-06 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883437+0000", 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010347 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.073 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.073 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:35.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:35.074 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:35.073+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:35.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.099+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f1cc0154e50 con 0x7f1cc00d0ac0 2024-09-19T05:43:35.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.101+0000 7f1ca7fff640 1 -- 172.21.15.2:0/96834068 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v46) ==== 81+0+34152 (secure 0 0 0) 0x7f1cb405cb30 con 0x7f1cc00d0ac0 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 46, 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "896e162c-4658-4743-b70e-7ec2b5bce950", 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-19T05:42:09.734568+0000", 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-19T05:43:34.587053+0000", 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-19T05:42:20.842106+0000", 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-19T05:43:19.394170+0000", 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5799936, 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-19T05:43:35.103 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "jewel", 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-19T05:43:35.104 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:21.863894+0000", 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:35.105 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:35.106 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "18", 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:35.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.108 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.109 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.110 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_metadata", 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:30.912746+0000", 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:35.111 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:35.112 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:35.113 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:35.114 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_bias": 4, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 16, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_priority": 5 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:35.115 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": "cephfs" 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 2.25, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 2.25, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2.25, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2.25, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_data", 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:31.312255+0000", 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:35.117 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 64, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 64, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 64, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 64, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 64, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.118 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "22", 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:35.119 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:35.120 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 64, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "target_size_ratio": 0.89999997615814209 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.121 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "data": "cephfs" 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.5, 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.5, 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5, 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5, 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:35.122 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "665c2ac5-a964-4f38-8fa6-f4e95cac9272", 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.123 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.124 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.125 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.126 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.127 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/288510134", 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/288510134", 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/288510134", 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/288510134", 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a4443f63-2c47-4c6f-a2df-6765b78afcc2", 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 41, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.129 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.130 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2074594312", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2074594312", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2074594312", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2074594312", 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.133 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "61465313-1ba1-4f26-b488-6ea73981419b", 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-19T05:43:35.134 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 40, 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.137 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2644191985", 2024-09-19T05:43:35.138 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2644191985", 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2644191985", 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2644191985", 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fa9d27d5-68d4-4467-a2a5-79f767a071aa", 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.139 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.140 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.143 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/3985782072", 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/3985782072", 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/3985782072", 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/3985782072", 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "85a040ff-ad8b-44ff-b82e-a36b05d83087", 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.144 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6816", 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.145 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6817", 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6818", 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6819", 2024-09-19T05:43:35.146 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6822", 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6823", 2024-09-19T05:43:35.147 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6820", 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6821", 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:35.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6817/2654957182", 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6819/2654957182", 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6823/2654957182", 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6821/2654957182", 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "f90f8fa4-4afc-4272-89eb-5e90123155d9", 2024-09-19T05:43:35.149 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6800", 2024-09-19T05:43:35.150 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6801", 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6802", 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6803", 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6806", 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6807", 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6804", 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6805", 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6801/3690741328", 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6803/3690741328", 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6807/3690741328", 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6805/3690741328", 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "48561580-ba9a-4096-9722-16c308f414ee", 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6824", 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6825", 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6826", 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6827", 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.157 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6830", 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6831", 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.158 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6828", 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6829", 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6825/593730548", 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6827/593730548", 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6831/593730548", 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6829/593730548", 2024-09-19T05:43:35.159 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "d1ee7357-25a5-432c-b96a-e808139a1cf3", 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:35.160 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6808", 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6809", 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.161 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6810", 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6811", 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6814", 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6815", 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6812", 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6813", 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6809/1657801827", 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6811/1657801827", 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6815/1657801827", 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6813/1657801827", 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.165 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.148620+0000", 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.166 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.921396+0000", 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.980898+0000", 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:43:35.167 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.184322+0000", 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.168 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.139258+0000", 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.479047+0000", 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.432111+0000", 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:35.170 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.402960+0000", 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-19T05:43:35.171 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-19T05:43:35.172 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-19T05:43:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-19T05:43:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-19T05:43:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-19T05:43:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-19T05:43:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-19T05:43:35.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.173 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cac06c2b0 msgr2=0x7f1cac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cac06c2b0 0x7f1cac06e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009560 tx=0x7f1cb0009190 comp rx=0 tx=0).stop 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cc00d0ac0 msgr2=0x7f1cc0157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cc00d0ac0 0x7f1cc0157af0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1cb400c400 tx=0x7f1cb400c8c0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 shutdown_connections 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cac06c2b0 0x7f1cac06e770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cc015eb50 0x7f1cc0158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cc00d0ac0 0x7f1cc0157af0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 --2- 172.21.15.2:0/96834068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cc00d06e0 0x7f1cc00b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 >> 172.21.15.2:0/96834068 conn(0x7f1cc006d740 msgr2=0x7f1cc01518a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:35.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.103+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 shutdown_connections 2024-09-19T05:43:35.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.104+0000 7f1cc60ba640 1 -- 172.21.15.2:0/96834068 wait complete. 2024-09-19T05:43:35.176 INFO:tasks.thrashosds.thrasher:j is {'epoch': 46, 'fsid': '896e162c-4658-4743-b70e-7ec2b5bce950', 'created': '2024-09-19T05:42:09.734568+0000', 'modified': '2024-09-19T05:43:34.587053+0000', 'last_up_change': '2024-09-19T05:42:20.842106+0000', 'last_in_change': '2024-09-19T05:43:19.394170+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:42:21.863894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:42:30.912746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:42:31.312255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '665c2ac5-a964-4f38-8fa6-f4e95cac9272', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 288510134}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 288510134}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 288510134}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 288510134}]}, 'public_addr': '172.21.15.2:6809/288510134', 'cluster_addr': '172.21.15.2:6811/288510134', 'heartbeat_back_addr': '172.21.15.2:6815/288510134', 'heartbeat_front_addr': '172.21.15.2:6813/288510134', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a4443f63-2c47-4c6f-a2df-6765b78afcc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2074594312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2074594312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2074594312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2074594312}]}, 'public_addr': '172.21.15.2:6817/2074594312', 'cluster_addr': '172.21.15.2:6819/2074594312', 'heartbeat_back_addr': '172.21.15.2:6823/2074594312', 'heartbeat_front_addr': '172.21.15.2:6821/2074594312', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61465313-1ba1-4f26-b488-6ea73981419b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2644191985}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2644191985}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2644191985}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2644191985}]}, 'public_addr': '172.21.15.2:6825/2644191985', 'cluster_addr': '172.21.15.2:6827/2644191985', 'heartbeat_back_addr': '172.21.15.2:6831/2644191985', 'heartbeat_front_addr': '172.21.15.2:6829/2644191985', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fa9d27d5-68d4-4467-a2a5-79f767a071aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 3985782072}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 3985782072}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 3985782072}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 3985782072}]}, 'public_addr': '172.21.15.2:6801/3985782072', 'cluster_addr': '172.21.15.2:6803/3985782072', 'heartbeat_back_addr': '172.21.15.2:6807/3985782072', 'heartbeat_front_addr': '172.21.15.2:6805/3985782072', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85a040ff-ad8b-44ff-b82e-a36b05d83087', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 2654957182}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 2654957182}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 2654957182}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 2654957182}]}, 'public_addr': '172.21.15.123:6817/2654957182', 'cluster_addr': '172.21.15.123:6819/2654957182', 'heartbeat_back_addr': '172.21.15.123:6823/2654957182', 'heartbeat_front_addr': '172.21.15.123:6821/2654957182', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f90f8fa4-4afc-4272-89eb-5e90123155d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 3690741328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 3690741328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 3690741328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 3690741328}]}, 'public_addr': '172.21.15.123:6801/3690741328', 'cluster_addr': '172.21.15.123:6803/3690741328', 'heartbeat_back_addr': '172.21.15.123:6807/3690741328', 'heartbeat_front_addr': '172.21.15.123:6805/3690741328', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '48561580-ba9a-4096-9722-16c308f414ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 593730548}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 593730548}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 593730548}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 593730548}]}, 'public_addr': '172.21.15.123:6825/593730548', 'cluster_addr': '172.21.15.123:6827/593730548', 'heartbeat_back_addr': '172.21.15.123:6831/593730548', 'heartbeat_front_addr': '172.21.15.123:6829/593730548', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd1ee7357-25a5-432c-b96a-e808139a1cf3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 1657801827}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 1657801827}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 1657801827}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 1657801827}]}, 'public_addr': '172.21.15.123:6809/1657801827', 'cluster_addr': '172.21.15.123:6811/1657801827', 'heartbeat_back_addr': '172.21.15.123:6815/1657801827', 'heartbeat_front_addr': '172.21.15.123:6813/1657801827', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.148620+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.921396+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.980898+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.184322+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.139258+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.479047+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.432111+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.402960+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T05:43:35.177 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-19T05:43:35.177 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:35.173+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:35.213 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:35.213 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:35.213 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:35.213 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:35.222 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:35.306 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:35.305+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:35.375 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:35.374+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:35.381 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:35.381 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:35.381 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:35.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:35.476 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:35.475+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:35.528 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.568465365999998, 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002950785, 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975069+0000", 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975082+0000", 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3139000000000001e-05 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975096+0000", 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4250000000000001e-05 2024-09-19T05:43:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975106+0000", 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4949999999999997e-06 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975244+0000", 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013790100000000001 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:35.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122292 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975481+0000", 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011483499999999999 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977502+0000", 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020215849999999998 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977932+0000", 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000429877 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5309000000000003e-05 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978013+0000", 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5796000000000001e-05 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.551 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.568387014000002, 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033652420000000001, 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-07 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975145+0000", 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975154+0000", 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0219999999999993e-06 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975161+0000", 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9870000000000002e-06 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975720+0000", 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055893000000000002 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975804+0000", 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4025999999999998e-05 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975919+0000", 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115009 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978071+0000", 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002151563 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978433+0000", 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036205599999999998 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7867999999999997e-05 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978506+0000", 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5397e-05 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:28 2.b 2:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.568356190999999, 2024-09-19T05:43:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030203629999999999, 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2099999999999997e-07 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975175+0000", 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1109999999999999e-06 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975182+0000", 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975188+0000", 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4090000000000002e-06 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5392000000000001e-05 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112596 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975488+0000", 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013297499999999999 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977490+0000", 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020015580000000001 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978100+0000", 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060999199999999996 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978150+0000", 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0287000000000002e-05 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978192+0000", 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1635999999999999e-05 2024-09-19T05:43:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:34 2.b 2:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.568095556000003, 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028916179999999999, 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975433+0000", 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9599999999999999e-07 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975438+0000", 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975448+0000", 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0145e-05 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975455+0000", 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9149999999999999e-06 2024-09-19T05:43:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975979+0000", 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052412100000000001 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976090+0000", 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111307 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978324+0000", 2024-09-19T05:43:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022336130000000002 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.568034916000002, 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026148299999999998, 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975496+0000", 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975504+0000", 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975510+0000", 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:43:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975620+0000", 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109602 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975717+0000", 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7676999999999993e-05 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978108+0000", 2024-09-19T05:43:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023904070000000002 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:4 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.567738167000002, 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022744829999999999, 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6600000000000005e-07 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975797+0000", 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3790000000000003e-06 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975805+0000", 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975812+0000", 2024-09-19T05:43:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.029e-06 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975861+0000", 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8266999999999997e-05 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975882+0000", 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896000000000001e-05 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978029+0000", 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021471490000000001 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978064+0000", 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:5 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.566466621000004, 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001037481, 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977066+0000", 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2250000000000002e-06 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977087+0000", 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977095+0000", 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7889999999999995e-06 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977159+0000", 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977190+0000", 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1526999999999998e-05 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000881224 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978099+0000", 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6962999999999999e-05 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.565885043999998, 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046917399999999999, 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2399999999999997e-07 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977647+0000", 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977671+0000", 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3926000000000001e-05 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977679+0000", 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.593e-06 2024-09-19T05:43:35.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977789+0000", 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109969 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978112+0000", 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032325799999999999 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.565806694000003, 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029909400000000002, 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977722+0000", 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977725+0000", 2024-09-19T05:43:35.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3830000000000001e-06 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977737+0000", 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2194999999999999e-05 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977920+0000", 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017706699999999999 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977927+0000", 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978020+0000", 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3358999999999998e-05 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:28 2.b e22/19 ondisk, result = 0)", 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.565573282999999, 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024615999999999999, 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977960+0000", 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3630000000000004e-06 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977970+0000", 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977978+0000", 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0209999999999999e-06 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978088+0000", 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109578 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9630000000000001e-06 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978201+0000", 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106018 2024-09-19T05:43:35.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.b epoch 22/19 rep_tid 4 lcod 22'2)", 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.565536879, 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033492, 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:35.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977999+0000", 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8690000000000004e-06 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978007+0000", 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3929999999999999e-06 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978223+0000", 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000211051 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978326+0000", 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102874 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:25 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.565345882999999, 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033417400000000002, 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978186+0000", 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6349999999999999e-06 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978195+0000", 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7870000000000007e-06 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978367+0000", 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017261800000000001 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978420+0000", 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2339999999999997e-05 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978427+0000", 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.25e-06 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978516+0000", 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9038999999999999e-05 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:26 2.1f e22/19 2:fe07bba1:::100.00000000.inode:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.564784148999998, 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047866199999999998, 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:35.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978750+0000", 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4809999999999997e-06 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978759+0000", 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1300000000000007e-06 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978763+0000", 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3290000000000004e-06 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978801+0000", 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8127999999999998e-05 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978815+0000", 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3523000000000001e-05 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979197+0000", 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038223899999999998 2024-09-19T05:43:35.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979222+0000", 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5219999999999999e-05 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1f epoch 22/19 rep_tid 9 entries 22'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.564341724999998, 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033723899999999997, 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:35.604 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:35.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979187+0000", 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4270000000000002e-06 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979206+0000", 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313000000000001e-05 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979214+0000", 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420000000000001e-06 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979253+0000", 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.892e-05 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979523+0000", 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000270547 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.607 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:35.608 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:35.608 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:35.609 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:35.610 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:35.608+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:35.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.638+0000 7f5cd00a5640 1 Processor -- start 2024-09-19T05:43:35.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.638+0000 7f5cd00a5640 1 -- start start 2024-09-19T05:43:35.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.639+0000 7f5cd00a5640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5cc80cd8c0 0x7f5cc80c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.639+0000 7f5cd00a5640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc815ebe0 con 0x7f5cc815aa90 2024-09-19T05:43:35.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.639+0000 7f5cd00a5640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc80b2e00 con 0x7f5cc80cd4e0 2024-09-19T05:43:35.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.639+0000 7f5cd00a5640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc815eec0 con 0x7f5cc80cd8c0 2024-09-19T05:43:35.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.639+0000 7f5ccd640640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5cc815aa90 0x7f5cc815df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:39754/0 (socket says 172.21.15.2:39754) 2024-09-19T05:43:35.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.639+0000 7f5ccd640640 1 -- 172.21.15.2:0/3225732685 learned_addr learned my addr 172.21.15.2:0/3225732685 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971681363 0 0) 0x7f5cc815ebe0 con 0x7f5cc815aa90 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cb4003520 con 0x7f5cc815aa90 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621764467 0 0) 0x7f5cc80b2e00 con 0x7f5cc80cd4e0 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cc815ebe0 con 0x7f5cc80cd4e0 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5cce642640 1 --2- 172.21.15.2:0/3225732685 >> v2:172.21.15.123:3300/0 conn(0x7f5cc80cd8c0 0x7f5cc80c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1433361580 0 0) 0x7f5cc815ebe0 con 0x7f5cc80cd4e0 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 >> v2:172.21.15.123:3300/0 conn(0x7f5cc80cd8c0 msgr2=0x7f5cc80c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 --2- 172.21.15.2:0/3225732685 >> v2:172.21.15.123:3300/0 conn(0x7f5cc80cd8c0 0x7f5cc80c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 >> v1:172.21.15.2:6789/0 conn(0x7f5cc815aa90 legacy=0x7f5cc815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc80b30e0 con 0x7f5cc80cd4e0 2024-09-19T05:43:35.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f5cc4001670 con 0x7f5cc80cd4e0 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 >> v1:172.21.15.123:6789/0 conn(0x7f5cc80cd4e0 legacy=0x7f5cc80b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cb4004ba0 0x7f5cb4004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb40037a0 0x7f5cb4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb4004160 0x7f5cb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc815eec0 con 0x7f5cb4004ba0 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb400c090 con 0x7f5cb4004160 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb400c290 con 0x7f5cb40037a0 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5cce642640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb4004160 0x7f5cb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb40037a0 0x7f5cb4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.640+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cb4004ba0 0x7f5cb4004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.641+0000 7f5cce642640 1 -- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb40037a0 msgr2=0x7f5cb4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.641+0000 7f5cce642640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb40037a0 0x7f5cb4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.641+0000 7f5cce642640 1 -- 172.21.15.2:0/3225732685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cb4004ba0 msgr2=0x7f5cb4004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:35.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.641+0000 7f5cce642640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cb4004ba0 0x7f5cb4004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.641+0000 7f5cce642640 1 -- 172.21.15.2:0/3225732685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc80b30e0 con 0x7f5cb4004160 2024-09-19T05:43:35.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.641+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cb4004ba0 0x7f5cb4004f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:35.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.641+0000 7f5cce642640 1 -- 172.21.15.2:0/3225732685 reap_dead start 2024-09-19T05:43:35.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.642+0000 7f5cce642640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb4004160 0x7f5cb400b9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0007360 tx=0x7f5cb4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5f3fba1d259a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.642+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4007c40 con 0x7f5cb4004160 2024-09-19T05:43:35.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.642+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc0003bd0 con 0x7f5cb4004160 2024-09-19T05:43:35.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.642+0000 7f5ccce3f640 1 -- 172.21.15.2:0/3225732685 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0009be0 con 0x7f5cb4004160 2024-09-19T05:43:35.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.643+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb4004160 msgr2=0x7f5cb400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.643+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb4004160 0x7f5cb400b9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0007360 tx=0x7f5cb4005830 comp rx=0 tx=0).stop 2024-09-19T05:43:35.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.643+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3225732685 shutdown_connections 2024-09-19T05:43:35.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.643+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3225732685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb4004160 0x7f5cb400b9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.643+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3225732685 >> 172.21.15.2:0/3225732685 conn(0x7f5cc8067b30 msgr2=0x7f5cc8067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:35.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.643+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3225732685 shutdown_connections 2024-09-19T05:43:35.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.643+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3225732685 wait complete. 2024-09-19T05:43:35.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.645+0000 7f5cd00a5640 1 Processor -- start 2024-09-19T05:43:35.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.645+0000 7f5cd00a5640 1 -- start start 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cd00a5640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb4004160 0x7f5cc80cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cd00a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8151270 0x7f5cc81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cd00a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cc8156d30 0x7f5cc81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cd00a5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc80b3bc0 con 0x7f5cc8156d30 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cd00a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc80b36a0 con 0x7f5cc8151270 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cd00a5640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc80b3d40 con 0x7f5cb4004160 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cc8156d30 0x7f5cc81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5ccde41640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb4004160 0x7f5cc80cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cc8156d30 0x7f5cc81571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36060/0 (socket says 172.21.15.2:36060) 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 -- 172.21.15.2:0/3071749433 learned_addr learned my addr 172.21.15.2:0/3071749433 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8151270 0x7f5cc81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb4004160 msgr2=0x7f5cc80cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb4004160 0x7f5cc80cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8151270 msgr2=0x7f5cc81516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8151270 0x7f5cc81516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5cce642640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc80cfdb0 con 0x7f5cc8156d30 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.646+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5cb4004160 0x7f5cc80cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.647+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8151270 0x7f5cc81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.647+0000 7f5cce642640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cc8156d30 0x7f5cc81571e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cc002b0a0 tx=0x7f5cc0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.647+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0009be0 con 0x7f5cc8156d30 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.647+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc80d0090 con 0x7f5cc8156d30 2024-09-19T05:43:35.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.647+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc002e630 con 0x7f5cc8156d30 2024-09-19T05:43:35.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.647+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc002e9e0 con 0x7f5cc8156d30 2024-09-19T05:43:35.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.647+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc80d0650 con 0x7f5cc8156d30 2024-09-19T05:43:35.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.648+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5cc80b40d0 con 0x7f5cc8156d30 2024-09-19T05:43:35.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.649+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5cc002eb80 con 0x7f5cc8156d30 2024-09-19T05:43:35.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.649+0000 7f5cbe7fc640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5cac06c2e0 0x7f5cac06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.649+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f5cc00abd50 con 0x7f5cc8156d30 2024-09-19T05:43:35.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.649+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v46) ==== 54+0+16 (secure 0 0 0) 0x7f5cc00af020 con 0x7f5cc8156d30 2024-09-19T05:43:35.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.649+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5cac06c2e0 0x7f5cac06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.650+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5cc81f1940 0x7f5cc81f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.650+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5cc8158e40 con 0x7f5cc81f1940 2024-09-19T05:43:35.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.650+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5cc81f1940 0x7f5cc81f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.650+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5cac06c2e0 0x7f5cac06e7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40017e0 tx=0x7f5cc4001490 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.650+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5cc81f1940 0x7f5cc81f1d20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.651+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc8158e40 con 0x7f5cc81f1940 2024-09-19T05:43:35.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.666+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc80b6020 con 0x7f5cc81f1940 2024-09-19T05:43:35.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.667+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc80b6020 con 0x7f5cc81f1940 2024-09-19T05:43:35.669 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.668+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5cc81f3a30 0x7f5cc81f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.668+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5cc81f4350 con 0x7f5cc81f3a30 2024-09-19T05:43:35.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.668+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5cc81f3a30 0x7f5cc81f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.668+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5cc81f3a30 0x7f5cc81f3e10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.669+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc81f4350 con 0x7f5cc81f3a30 2024-09-19T05:43:35.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.698+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc81f14b0 con 0x7f5cc81f3a30 2024-09-19T05:43:35.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.700+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc81f14b0 con 0x7f5cc81f3a30 2024-09-19T05:43:35.702 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:35.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.701+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5cc8062cb0 0x7f5cc80630e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.701+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5cc8200880 con 0x7f5cc8062cb0 2024-09-19T05:43:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.701+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5cc8062cb0 0x7f5cc80630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.701+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5cc8062cb0 0x7f5cc80630e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.702+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc8200880 con 0x7f5cc8062cb0 2024-09-19T05:43:35.709 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:35.708+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:35.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.716+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc81599e0 con 0x7f5cc8062cb0 2024-09-19T05:43:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.718+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc81599e0 con 0x7f5cc8062cb0 2024-09-19T05:43:35.719 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.718+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5cc802c640 0x7f5cc802ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.718+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5cc815a4c0 con 0x7f5cc802c640 2024-09-19T05:43:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.718+0000 7f5cce642640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5cc802c640 0x7f5cc802ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.719+0000 7f5cce642640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5cc802c640 0x7f5cc802ca40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.719+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc815a4c0 con 0x7f5cc802c640 2024-09-19T05:43:35.721 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:35.721 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:35.721 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:35.721 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:35.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.747+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc802d250 con 0x7f5cc802c640 2024-09-19T05:43:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.749+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc802d250 con 0x7f5cc802c640 2024-09-19T05:43:35.750 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.749+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5cc802d850 0x7f5cc8204a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.749+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5cc815a670 con 0x7f5cc802d850 2024-09-19T05:43:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.749+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5cc802d850 0x7f5cc8204a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.750+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5cc802d850 0x7f5cc8204a50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.750+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc815a670 con 0x7f5cc802d850 2024-09-19T05:43:35.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.764+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc815a670 con 0x7f5cc802d850 2024-09-19T05:43:35.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.766+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc815a670 con 0x7f5cc802d850 2024-09-19T05:43:35.767 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:35.768 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:35.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.766+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5cc82054a0 0x7f5cc8207890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.766+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5cc815a670 con 0x7f5cc82054a0 2024-09-19T05:43:35.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.767+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5cc82054a0 0x7f5cc8207890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.767+0000 7f5ccd640640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5cc82054a0 0x7f5cc8207890 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.768+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc815a670 con 0x7f5cc82054a0 2024-09-19T05:43:35.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.796+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc815a670 con 0x7f5cc82054a0 2024-09-19T05:43:35.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.798+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc815a670 con 0x7f5cc82054a0 2024-09-19T05:43:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:35.799 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:35.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.798+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5cc82082e0 0x7f5cc820a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.798+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5cc815a670 con 0x7f5cc82082e0 2024-09-19T05:43:35.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.798+0000 7f5cce642640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5cc82082e0 0x7f5cc820a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.799+0000 7f5cce642640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5cc82082e0 0x7f5cc820a6d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.799+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc815a670 con 0x7f5cc82082e0 2024-09-19T05:43:35.810 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:35.809+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:35.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.827+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc815a670 con 0x7f5cc82082e0 2024-09-19T05:43:35.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.828+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc815a670 con 0x7f5cc82082e0 2024-09-19T05:43:35.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:35.830 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:35.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.829+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5cc820b120 0x7f5cc820d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:35.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.829+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f5cc82102e0 con 0x7f5cc820b120 2024-09-19T05:43:35.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.829+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5cc820b120 0x7f5cc820d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:35.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.829+0000 7f5ccde41640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5cc820b120 0x7f5cc820d510 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:35.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.830+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc82102e0 con 0x7f5cc820b120 2024-09-19T05:43:35.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.857+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5cc820da50 con 0x7f5cc820b120 2024-09-19T05:43:35.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cbe7fc640 1 -- 172.21.15.2:0/3071749433 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5cc820da50 con 0x7f5cc820b120 2024-09-19T05:43:35.860 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5cc81f1940 msgr2=0x7f5cc81f1d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5cc81f1940 0x7f5cc81f1d20 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5cc81f3a30 msgr2=0x7f5cc81f3e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5cc81f3a30 0x7f5cc81f3e10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5cc8062cb0 msgr2=0x7f5cc80630e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5cc8062cb0 0x7f5cc80630e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cce642640 1 -- 172.21.15.2:0/3071749433 reap_dead start 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5cc802c640 msgr2=0x7f5cc802ca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5cc802c640 0x7f5cc802ca40 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5cc802d850 msgr2=0x7f5cc8204a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5cc802d850 0x7f5cc8204a50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5cc82054a0 msgr2=0x7f5cc8207890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5cc82054a0 0x7f5cc8207890 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5cc82082e0 msgr2=0x7f5cc820a6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.859+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5cc82082e0 0x7f5cc820a6d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5cc820b120 msgr2=0x7f5cc820d510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5cc820b120 0x7f5cc820d510 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cce642640 1 -- 172.21.15.2:0/3071749433 reap_dead start 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5cac06c2e0 msgr2=0x7f5cac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5cac06c2e0 0x7f5cac06e7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40017e0 tx=0x7f5cc4001490 comp rx=0 tx=0).stop 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cc8156d30 msgr2=0x7f5cc81571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cc8156d30 0x7f5cc81571e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cc002b0a0 tx=0x7f5cc0004ab0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 shutdown_connections 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5cac06c2e0 0x7f5cac06e7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 --2- 172.21.15.2:0/3071749433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5cc8156d30 0x7f5cc81571e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 >> 172.21.15.2:0/3071749433 conn(0x7f5cc8067b30 msgr2=0x7f5cc80cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 shutdown_connections 2024-09-19T05:43:35.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:35.860+0000 7f5cd00a5640 1 -- 172.21.15.2:0/3071749433 wait complete. 2024-09-19T05:43:35.873 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:35.874 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:35.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:35.916 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:35.915+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:35.985 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:35.985 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:35.994 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:36.037 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:36.036+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:36.096 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:36.096 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:36.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:36.112 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:36.111+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:36.213 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:36.212+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:36.247 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:36.247 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:36.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:36.314 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:36.312+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:36.359 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:36.360 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:36.368 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:36.414 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:36.413+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:36.511 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:36.511 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:36.515 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:36.513+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:36.519 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:36.616 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:36.615+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:36.625 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:43:36.664 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:36.664 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:36.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:36.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.733+0000 7f96d301e640 1 Processor -- start 2024-09-19T05:43:36.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.733+0000 7f96d301e640 1 -- start start 2024-09-19T05:43:36.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.733+0000 7f96d301e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96cc0cd4f0 0x7f96cc06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.733+0000 7f96d301e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc15b360 0x7f96cc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.733+0000 7f96d301e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96cc0701e0 con 0x7f96cc0706e0 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.733+0000 7f96d301e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f96cc070fb0 con 0x7f96cc0cd4f0 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.733+0000 7f96d301e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc06caf0 con 0x7f96cc15b360 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96cbfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f96cc0706e0 0x7f96cc15ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:39762/0 (socket says 172.21.15.2:39762) 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96cbfff640 1 -- 172.21.15.2:0/4186836237 learned_addr learned my addr 172.21.15.2:0/4186836237 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96d15bb640 1 --2- 172.21.15.2:0/4186836237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc15b360 0x7f96cc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96d0dba640 1 --2- 172.21.15.2:0/4186836237 >> v2:172.21.15.123:3300/0 conn(0x7f96cc0cd4f0 0x7f96cc06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96cb7fe640 1 -- 172.21.15.2:0/4186836237 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270114847 0 0) 0x7f96cc0701e0 con 0x7f96cc0706e0 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96cb7fe640 1 -- 172.21.15.2:0/4186836237 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96bc003520 con 0x7f96cc0706e0 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96d15bb640 1 -- 172.21.15.2:0/4186836237 >> v2:172.21.15.123:3300/0 conn(0x7f96cc0cd4f0 msgr2=0x7f96cc06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96d15bb640 1 --2- 172.21.15.2:0/4186836237 >> v2:172.21.15.123:3300/0 conn(0x7f96cc0cd4f0 0x7f96cc06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:36.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96d15bb640 1 -- 172.21.15.2:0/4186836237 >> v1:172.21.15.2:6789/0 conn(0x7f96cc0706e0 legacy=0x7f96cc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:36.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.734+0000 7f96d15bb640 1 -- 172.21.15.2:0/4186836237 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc06fc20 con 0x7f96cc15b360 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.735+0000 7f96d15bb640 1 --2- 172.21.15.2:0/4186836237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc15b360 0x7f96cc0c9110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96cc071130 tx=0x7f96c402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3967c95cb4dfdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.735+0000 7f96cb7fe640 1 -- 172.21.15.2:0/4186836237 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c402ae60 con 0x7f96cc15b360 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.735+0000 7f96cb7fe640 1 -- 172.21.15.2:0/4186836237 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c4007740 con 0x7f96cc15b360 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.735+0000 7f96cb7fe640 1 -- 172.21.15.2:0/4186836237 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4007b10 con 0x7f96cc15b360 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 -- 172.21.15.2:0/4186836237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc15b360 msgr2=0x7f96cc0c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 --2- 172.21.15.2:0/4186836237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc15b360 0x7f96cc0c9110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96cc071130 tx=0x7f96c402a640 comp rx=0 tx=0).stop 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 -- 172.21.15.2:0/4186836237 shutdown_connections 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 --2- 172.21.15.2:0/4186836237 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc15b360 0x7f96cc0c9110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 --2- 172.21.15.2:0/4186836237 >> v2:172.21.15.123:3300/0 conn(0x7f96cc0cd4f0 0x7f96cc06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 -- 172.21.15.2:0/4186836237 >> 172.21.15.2:0/4186836237 conn(0x7f96cc067950 msgr2=0x7f96cc067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 -- 172.21.15.2:0/4186836237 shutdown_connections 2024-09-19T05:43:36.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.736+0000 7f96d301e640 1 -- 172.21.15.2:0/4186836237 wait complete. 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.737+0000 7f96d301e640 1 Processor -- start 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.737+0000 7f96d301e640 1 -- start start 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d301e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc0706e0 0x7f96cc14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d301e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96cc0cd4f0 0x7f96cc1f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d301e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc14cfc0 0x7f96cc0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d301e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc0cd8d0 con 0x7f96cc0cd4f0 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d301e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc070c40 con 0x7f96cc0706e0 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d301e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc0cda50 con 0x7f96cc14cfc0 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96cc0cd4f0 0x7f96cc1f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d0dba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc0706e0 0x7f96cc14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d0dba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc0706e0 0x7f96cc14d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42702/0 (socket says 172.21.15.2:42702) 2024-09-19T05:43:36.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d0dba640 1 -- 172.21.15.2:0/3750666830 learned_addr learned my addr 172.21.15.2:0/3750666830 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.738+0000 7f96d15bb640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc14cfc0 0x7f96cc0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96d15bb640 1 -- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc0706e0 msgr2=0x7f96cc14d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96d15bb640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc0706e0 0x7f96cc14d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96d15bb640 1 -- 172.21.15.2:0/3750666830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96cc0cd4f0 msgr2=0x7f96cc1f0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96d15bb640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96cc0cd4f0 0x7f96cc1f0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96d15bb640 1 -- 172.21.15.2:0/3750666830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc14d460 con 0x7f96cc14cfc0 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96cbfff640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96cc0cd4f0 0x7f96cc1f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96d15bb640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc14cfc0 0x7f96cc0c6fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96c4031000 tx=0x7f96c4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96c97fa640 1 -- 172.21.15.2:0/3750666830 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c403f070 con 0x7f96cc14cfc0 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96c97fa640 1 -- 172.21.15.2:0/3750666830 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c40319c0 con 0x7f96cc14cfc0 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.739+0000 7f96c97fa640 1 -- 172.21.15.2:0/3750666830 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4031b70 con 0x7f96cc14cfc0 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.740+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc157c30 con 0x7f96cc14cfc0 2024-09-19T05:43:36.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.740+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc14e4c0 con 0x7f96cc14cfc0 2024-09-19T05:43:36.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.741+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc06c480 con 0x7f96cc14cfc0 2024-09-19T05:43:36.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.742+0000 7f96c97fa640 1 -- 172.21.15.2:0/3750666830 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f96c4031d10 con 0x7f96cc14cfc0 2024-09-19T05:43:36.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.742+0000 7f96c97fa640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96b006c2b0 0x7f96b006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:36.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.743+0000 7f96c97fa640 1 -- 172.21.15.2:0/3750666830 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f96c40ac840 con 0x7f96cc14cfc0 2024-09-19T05:43:36.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.743+0000 7f96d0dba640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96b006c2b0 0x7f96b006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:36.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.743+0000 7f96d0dba640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96b006c2b0 0x7f96b006e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009c00 tx=0x7f96c0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:36.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:36.747+0000 7f96c97fa640 1 -- 172.21.15.2:0/3750666830 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f96c407a960 con 0x7f96cc14cfc0 2024-09-19T05:43:36.756 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:36.755+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:36.817 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:36.818 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:36.818 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:36.817+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:36.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:36.918 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:36.917+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:36.970 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:36.971 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:36.979 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:37.019 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:37.019+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:37.120 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:37.119+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:37.122 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:37.122 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:37.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.123+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f96cc0ca660 con 0x7f96cc14cfc0 2024-09-19T05:43:37.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:37.221 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:37.219+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:37.234 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:37.234 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:37.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:37.322 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:37.321+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:37.411 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:37.411 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:37.423 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:37.424 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:37.422+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:37.524 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:37.522+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:37.548 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:37.548 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:37.560 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.602+0000 7f96c97fa640 1 -- 172.21.15.2:0/3750666830 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v47) ==== 87+0+0 (secure 0 0 0) 0x7f96c407ec60 con 0x7f96cc14cfc0 2024-09-19T05:43:37.604 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96b006c2b0 msgr2=0x7f96b006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96b006c2b0 0x7f96b006e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96c0009c00 tx=0x7f96c0009190 comp rx=0 tx=0).stop 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc14cfc0 msgr2=0x7f96cc0c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc14cfc0 0x7f96cc0c6fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96c4031000 tx=0x7f96c4004a40 comp rx=0 tx=0).stop 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 shutdown_connections 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96b006c2b0 0x7f96b006e770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96cc14cfc0 0x7f96cc0c6fc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96cc0cd4f0 0x7f96cc1f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 --2- 172.21.15.2:0/3750666830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc0706e0 0x7f96cc14d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.605+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 >> 172.21.15.2:0/3750666830 conn(0x7f96cc067950 msgr2=0x7f96cc150fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:37.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.606+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 shutdown_connections 2024-09-19T05:43:37.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:37.606+0000 7f96d301e640 1 -- 172.21.15.2:0/3750666830 wait complete. 2024-09-19T05:43:37.624 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:37.623+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:37.708 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:37.708 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:37.721 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:37.725 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:37.724+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:37.826 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:37.825+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:37.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:37.879 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:37.879 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:37.892 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:37.927 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:37.925+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:38.029 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:38.028+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.032+0000 7fe2aa296640 1 Processor -- start 2024-09-19T05:43:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.032+0000 7fe2aa296640 1 -- start start 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.033+0000 7fe2aa296640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe2a40c39d0 0x7fe2a415fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.033+0000 7fe2aa296640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe2a4072760 0x7fe2a4162f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.033+0000 7fe2aa296640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a40c1e90 con 0x7fe2a40c2b10 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.033+0000 7fe2aa296640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a40c2170 con 0x7fe2a4072760 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.033+0000 7fe2aa296640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a40c2450 con 0x7fe2a40c39d0 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.034+0000 7fe2a8833640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe2a40c2b10 0x7fe2a4072c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:39768/0 (socket says 172.21.15.2:39768) 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.034+0000 7fe2a8833640 1 -- 172.21.15.2:0/3588176153 learned_addr learned my addr 172.21.15.2:0/3588176153 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.034+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/3588176153 >> v2:172.21.15.123:3300/0 conn(0x7fe2a40c39d0 0x7fe2a415fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.034+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3500568944 0 0) 0x7fe2a40c1e90 con 0x7fe2a40c2b10 2024-09-19T05:43:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.034+0000 7fe29bfff640 1 --2- 172.21.15.2:0/3588176153 >> v2:172.21.15.2:3300/0 conn(0x7fe2a4072760 0x7fe2a4162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.034+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe290003520 con 0x7fe2a40c2b10 2024-09-19T05:43:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1155970370 0 0) 0x7fe290003520 con 0x7fe2a40c2b10 2024-09-19T05:43:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 >> v2:172.21.15.123:3300/0 conn(0x7fe2a40c39d0 msgr2=0x7fe2a415fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 --2- 172.21.15.2:0/3588176153 >> v2:172.21.15.123:3300/0 conn(0x7fe2a40c39d0 0x7fe2a415fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 >> v2:172.21.15.2:3300/0 conn(0x7fe2a4072760 msgr2=0x7fe2a4162f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 --2- 172.21.15.2:0/3588176153 >> v2:172.21.15.2:3300/0 conn(0x7fe2a4072760 0x7fe2a4162f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a415f0b0 con 0x7fe2a40c2b10 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7fe29c0019f0 con 0x7fe2a40c2b10 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe29bfff640 1 --2- 172.21.15.2:0/3588176153 >> v2:172.21.15.2:3300/0 conn(0x7fe2a4072760 0x7fe2a4162f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 >> v1:172.21.15.2:6789/0 conn(0x7fe2a40c2b10 legacy=0x7fe2a4072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe290003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.035+0000 7fe2a37fe640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2900043b0 0x7fe290004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.036+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe290003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.036+0000 7fe2a37fe640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe290005080 0x7fe29000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.036+0000 7fe29bfff640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2900043b0 0x7fe290004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.036+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a40c2170 con 0x7fe2900039b0 2024-09-19T05:43:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.036+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a40c2450 con 0x7fe2900043b0 2024-09-19T05:43:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.036+0000 7fe2a8833640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe290005080 0x7fe29000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.036+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe29000c060 con 0x7fe290005080 2024-09-19T05:43:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.037+0000 7fe2a3fff640 1 -- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe290005080 msgr2=0x7fe29000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.037+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe290005080 0x7fe29000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.037+0000 7fe2a3fff640 1 -- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2900043b0 msgr2=0x7fe290004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.037+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2900043b0 0x7fe290004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.037+0000 7fe2a3fff640 1 -- 172.21.15.2:0/3588176153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe290003520 con 0x7fe2900039b0 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.037+0000 7fe2a8833640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe290005080 0x7fe29000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.037+0000 7fe2a8833640 1 -- 172.21.15.2:0/3588176153 reap_dead start 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.038+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe290003d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe294009ac0 tx=0x7fe290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd673bd7338c2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.038+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294036070 con 0x7fe2900039b0 2024-09-19T05:43:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.038+0000 7fe2a37fe640 1 -- 172.21.15.2:0/3588176153 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe294031400 con 0x7fe2900039b0 2024-09-19T05:43:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.039+0000 7fe2aa296640 1 -- 172.21.15.2:0/3588176153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 msgr2=0x7fe290003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.039+0000 7fe2aa296640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe290003d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe294009ac0 tx=0x7fe290005830 comp rx=0 tx=0).stop 2024-09-19T05:43:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.039+0000 7fe2aa296640 1 -- 172.21.15.2:0/3588176153 shutdown_connections 2024-09-19T05:43:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.039+0000 7fe2aa296640 1 --2- 172.21.15.2:0/3588176153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe290003d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.039+0000 7fe2aa296640 1 -- 172.21.15.2:0/3588176153 >> 172.21.15.2:0/3588176153 conn(0x7fe2a40c9630 msgr2=0x7fe2a40c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.040+0000 7fe2aa296640 1 -- 172.21.15.2:0/3588176153 shutdown_connections 2024-09-19T05:43:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.040+0000 7fe2aa296640 1 -- 172.21.15.2:0/3588176153 wait complete. 2024-09-19T05:43:38.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.041+0000 7fe2aa296640 1 Processor -- start 2024-09-19T05:43:38.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.042+0000 7fe2aa296640 1 -- start start 2024-09-19T05:43:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2aa296640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe2a41f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2aa296640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2a4159bf0 0x7fe2a415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe2a41f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2a3fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe2a41f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36100/0 (socket says 172.21.15.2:36100) 2024-09-19T05:43:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2aa296640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2a415b800 0x7fe2a41f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2aa296640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a40c2f20 con 0x7fe2900039b0 2024-09-19T05:43:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2aa296640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a415f670 con 0x7fe2a4159bf0 2024-09-19T05:43:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2aa296640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a415fb90 con 0x7fe2a415b800 2024-09-19T05:43:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe2a3fff640 1 -- 172.21.15.2:0/1044679888 learned_addr learned my addr 172.21.15.2:0/1044679888 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:38.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.043+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2a4159bf0 0x7fe2a415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a3fff640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2a415b800 msgr2=0x7fe2a41f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe2a415b800 0x7fe2a41f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a3fff640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2a4159bf0 msgr2=0x7fe2a415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2a4159bf0 0x7fe2a415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a3fff640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a415e440 con 0x7fe2900039b0 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2a4159bf0 0x7fe2a415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe2a41f52f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe29402abb0 tx=0x7fe294004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.047 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294040020 con 0x7fe2900039b0 2024-09-19T05:43:38.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2940308d0 con 0x7fe2900039b0 2024-09-19T05:43:38.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.044+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294036040 con 0x7fe2900039b0 2024-09-19T05:43:38.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.045+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a415e6b0 con 0x7fe2900039b0 2024-09-19T05:43:38.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.045+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a415ec70 con 0x7fe2900039b0 2024-09-19T05:43:38.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.047+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe294043020 con 0x7fe2900039b0 2024-09-19T05:43:38.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.047+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe2a415bc20 con 0x7fe2900039b0 2024-09-19T05:43:38.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.048+0000 7fe2a17fa640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe28406c2b0 0x7fe28406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.048+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fe2940aba20 con 0x7fe2900039b0 2024-09-19T05:43:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.048+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v47) ==== 54+0+16 (secure 0 0 0) 0x7fe29404a030 con 0x7fe2900039b0 2024-09-19T05:43:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.049+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe28406c2b0 0x7fe28406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.049+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe2a415a030 0x7fe2a41f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.049+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe28406c2b0 0x7fe28406e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2a406c4f0 tx=0x7fe28c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.049+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2a415ee40 con 0x7fe2a415a030 2024-09-19T05:43:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.049+0000 7fe2a8833640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe2a415a030 0x7fe2a41f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.050+0000 7fe2a8833640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe2a415a030 0x7fe2a41f8320 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.051+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a415ee40 con 0x7fe2a415a030 2024-09-19T05:43:38.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.080+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a40c28f0 con 0x7fe2a415a030 2024-09-19T05:43:38.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.083+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a40c28f0 con 0x7fe2a415a030 2024-09-19T05:43:38.085 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:38.086 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:38.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.083+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe2a4162cd0 0x7fe2a41580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.083+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe2a4070210 con 0x7fe2a4162cd0 2024-09-19T05:43:38.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.083+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe2a4162cd0 0x7fe2a41580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.084+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe2a4162cd0 0x7fe2a41580f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.085+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a4070210 con 0x7fe2a4162cd0 2024-09-19T05:43:38.087 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:38.087 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:38.097 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:38.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.105+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a40cc770 con 0x7fe2a4162cd0 2024-09-19T05:43:38.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.107+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a40cc770 con 0x7fe2a4162cd0 2024-09-19T05:43:38.109 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:38.109 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:38.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.108+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe2a4158b40 0x7fe2a4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.108+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe2a4158f20 con 0x7fe2a4158b40 2024-09-19T05:43:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.108+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe2a4158b40 0x7fe2a4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.109+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe2a4158b40 0x7fe2a4062630 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.110+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a4158f20 con 0x7fe2a4158b40 2024-09-19T05:43:38.128 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:38.127+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:38.139 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.138+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a4158f20 con 0x7fe2a4158b40 2024-09-19T05:43:38.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.141+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a4158f20 con 0x7fe2a4158b40 2024-09-19T05:43:38.142 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:38.143 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:38.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.141+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe2a41e82e0 0x7fe2a41e8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.142+0000 7fe2a8833640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe2a41e82e0 0x7fe2a41e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.142+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe2a4158f20 con 0x7fe2a41e82e0 2024-09-19T05:43:38.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.143+0000 7fe2a8833640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe2a41e82e0 0x7fe2a41e8720 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.144+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a4158f20 con 0x7fe2a41e82e0 2024-09-19T05:43:38.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.171+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a4158f20 con 0x7fe2a41e82e0 2024-09-19T05:43:38.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.174+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a4158f20 con 0x7fe2a41e82e0 2024-09-19T05:43:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:38.176 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:38.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.174+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe2a402c640 0x7fe2a402ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.175+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe2a41e9170 con 0x7fe2a402c640 2024-09-19T05:43:38.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.175+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe2a402c640 0x7fe2a402ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.175+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe2a402c640 0x7fe2a402ca80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.176+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a41e9170 con 0x7fe2a402c640 2024-09-19T05:43:38.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.203+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a41e9170 con 0x7fe2a402c640 2024-09-19T05:43:38.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.205+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a41e9170 con 0x7fe2a402c640 2024-09-19T05:43:38.206 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:38.207 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:38.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.205+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe2a402d4d0 0x7fe2a420b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.206+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe2a41e9270 con 0x7fe2a402d4d0 2024-09-19T05:43:38.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.206+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe2a402d4d0 0x7fe2a420b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.206+0000 7fe29bfff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe2a402d4d0 0x7fe2a420b3a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.207+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a41e9270 con 0x7fe2a402d4d0 2024-09-19T05:43:38.229 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:38.228+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:38.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.235+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a402d970 con 0x7fe2a402d4d0 2024-09-19T05:43:38.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.237+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a402d970 con 0x7fe2a402d4d0 2024-09-19T05:43:38.238 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.237+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe2a402d910 0x7fe2a420de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.237+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe2a4210e80 con 0x7fe2a402d910 2024-09-19T05:43:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.237+0000 7fe2a8833640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe2a402d910 0x7fe2a420de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.238+0000 7fe2a8833640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe2a402d910 0x7fe2a420de00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.239+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a4210e80 con 0x7fe2a402d910 2024-09-19T05:43:38.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.266+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a420fba0 con 0x7fe2a402d910 2024-09-19T05:43:38.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.268+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a420fba0 con 0x7fe2a402d910 2024-09-19T05:43:38.269 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:38.270 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:38.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.269+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe2a42102c0 0x7fe2a4210700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:38.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.269+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe2a4214250 con 0x7fe2a42102c0 2024-09-19T05:43:38.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.269+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe2a42102c0 0x7fe2a4210700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:38.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.269+0000 7fe2a3fff640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe2a42102c0 0x7fe2a4210700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:38.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.270+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a4214250 con 0x7fe2a42102c0 2024-09-19T05:43:38.289 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.406592399999994, 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.767507592000001, 2024-09-19T05:43:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:43:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:43:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.767465063000003, 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.767347504, 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:43:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:43:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.738090169000003, 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.738033231000003, 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:43:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:43:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:43:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.583973498999995, 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:43:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:43:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:43:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.583901167999997, 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:43:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.583887618999995, 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:43:38.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.178720717999994, 2024-09-19T05:43:38.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.178642120999996, 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:43:38.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.178630182999996, 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:43:38.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:43:38.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:43:38.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.178619103000003, 2024-09-19T05:43:38.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:43:38.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:43:38.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.178544537999997, 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.167117997999995, 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:43:38.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:43:38.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 74.167055856000005, 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 73.689832433999996, 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:38.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:38.386 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 73.689811070000005, 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:43:38.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 73.689793367999997, 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:43:38.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:43:38.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 73.689739982999996, 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:38.398 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:43:38.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:43:38.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.404 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.404 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.297+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe2a42130d0 con 0x7fe2a42102c0 2024-09-19T05:43:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.299+0000 7fe2a17fa640 1 -- 172.21.15.2:0/1044679888 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe2a42130d0 con 0x7fe2a42102c0 2024-09-19T05:43:38.405 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:38.405 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe2a415a030 msgr2=0x7fe2a41f8320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe2a415a030 0x7fe2a41f8320 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe2a4162cd0 msgr2=0x7fe2a41580f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe2a4162cd0 0x7fe2a41580f0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe2a4158b40 msgr2=0x7fe2a4062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe2a4158b40 0x7fe2a4062630 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe2a41e82e0 msgr2=0x7fe2a41e8720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe2a41e82e0 0x7fe2a41e8720 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe2a402c640 msgr2=0x7fe2a402ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe2a402c640 0x7fe2a402ca80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe2a402d4d0 msgr2=0x7fe2a420b3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fe2a402d4d0 0x7fe2a420b3a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe2a402d910 msgr2=0x7fe2a420de00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe2a402d910 0x7fe2a420de00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe2a42102c0 msgr2=0x7fe2a4210700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe2a42102c0 0x7fe2a4210700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2a8833640 1 -- 172.21.15.2:0/1044679888 reap_dead start 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2a8833640 1 -- 172.21.15.2:0/1044679888 reap_dead start 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2a8833640 1 -- 172.21.15.2:0/1044679888 reap_dead start 2024-09-19T05:43:38.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2a8833640 1 -- 172.21.15.2:0/1044679888 reap_dead start 2024-09-19T05:43:38.408 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.300+0000 7fe2a8833640 1 -- 172.21.15.2:0/1044679888 reap_dead start 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe28406c2b0 msgr2=0x7fe28406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe28406c2b0 0x7fe28406e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe2a406c4f0 tx=0x7fe28c008040 comp rx=0 tx=0).stop 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 msgr2=0x7fe2a41f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe2a41f52f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe29402abb0 tx=0x7fe294004730 comp rx=0 tx=0).stop 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 shutdown_connections 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe28406c2b0 0x7fe28406e770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 --2- 172.21.15.2:0/1044679888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe2900039b0 0x7fe2a41f52f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 >> 172.21.15.2:0/1044679888 conn(0x7fe2a40c9630 msgr2=0x7fe2a406ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 shutdown_connections 2024-09-19T05:43:38.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:38.301+0000 7fe2aa296640 1 -- 172.21.15.2:0/1044679888 wait complete. 2024-09-19T05:43:38.410 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:38.404+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:38.506 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:38.505+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:38.562 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:38.562 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:38.571 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:38.607 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:38.605+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:38.707 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:38.706+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:38.723 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:38.723 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:38.723 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:38.723 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:38.723 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:38.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:38.809 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:38.808+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.867399726000002, 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057488729999999998, 2024-09-19T05:43:38.845 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974832+0000", 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974836+0000", 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0920000000000002e-06 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974847+0000", 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1939999999999999e-05 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974858+0000", 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0909e-05 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974867+0000", 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8170000000000005e-06 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974988+0000", 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012085 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975118+0000", 2024-09-19T05:43:38.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012971200000000001 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012547200000000001 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977324+0000", 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020808390000000001 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980518+0000", 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003194119 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980553+0000", 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4425000000000003e-05 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980581+0000", 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8697999999999999e-05 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.867342163000004, 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005780449, 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:38.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974891+0000", 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0100000000000005e-07 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974894+0000", 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.676e-06 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974904+0000", 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4409999999999999e-06 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-06 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975505+0000", 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059498000000000005 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975574+0000", 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8924000000000006e-05 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975692+0000", 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118628 2024-09-19T05:43:38.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023193419999999998 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980613+0000", 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002601187 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980646+0000", 2024-09-19T05:43:38.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3198000000000003e-05 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980670+0000", 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4212000000000001e-05 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:38.856 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.867269686999997, 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058968400000000004, 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974962+0000", 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974963+0000", 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-07 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974971+0000", 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8949999999999998e-06 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5089999999999996e-06 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974989+0000", 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.029e-05 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975895+0000", 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090619599999999995 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976014+0000", 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119403 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976128+0000", 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113409 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978481+0000", 2024-09-19T05:43:38.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023529559999999998 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980809+0000", 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023287809999999998 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5664e-05 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980859+0000", 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-05 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.867232428999998, 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061632800000000001, 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.862 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975000+0000", 2024-09-19T05:43:38.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975011+0000", 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0311e-05 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975019+0000", 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0669999999999992e-06 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975024+0000", 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976431+0000", 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014063179999999999 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976555+0000", 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123857 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976663+0000", 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108027 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978487+0000", 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018248369999999999 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981113+0000", 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026259199999999999 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981142+0000", 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8663999999999999e-05 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.981163+0000", 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1004000000000001e-05 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.867192627000001, 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082495889999999999, 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975040+0000", 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975042+0000", 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.356e-06 2024-09-19T05:43:38.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975055+0000", 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976977+0000", 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001921208 2024-09-19T05:43:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977073+0000", 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6780999999999994e-05 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983289+0000", 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062156800000000003 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.866363466000003, 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020181579999999999, 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975869+0000", 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975870+0000", 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1909999999999999e-06 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:38.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8680000000000001e-05 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975899+0000", 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0577999999999999e-05 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975907+0000", 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975982+0000", 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5468000000000003e-05 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976005+0000", 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2736999999999999e-05 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977844+0000", 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018389120000000001 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977887+0000", 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2802000000000001e-05 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 4 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.865938517000004, 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001608595, 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:38.876 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976294+0000", 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0200000000000001e-07 2024-09-19T05:43:38.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976302+0000", 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976320+0000", 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7861000000000001e-05 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976343+0000", 2024-09-19T05:43:38.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3397e-05 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976432+0000", 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8480999999999995e-05 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977902+0000", 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014703330000000001 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.861745528999997, 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000189815, 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:38.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980487+0000", 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6400000000000001e-07 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980488+0000", 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.612e-06 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980493+0000", 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4519999999999999e-06 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980495+0000", 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980603+0000", 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107431 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980608+0000", 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5509999999999999e-06 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.980676+0000", 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8050000000000001e-05 2024-09-19T05:43:38.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 4 lcod 22'5)", 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.859163617, 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226212, 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983069+0000", 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983070+0000", 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3370000000000001e-06 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983077+0000", 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.046e-06 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983097+0000", 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0377000000000002e-05 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983110+0000", 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2843000000000001e-05 2024-09-19T05:43:38.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983184+0000", 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4115000000000004e-05 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983295+0000", 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110494 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.858751857000001, 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048318980000000003, 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983480+0000", 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983481+0000", 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983486+0000", 2024-09-19T05:43:38.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983496+0000", 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6909999999999992e-06 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983502+0000", 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983544+0000", 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1566999999999997e-05 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983587+0000", 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3662999999999999e-05 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.983676+0000", 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8398000000000003e-05 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.984177+0000", 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050093900000000005 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988207+0000", 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040303159999999999 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988257+0000", 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9685999999999998e-05 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.988312+0000", 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609000000000002e-05 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~256 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 52.288368446, 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019086660000000001, 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553864+0000", 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553866+0000", 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553876+0000", 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553895+0000", 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8301e-05 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553906+0000", 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0772e-05 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.553993+0000", 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7175000000000005e-05 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554103+0000", 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110129 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.554274+0000", 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017096799999999999 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555045+0000", 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000771304 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555678+0000", 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00063277500000000003 2024-09-19T05:43:38.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555730+0000", 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2346999999999999e-05 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:46.555772+0000", 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2126000000000003e-05 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1196~3064 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.971701492000001, 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023214440000000002, 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870531+0000", 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870534+0000", 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.089e-06 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870549+0000", 2024-09-19T05:43:38.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5319e-05 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870582+0000", 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2845000000000001e-05 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870596+0000", 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4095e-05 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870680+0000", 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3948000000000003e-05 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870789+0000", 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108793 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.870939+0000", 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150106 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.871942+0000", 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001003653 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872729+0000", 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078618600000000002 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872792+0000", 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3126999999999994e-05 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872852+0000", 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0282999999999999e-05 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:41 2.14 e22/19 ondisk, result = 0)", 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.969616082000002, 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024422300000000001, 2024-09-19T05:43:38.904 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872616+0000", 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872617+0000", 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3179999999999999e-06 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872624+0000", 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.251e-06 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872639+0000", 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5585000000000001e-05 2024-09-19T05:43:38.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872650+0000", 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.04e-05 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872711+0000", 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872721+0000", 2024-09-19T05:43:38.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6309999999999998e-06 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:48.872860+0000", 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013940900000000001 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 18'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@18'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:38.908 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.364893931000001, 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084064700000000003, 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477338+0000", 2024-09-19T05:43:38.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477339+0000", 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000002e-07 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477650+0000", 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031168299999999999 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477668+0000", 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7745e-05 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477689+0000", 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0644000000000001e-05 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477763+0000", 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3919999999999997e-05 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.477772+0000", 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2609999999999999e-06 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.478179+0000", 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00040692799999999998 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(1.0 33/32 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 18'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@18'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.362561955, 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010886400000000001, 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479670+0000", 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.479671+0000", 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7300000000000005e-07 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480001+0000", 2024-09-19T05:43:38.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033000999999999999 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480025+0000", 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4053000000000002e-05 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480044+0000", 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.925e-05 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480142+0000", 2024-09-19T05:43:38.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7955000000000007e-05 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480156+0000", 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3553999999999999e-05 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:10.480759+0000", 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060294500000000002 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(1.0 40/39 [PushReplyOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head)])", 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.115057863000001, 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023152599999999999, 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727174+0000", 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727175+0000", 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5000000000000003e-07 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727179+0000", 2024-09-19T05:43:38.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0620000000000002e-06 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727190+0000", 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.116e-05 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727196+0000", 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.404e-06 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727258+0000", 2024-09-19T05:43:38.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1362e-05 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.727406+0000", 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014798799999999999 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(1.0 40/39 [PushReplyOp(1:30306672:devicehealth::main.db.0000000000000000:head)])", 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.11308786, 2024-09-19T05:43:38.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204537, 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729144+0000", 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729145+0000", 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5699999999999998e-07 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729148+0000", 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.495e-06 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729156+0000", 2024-09-19T05:43:38.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3590000000000008e-06 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729165+0000", 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0390000000000005e-06 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729208+0000", 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.316e-05 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:20.729349+0000", 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014112700000000001 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:43 2.14 2:2e2fa760:::200.00000001:head [write 4260~1730 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.960388878, 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015875220000000001, 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.924 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881843+0000", 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881846+0000", 2024-09-19T05:43:38.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0019999999999998e-06 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881862+0000", 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5673999999999999e-05 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881887+0000", 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4825999999999999e-05 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881901+0000", 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3866000000000001e-05 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881991+0000", 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0592000000000003e-05 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882055+0000", 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3559000000000006e-05 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882147+0000", 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1955999999999998e-05 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882985+0000", 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083855599999999996 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883339+0000", 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035373799999999998 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883390+0000", 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883431+0000", 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0593000000000003e-05 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.929 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:44 2.14 2:292cf221:::200.00000000:head [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e42)", 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.960299468000001, 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001741942, 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881933+0000", 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881937+0000", 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9990000000000002e-06 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881948+0000", 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0711e-05 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.881954+0000", 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2260000000000002e-06 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882539+0000", 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000584675 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882625+0000", 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6278000000000003e-05 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.882741+0000", 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115909 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883444+0000", 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070269799999999997 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883601+0000", 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000157825 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883640+0000", 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8702000000000003e-05 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883675+0000", 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4445e-05 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:43 2.14 e42/19 ondisk, result = 0)", 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.958989353, 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019390400000000001, 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883243+0000", 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883244+0000", 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3699999999999999e-07 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883248+0000", 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2100000000000003e-06 2024-09-19T05:43:38.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883259+0000", 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0824e-05 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883275+0000", 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6415000000000001e-05 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883325+0000", 2024-09-19T05:43:38.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0161999999999998e-05 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883333+0000", 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9859999999999994e-06 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.883437+0000", 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010347 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:38.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:38.939 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:38.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:38.941 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:38.939+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:39.040 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:39.039+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:39.051 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:39.051 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:39.051 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:39.051 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:39.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:39.142 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:39.140+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:39.176 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:39.177 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:39.177 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:39.177 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:39.177 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:39.187 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:39.242 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:39.240+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:39.326 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:39.326 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.349513565999999, 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002950785, 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975069+0000", 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-19T05:43:39.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975082+0000", 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3139000000000001e-05 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975096+0000", 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4250000000000001e-05 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975106+0000", 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4949999999999997e-06 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975244+0000", 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013790100000000001 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122292 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975481+0000", 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011483499999999999 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977502+0000", 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020215849999999998 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977932+0000", 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000429877 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5309000000000003e-05 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978013+0000", 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5796000000000001e-05 2024-09-19T05:43:39.332 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.349435214000003, 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033652420000000001, 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-07 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975145+0000", 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975154+0000", 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0219999999999993e-06 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975161+0000", 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9870000000000002e-06 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975720+0000", 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055893000000000002 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975804+0000", 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4025999999999998e-05 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975919+0000", 2024-09-19T05:43:39.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115009 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978071+0000", 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002151563 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978433+0000", 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036205599999999998 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:39.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7867999999999997e-05 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978506+0000", 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5397e-05 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:28 2.b 2:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.349404391, 2024-09-19T05:43:39.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030203629999999999, 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2099999999999997e-07 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975175+0000", 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1109999999999999e-06 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975182+0000", 2024-09-19T05:43:39.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975188+0000", 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4090000000000002e-06 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5392000000000001e-05 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112596 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975488+0000", 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013297499999999999 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977490+0000", 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020015580000000001 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978100+0000", 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060999199999999996 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978150+0000", 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0287000000000002e-05 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978192+0000", 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1635999999999999e-05 2024-09-19T05:43:39.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:34 2.b 2:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.349143755999997, 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028916179999999999, 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975433+0000", 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9599999999999999e-07 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975438+0000", 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975448+0000", 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0145e-05 2024-09-19T05:43:39.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975455+0000", 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9149999999999999e-06 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975979+0000", 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052412100000000001 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976090+0000", 2024-09-19T05:43:39.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111307 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978324+0000", 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022336130000000002 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.349083116000003, 2024-09-19T05:43:39.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026148299999999998, 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975496+0000", 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975504+0000", 2024-09-19T05:43:39.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975510+0000", 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975620+0000", 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109602 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975717+0000", 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7676999999999993e-05 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978108+0000", 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023904070000000002 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:4 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:39.352 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.348786367000002, 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022744829999999999, 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6600000000000005e-07 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975797+0000", 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3790000000000003e-06 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975805+0000", 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975812+0000", 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.029e-06 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975861+0000", 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8266999999999997e-05 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975882+0000", 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896000000000001e-05 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978029+0000", 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021471490000000001 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978064+0000", 2024-09-19T05:43:39.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:5 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.347514820999997, 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001037481, 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:43:39.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977066+0000", 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2250000000000002e-06 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977087+0000", 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977095+0000", 2024-09-19T05:43:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7889999999999995e-06 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977159+0000", 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977190+0000", 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1526999999999998e-05 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000881224 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978099+0000", 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6962999999999999e-05 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.361 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.346933243999999, 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046917399999999999, 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2399999999999997e-07 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977647+0000", 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977671+0000", 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3926000000000001e-05 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977679+0000", 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.593e-06 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977789+0000", 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109969 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978112+0000", 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032325799999999999 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.346854894000003, 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029909400000000002, 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:39.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977722+0000", 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977725+0000", 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3830000000000001e-06 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977737+0000", 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2194999999999999e-05 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-19T05:43:39.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977920+0000", 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017706699999999999 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977927+0000", 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978020+0000", 2024-09-19T05:43:39.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3358999999999998e-05 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:28 2.b e22/19 ondisk, result = 0)", 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.346621483, 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024615999999999999, 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:39.369 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-19T05:43:39.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977960+0000", 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3630000000000004e-06 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977970+0000", 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977978+0000", 2024-09-19T05:43:39.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0209999999999999e-06 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978088+0000", 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109578 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9630000000000001e-06 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978201+0000", 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106018 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.b epoch 22/19 rep_tid 4 lcod 22'2)", 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.346585079, 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033492, 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977999+0000", 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8690000000000004e-06 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978007+0000", 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3929999999999999e-06 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978223+0000", 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000211051 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978326+0000", 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102874 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.376 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:25 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.346394083, 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033417400000000002, 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978186+0000", 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6349999999999999e-06 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978195+0000", 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7870000000000007e-06 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978367+0000", 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017261800000000001 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978420+0000", 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2339999999999997e-05 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978427+0000", 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.25e-06 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978516+0000", 2024-09-19T05:43:39.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9038999999999999e-05 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:26 2.1f e22/19 2:fe07bba1:::100.00000000.inode:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.345832348999998, 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047866199999999998, 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:39.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978750+0000", 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4809999999999997e-06 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978759+0000", 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1300000000000007e-06 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978763+0000", 2024-09-19T05:43:39.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3290000000000004e-06 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978801+0000", 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8127999999999998e-05 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978815+0000", 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3523000000000001e-05 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:39.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979197+0000", 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038223899999999998 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979222+0000", 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5219999999999999e-05 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1f epoch 22/19 rep_tid 9 entries 22'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:39.385 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.345389924999999, 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033723899999999997, 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979187+0000", 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4270000000000002e-06 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979206+0000", 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313000000000001e-05 2024-09-19T05:43:39.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979214+0000", 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420000000000001e-06 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979253+0000", 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.892e-05 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979523+0000", 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000270547 2024-09-19T05:43:39.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.389 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:39.389 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:39.389 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:39.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:39.391 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:39.388+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:39.491 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:39.489+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:39.509 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:39.509 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:39.509 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:39.509 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:39.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:39.592 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:39.590+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:39.625 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:43:39.625 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:39.626 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:39.626 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:39.626 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:39.626 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:39.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:39.693 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:39.691+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:39.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.730+0000 7f105d21e640 1 Processor -- start 2024-09-19T05:43:39.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.731+0000 7f105d21e640 1 -- start start 2024-09-19T05:43:39.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.731+0000 7f105d21e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10580680f0 0x7f10580b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:39.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.731+0000 7f105d21e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10580b4930 con 0x7f105815cb90 2024-09-19T05:43:39.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.731+0000 7f105d21e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10580b4650 con 0x7f1058067d10 2024-09-19T05:43:39.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105d21e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f10580c90c0 con 0x7f10580680f0 2024-09-19T05:43:39.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105659c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10580680f0 0x7f10580b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:39.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105759e640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f105815cb90 0x7f1058152660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:39784/0 (socket says 172.21.15.2:39784) 2024-09-19T05:43:39.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105659c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10580680f0 0x7f10580b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42736/0 (socket says 172.21.15.2:42736) 2024-09-19T05:43:39.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105759e640 1 -- 172.21.15.2:0/3003220483 learned_addr learned my addr 172.21.15.2:0/3003220483 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:39.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f1056d9d640 1 --1- 172.21.15.2:0/3003220483 >> v1:172.21.15.123:6789/0 conn(0x7f1058067d10 0x7f105815c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:33738/0 (socket says 172.21.15.2:33738) 2024-09-19T05:43:39.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105659c640 1 -- 172.21.15.2:0/3003220483 >> v1:172.21.15.123:6789/0 conn(0x7f1058067d10 legacy=0x7f105815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:39.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105659c640 1 -- 172.21.15.2:0/3003220483 >> v1:172.21.15.2:6789/0 conn(0x7f105815cb90 legacy=0x7f1058152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:39.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f1055d9b640 1 -- 172.21.15.2:0/3003220483 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559795825 0 0) 0x7f10580b4930 con 0x7f105815cb90 2024-09-19T05:43:39.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.732+0000 7f105659c640 1 -- 172.21.15.2:0/3003220483 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10580c93a0 con 0x7f10580680f0 2024-09-19T05:43:39.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.733+0000 7f105659c640 1 --2- 172.21.15.2:0/3003220483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f10580b3a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097a0 tx=0x7f104c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ae83b885dcda25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:39.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.733+0000 7f1055d9b640 1 -- 172.21.15.2:0/3003220483 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c037070 con 0x7f10580680f0 2024-09-19T05:43:39.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.733+0000 7f1055d9b640 1 -- 172.21.15.2:0/3003220483 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f104c032440 con 0x7f10580680f0 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.733+0000 7f1055d9b640 1 -- 172.21.15.2:0/3003220483 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c032810 con 0x7f10580680f0 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.734+0000 7f105d21e640 1 -- 172.21.15.2:0/3003220483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 msgr2=0x7f10580b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.734+0000 7f105d21e640 1 --2- 172.21.15.2:0/3003220483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f10580b3a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f104c0097a0 tx=0x7f104c02a230 comp rx=0 tx=0).stop 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.734+0000 7f105d21e640 1 -- 172.21.15.2:0/3003220483 shutdown_connections 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.734+0000 7f105d21e640 1 --2- 172.21.15.2:0/3003220483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f10580b3a90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.734+0000 7f105d21e640 1 -- 172.21.15.2:0/3003220483 >> 172.21.15.2:0/3003220483 conn(0x7f1058070fc0 msgr2=0x7f105806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.734+0000 7f105d21e640 1 -- 172.21.15.2:0/3003220483 shutdown_connections 2024-09-19T05:43:39.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.734+0000 7f105d21e640 1 -- 172.21.15.2:0/3003220483 wait complete. 2024-09-19T05:43:39.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 Processor -- start 2024-09-19T05:43:39.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 -- start start 2024-09-19T05:43:39.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1058067d10 0x7f105815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f1058151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105815cb90 0x7f10581f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f10580b41e0 con 0x7f105815cb90 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10580b4060 con 0x7f10580680f0 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.736+0000 7f105d21e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f10580b4360 con 0x7f1058067d10 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105815cb90 0x7f10581f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f1058151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f1056d9d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1058067d10 0x7f105815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f1056d9d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1058067d10 0x7f105815ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:53890/0 (socket says 172.21.15.2:53890) 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105815cb90 0x7f10581f0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36106/0 (socket says 172.21.15.2:36106) 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f1058151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42746/0 (socket says 172.21.15.2:42746) 2024-09-19T05:43:39.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f1056d9d640 1 -- 172.21.15.2:0/1729610439 learned_addr learned my addr 172.21.15.2:0/1729610439 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 -- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1058067d10 msgr2=0x7f105815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1058067d10 0x7f105815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 -- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 msgr2=0x7f1058151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f1058151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 -- 172.21.15.2:0/1729610439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058153e00 con 0x7f105815cb90 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f1056d9d640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1058067d10 0x7f105815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105659c640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f1058151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f105759e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105815cb90 0x7f10581f0520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f105000c700 tx=0x7f105000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.737+0000 7f103f7fe640 1 -- 172.21.15.2:0/1729610439 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050018070 con 0x7f105815cb90 2024-09-19T05:43:39.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.738+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10581f0040 con 0x7f105815cb90 2024-09-19T05:43:39.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.738+0000 7f103f7fe640 1 -- 172.21.15.2:0/1729610439 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105000ce10 con 0x7f105815cb90 2024-09-19T05:43:39.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.738+0000 7f103f7fe640 1 -- 172.21.15.2:0/1729610439 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105000a880 con 0x7f105815cb90 2024-09-19T05:43:39.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.738+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1058155740 con 0x7f105815cb90 2024-09-19T05:43:39.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.739+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10581f0210 con 0x7f105815cb90 2024-09-19T05:43:39.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.739+0000 7f103f7fe640 1 -- 172.21.15.2:0/1729610439 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f105000aa20 con 0x7f105815cb90 2024-09-19T05:43:39.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.740+0000 7f103f7fe640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f104406c2e0 0x7f104406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:39.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.740+0000 7f103f7fe640 1 -- 172.21.15.2:0/1729610439 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f105008a8d0 con 0x7f105815cb90 2024-09-19T05:43:39.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.740+0000 7f1056d9d640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f104406c2e0 0x7f104406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:39.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.741+0000 7f1056d9d640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f104406c2e0 0x7f104406e7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1048000f60 tx=0x7f1048007f00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:39.744 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:39.744 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:39.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.762113726999999, 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:43:39.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:43:39.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.761238069999997, 2024-09-19T05:43:39.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:43:39.755 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760613890000002, 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:43:39.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:43:39.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:39.760 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760493152000002, 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:43:39.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760453863000002, 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.765 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:43:39.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:43:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760389363000002, 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:43:39.769 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:43:39.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.774 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760325944999998, 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:43:39.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:43:39.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.76027629, 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:43:39.780 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:43:39.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:43:39.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:43:39.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:43:39.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:39.787 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760220121000003, 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:39.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:43:39.792 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760193129999998, 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:43:39.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.798 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760114682999998, 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:43:39.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:43:39.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760081002, 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:39.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:43:39.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.809 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760057990999996, 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:43:39.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:43:39.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.760024596000001, 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.815 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:43:39.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:43:39.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.759981916999998, 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:43:39.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:43:39.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.824 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.759942752000001, 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.826 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:43:39.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:43:39.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.759910724999997, 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:39.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.833 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.759712192000002, 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:39.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.759641014000003, 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:39.840 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:39.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.843 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 61.753424856000002, 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:39.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:39.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:43:39.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:39.848 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:39.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.745+0000 7f103f7fe640 1 -- 172.21.15.2:0/1729610439 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f10500588e0 con 0x7f105815cb90 2024-09-19T05:43:39.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:39.850 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:39.848+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:39.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:39.934+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f105815c790 con 0x7f105815cb90 2024-09-19T05:43:39.950 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:39.949+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:39.994 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:39.995 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:39.995 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:39.995 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:40.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:40.051 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:40.049+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:40.146 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:40.146 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:40.146 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:40.146 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:40.146 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:40.152 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:40.151+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:40.154 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:40.178 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:40.178 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:40.178 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-19T05:43:40.178 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T05:43:40.178 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T05:43:40.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-19T05:43:40.254 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:40.252+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:40.298 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:40.299 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T05:43:40.299 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:40.300 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:40.300 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:40.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:40.300 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:40.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.300 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.779525148000005, 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:43:40.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:43:40.305 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.779167180000002, 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:40.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:43:40.309 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.778666788999999, 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.312 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:43:40.314 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.771680556000007, 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:43:40.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:43:40.318 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.771222748, 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:43:40.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:43:40.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.748509127999995, 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:43:40.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:43:40.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:43:40.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.748014182999995, 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:43:40.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:43:40.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:43:40.332 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.186211227000001, 2024-09-19T05:43:40.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:43:40.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:43:40.337 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.185453451000001, 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:43:40.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:43:40.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.184872404000004, 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:43:40.342 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:43:40.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.184237358000004, 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.347 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:43:40.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:43:40.350 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.351 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.178430852000005, 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.352 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.353 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.354 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.355 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:43:40.356 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 76.177774679999999, 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.357 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.358 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:43:40.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 75.701226005999999, 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:43:40.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 75.700857692, 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:43:40.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:43:40.369 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 75.700478782000005, 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.371 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:43:40.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 75.699781928999997, 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:43:40.374 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:43:40.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:43:40.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 75.693997315000004, 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:43:40.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:43:40.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.322882301, 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.383 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.384 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:43:40.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:40.387 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 62.322839217999999, 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:40.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:43:40.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:43:40.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:40.391 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:40.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:40.398 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:40.390+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:40.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:40.493 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:40.491+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:40.500 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:40.501 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:40.501 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:40.501 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:40.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:40.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.557+0000 7f96e75c4640 1 Processor -- start 2024-09-19T05:43:40.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.557+0000 7f96e75c4640 1 -- start start 2024-09-19T05:43:40.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e75c4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e00cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e75c4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f96e0158570 0x7f96e00cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e75c4640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96e00b0c70 con 0x7f96e00ceac0 2024-09-19T05:43:40.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e75c4640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f96e00b0f40 con 0x7f96e0158570 2024-09-19T05:43:40.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e75c4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96e00b1220 con 0x7f96e00ce6e0 2024-09-19T05:43:40.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e5b61640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f96e0158570 0x7f96e00cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e5b61640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f96e0158570 0x7f96e00cba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:36114/0 (socket says 172.21.15.2:36114) 2024-09-19T05:43:40.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e5b61640 1 -- 172.21.15.2:0/2945247451 learned_addr learned my addr 172.21.15.2:0/2945247451 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.559+0000 7f96e5360640 1 --2- 172.21.15.2:0/2945247451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e00cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.560+0000 7f96e5b61640 1 -- 172.21.15.2:0/2945247451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 msgr2=0x7f96e00cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.560+0000 7f96e5b61640 1 --2- 172.21.15.2:0/2945247451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e00cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.560+0000 7f96e5b61640 1 -- 172.21.15.2:0/2945247451 >> v1:172.21.15.123:6789/0 conn(0x7f96e00ceac0 legacy=0x7f96e0157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.560+0000 7f96e5b61640 1 -- 172.21.15.2:0/2945247451 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e006cdc0 con 0x7f96e0158570 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.561+0000 7f96e5b61640 1 --2- 172.21.15.2:0/2945247451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e0158570 0x7f96e00cba70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e50 tx=0x7f96d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd48ea35a72bb3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.561+0000 7f96d7fff640 1 -- 172.21.15.2:0/2945247451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8036070 con 0x7f96e0158570 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.561+0000 7f96d7fff640 1 -- 172.21.15.2:0/2945247451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d8031600 con 0x7f96e0158570 2024-09-19T05:43:40.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.561+0000 7f96d7fff640 1 -- 172.21.15.2:0/2945247451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d80319d0 con 0x7f96e0158570 2024-09-19T05:43:40.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 -- 172.21.15.2:0/2945247451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e0158570 msgr2=0x7f96e00cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 --2- 172.21.15.2:0/2945247451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e0158570 0x7f96e00cba70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e50 tx=0x7f96d802a680 comp rx=0 tx=0).stop 2024-09-19T05:43:40.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 -- 172.21.15.2:0/2945247451 shutdown_connections 2024-09-19T05:43:40.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 --2- 172.21.15.2:0/2945247451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e0158570 0x7f96e00cba70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 --2- 172.21.15.2:0/2945247451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e00cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 -- 172.21.15.2:0/2945247451 >> 172.21.15.2:0/2945247451 conn(0x7f96e0067950 msgr2=0x7f96e0067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:40.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 -- 172.21.15.2:0/2945247451 shutdown_connections 2024-09-19T05:43:40.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.562+0000 7f96e75c4640 1 -- 172.21.15.2:0/2945247451 wait complete. 2024-09-19T05:43:40.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.564+0000 7f96e75c4640 1 Processor -- start 2024-09-19T05:43:40.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.564+0000 7f96e75c4640 1 -- start start 2024-09-19T05:43:40.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e75c4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e01e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e75c4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e00ceac0 0x7f96e01e4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e75c4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e0158570 0x7f96e01e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e75c4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96e006d380 con 0x7f96e00ceac0 2024-09-19T05:43:40.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e75c4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96e006f780 con 0x7f96e0158570 2024-09-19T05:43:40.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e75c4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96e006d500 con 0x7f96e00ce6e0 2024-09-19T05:43:40.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e5b61640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e0158570 0x7f96e01e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e5360640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e01e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e5b61640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e0158570 0x7f96e01e15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:42756/0 (socket says 172.21.15.2:42756) 2024-09-19T05:43:40.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 learned_addr learned my addr 172.21.15.2:0/1338576824 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e5360640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e01e5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:53910/0 (socket says 172.21.15.2:53910) 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.565+0000 7f96e4b5f640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e00ceac0 0x7f96e01e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 msgr2=0x7f96e01e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e5b61640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e01e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e00ceac0 msgr2=0x7f96e01e4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e5b61640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96e00ceac0 0x7f96e01e4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e00cbde0 con 0x7f96e0158570 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e00ce6e0 0x7f96e01e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e5b61640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e0158570 0x7f96e01e15e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e50 tx=0x7f96d8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8041020 con 0x7f96e0158570 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d80077a0 con 0x7f96e0158570 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8036040 con 0x7f96e0158570 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.566+0000 7f96e75c4640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e01e0bc0 con 0x7f96e0158570 2024-09-19T05:43:40.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.567+0000 7f96e75c4640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e01f1690 con 0x7f96e0158570 2024-09-19T05:43:40.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.568+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f96bc005270 con 0x7f96e0158570 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.568+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f96d8044020 con 0x7f96e0158570 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.569+0000 7f96d5ffb640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96c4074ac0 0x7f96c4076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.569+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f96d80abec0 con 0x7f96e0158570 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.569+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96c4074ac0 0x7f96c4076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.569+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v47) ==== 54+0+16 (secure 0 0 0) 0x7f96d804b030 con 0x7f96e0158570 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.570+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96bc000fa0 0x7f96bc0045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.570+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96bc005550 con 0x7f96bc000fa0 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.570+0000 7f96e4b5f640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96bc000fa0 0x7f96bc0045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.570+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96c4074ac0 0x7f96c4076f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009f30 tx=0x7f96dc009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.570+0000 7f96e4b5f640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96bc000fa0 0x7f96bc0045e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.571+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc005550 con 0x7f96bc000fa0 2024-09-19T05:43:40.594 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:40.592+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:40.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.600+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc004fe0 con 0x7f96bc000fa0 2024-09-19T05:43:40.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.603+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc004fe0 con 0x7f96bc000fa0 2024-09-19T05:43:40.605 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:40.606 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:40.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.603+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96bc001d00 0x7f96bc002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.603+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f96bc00b390 con 0x7f96bc001d00 2024-09-19T05:43:40.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.604+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96bc001d00 0x7f96bc002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.605+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96bc001d00 0x7f96bc002990 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.606+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc00b390 con 0x7f96bc001d00 2024-09-19T05:43:40.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.608+0000 7f103f7fe640 1 -- 172.21.15.2:0/1729610439 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v48) ==== 97+0+0 (secure 0 0 0) 0x7f105005cbe0 con 0x7f105815cb90 2024-09-19T05:43:40.610 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f104406c2e0 msgr2=0x7f104406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f104406c2e0 0x7f104406e7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1048000f60 tx=0x7f1048007f00 comp rx=0 tx=0).stop 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105815cb90 msgr2=0x7f10581f0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105815cb90 0x7f10581f0520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f105000c700 tx=0x7f105000cbc0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 shutdown_connections 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f104406c2e0 0x7f104406e7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10580680f0 0x7f1058151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f105815cb90 0x7f10581f0520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 --2- 172.21.15.2:0/1729610439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1058067d10 0x7f105815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.611+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 >> 172.21.15.2:0/1729610439 conn(0x7f1058070fc0 msgr2=0x7f10580cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:40.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.612+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 shutdown_connections 2024-09-19T05:43:40.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.612+0000 7f105d21e640 1 -- 172.21.15.2:0/1729610439 wait complete. 2024-09-19T05:43:40.615 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:40.615 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:40.615 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:40.615 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:40.615 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:40.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:40.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.635+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc0031a0 con 0x7f96bc001d00 2024-09-19T05:43:40.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.636+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc0031a0 con 0x7f96bc001d00 2024-09-19T05:43:40.639 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:40.639 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:40.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.637+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96bc003c20 0x7f96bc009c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.637+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f96bc00a420 con 0x7f96bc003c20 2024-09-19T05:43:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.637+0000 7f96e4b5f640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96bc003c20 0x7f96bc009c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.637+0000 7f96e4b5f640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96bc003c20 0x7f96bc009c80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.638+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc00a420 con 0x7f96bc003c20 2024-09-19T05:43:40.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.658+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc00a990 con 0x7f96bc003c20 2024-09-19T05:43:40.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.660+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc00a990 con 0x7f96bc003c20 2024-09-19T05:43:40.663 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:40.663 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:40.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.661+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96bc00af00 0x7f96bc00fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.661+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f96bc012a20 con 0x7f96bc00af00 2024-09-19T05:43:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.661+0000 7f96e5b61640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96bc00af00 0x7f96bc00fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.661+0000 7f96e5b61640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96bc00af00 0x7f96bc00fa70 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.662+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc012a20 con 0x7f96bc00af00 2024-09-19T05:43:40.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.676+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc0117c0 con 0x7f96bc00af00 2024-09-19T05:43:40.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.679+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc0117c0 con 0x7f96bc00af00 2024-09-19T05:43:40.681 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.679+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96bc012320 0x7f96bc010000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.679+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f96bc016380 con 0x7f96bc012320 2024-09-19T05:43:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.680+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96bc012320 0x7f96bc010000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.680+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96bc012320 0x7f96bc010000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.681+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc016380 con 0x7f96bc012320 2024-09-19T05:43:40.694 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:40.693+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:40.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.695+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc010540 con 0x7f96bc012320 2024-09-19T05:43:40.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.696+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc010540 con 0x7f96bc012320 2024-09-19T05:43:40.699 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:40.699 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:40.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.697+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96bc010cb0 0x7f96bc011110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.697+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f96bc019630 con 0x7f96bc010cb0 2024-09-19T05:43:40.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.697+0000 7f96e4b5f640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96bc010cb0 0x7f96bc011110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.697+0000 7f96e4b5f640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96bc010cb0 0x7f96bc011110 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.700 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.698+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc019630 con 0x7f96bc010cb0 2024-09-19T05:43:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.712+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc016d20 con 0x7f96bc010cb0 2024-09-19T05:43:40.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.714+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc016d20 con 0x7f96bc010cb0 2024-09-19T05:43:40.716 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.714+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96bc017440 0x7f96bc017880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.714+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f96bc01ca20 con 0x7f96bc017440 2024-09-19T05:43:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.714+0000 7f96e5b61640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96bc017440 0x7f96bc017880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.715+0000 7f96e5b61640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96bc017440 0x7f96bc017880 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.715+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc01ca20 con 0x7f96bc017440 2024-09-19T05:43:40.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.729+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc017ed0 con 0x7f96bc017440 2024-09-19T05:43:40.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.731+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc017ed0 con 0x7f96bc017440 2024-09-19T05:43:40.734 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:40.734 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:40.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.731+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96bc018550 0x7f96bc0189d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:40.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.732+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f96bc0190e0 con 0x7f96bc018550 2024-09-19T05:43:40.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.732+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96bc018550 0x7f96bc0189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:40.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.732+0000 7f96e5360640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96bc018550 0x7f96bc0189d0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:40.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.733+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96bc0190e0 con 0x7f96bc018550 2024-09-19T05:43:40.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.747+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96bc0190b0 con 0x7f96bc018550 2024-09-19T05:43:40.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96d5ffb640 1 -- 172.21.15.2:0/1338576824 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96bc0190b0 con 0x7f96bc018550 2024-09-19T05:43:40.751 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:40.752 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:40.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96bc000fa0 msgr2=0x7f96bc0045e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f96bc000fa0 0x7f96bc0045e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96bc001d00 msgr2=0x7f96bc002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f96bc001d00 0x7f96bc002990 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96bc003c20 msgr2=0x7f96bc009c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f96bc003c20 0x7f96bc009c80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96bc00af00 msgr2=0x7f96bc00fa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f96bc00af00 0x7f96bc00fa70 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96bc012320 msgr2=0x7f96bc010000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f96bc012320 0x7f96bc010000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96bc010cb0 msgr2=0x7f96bc011110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f96bc010cb0 0x7f96bc011110 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96bc017440 msgr2=0x7f96bc017880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f96bc017440 0x7f96bc017880 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96bc018550 msgr2=0x7f96bc0189d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f96bc018550 0x7f96bc0189d0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 reap_dead start 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 reap_dead start 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96c4074ac0 msgr2=0x7f96c4076f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96c4074ac0 0x7f96c4076f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009f30 tx=0x7f96dc009190 comp rx=0 tx=0).stop 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e0158570 msgr2=0x7f96e01e15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.749+0000 7f96c37fe640 1 --2- 172.21.15.2:0/1338576824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e0158570 0x7f96e01e15e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009e50 tx=0x7f96d8004a40 comp rx=0 tx=0).stop 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.750+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 reap_dead start 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.750+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 reap_dead start 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.750+0000 7f96e5b61640 1 -- 172.21.15.2:0/1338576824 reap_dead start 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.750+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 shutdown_connections 2024-09-19T05:43:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.750+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 >> 172.21.15.2:0/1338576824 conn(0x7f96e0067950 msgr2=0x7f96e00cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.750+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 shutdown_connections 2024-09-19T05:43:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:40.750+0000 7f96c37fe640 1 -- 172.21.15.2:0/1338576824 wait complete. 2024-09-19T05:43:40.769 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:40.770 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:40.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:40.805 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:40.803+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:40.882 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:40.882 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:40.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:40.934 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:40.933+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:40.993 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:40.993 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:40.997 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:40.995+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:41.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:41.098 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:41.096+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:41.145 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:41.145 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:41.153 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:41.199 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:41.198+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:41.267 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:41.267 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:41.277 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:41.299 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:41.298+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:41.387 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:41.387 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:41.396 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:41.440 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:41.438+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:41.501 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:41.499+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:41.507 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:41.508 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:41.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:41.604 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:41.602+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:41.668 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:41.669 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:41.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:41.719 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:41.717+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:41.803 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:41.801+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:41.829 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:41.830 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:41.840 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:41.925 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:41.923+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:41.992 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:41.992 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:42.002 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:42.043 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:42.041+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:42.105 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:42.104+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:42.151 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:42.151 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:42.160 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:42.206 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:42.205+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:42.282 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:42.282 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:42.295 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:42.307 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:42.305+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:42.408 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:42.406+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:42.464 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:42.464 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:42.476 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:42.521 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:42.519+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:42.610 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:42.607+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:42.626 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:43:42.649 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:42.649 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:42.657 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:42.742 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:42.740+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:42.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.740+0000 7f069f00e640 1 Processor -- start 2024-09-19T05:43:42.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.740+0000 7f069f00e640 1 -- start start 2024-09-19T05:43:42.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.741+0000 7f069f00e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0698154c20 0x7f06980cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.741+0000 7f069f00e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f06980caf40 0x7f069815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.741+0000 7f069f00e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06980ccbc0 con 0x7f0698155000 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069f00e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f06980cc600 con 0x7f0698154c20 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069f00e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f06980cc8e0 con 0x7f06980caf40 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069cdaa640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0698154c20 0x7f06980cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f0697fff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f0698155000 0x7f0698073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49156/0 (socket says 172.21.15.2:49156) 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f0697fff640 1 -- 172.21.15.2:0/3375010207 learned_addr learned my addr 172.21.15.2:0/3375010207 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069d5ab640 1 --2- 172.21.15.2:0/3375010207 >> v2:172.21.15.123:3300/0 conn(0x7f06980caf40 0x7f069815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069cdaa640 1 -- 172.21.15.2:0/3375010207 >> v2:172.21.15.123:3300/0 conn(0x7f06980caf40 msgr2=0x7f069815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069cdaa640 1 --2- 172.21.15.2:0/3375010207 >> v2:172.21.15.123:3300/0 conn(0x7f06980caf40 0x7f069815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f06977fe640 1 -- 172.21.15.2:0/3375010207 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3306475375 0 0) 0x7f06980ccbc0 con 0x7f0698155000 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069cdaa640 1 -- 172.21.15.2:0/3375010207 >> v1:172.21.15.2:6789/0 conn(0x7f0698155000 legacy=0x7f0698073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.742+0000 7f069cdaa640 1 -- 172.21.15.2:0/3375010207 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06980cbc60 con 0x7f0698154c20 2024-09-19T05:43:42.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.744+0000 7f069cdaa640 1 --2- 172.21.15.2:0/3375010207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698154c20 0x7f06980cd9e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06980c5de0 tx=0x7f068c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a83adb7abf5cc73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.744+0000 7f06977fe640 1 -- 172.21.15.2:0/3375010207 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c037070 con 0x7f0698154c20 2024-09-19T05:43:42.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.744+0000 7f06977fe640 1 -- 172.21.15.2:0/3375010207 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c032440 con 0x7f0698154c20 2024-09-19T05:43:42.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.744+0000 7f06977fe640 1 -- 172.21.15.2:0/3375010207 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c032810 con 0x7f0698154c20 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 -- 172.21.15.2:0/3375010207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698154c20 msgr2=0x7f06980cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 --2- 172.21.15.2:0/3375010207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698154c20 0x7f06980cd9e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06980c5de0 tx=0x7f068c02a1a0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 -- 172.21.15.2:0/3375010207 shutdown_connections 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 --2- 172.21.15.2:0/3375010207 >> v2:172.21.15.123:3300/0 conn(0x7f06980caf40 0x7f069815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 --2- 172.21.15.2:0/3375010207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698154c20 0x7f06980cd9e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 -- 172.21.15.2:0/3375010207 >> 172.21.15.2:0/3375010207 conn(0x7f069806d0e0 msgr2=0x7f069806d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 -- 172.21.15.2:0/3375010207 shutdown_connections 2024-09-19T05:43:42.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.745+0000 7f069f00e640 1 -- 172.21.15.2:0/3375010207 wait complete. 2024-09-19T05:43:42.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.748+0000 7f069f00e640 1 Processor -- start 2024-09-19T05:43:42.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.748+0000 7f069f00e640 1 -- start start 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.749+0000 7f069f00e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06980caf40 0x7f06980c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.749+0000 7f069f00e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0698154c20 0x7f0698146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069f00e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698155000 0x7f0698147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069f00e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0698073a50 con 0x7f0698155000 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069cdaa640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06980caf40 0x7f06980c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069cdaa640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06980caf40 0x7f06980c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:43716/0 (socket says 172.21.15.2:43716) 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069cdaa640 1 -- 172.21.15.2:0/1679494634 learned_addr learned my addr 172.21.15.2:0/1679494634 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f0697fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0698154c20 0x7f0698146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069d5ab640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698155000 0x7f0698147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06980cb430 con 0x7f0698154c20 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0698073bd0 con 0x7f06980caf40 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069d5ab640 1 -- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06980caf40 msgr2=0x7f06980c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069d5ab640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06980caf40 0x7f06980c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069d5ab640 1 -- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0698154c20 msgr2=0x7f0698146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069d5ab640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0698154c20 0x7f0698146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f069d5ab640 1 -- 172.21.15.2:0/1679494634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0698147f90 con 0x7f0698155000 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.750+0000 7f0697fff640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0698154c20 0x7f0698146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.751+0000 7f069d5ab640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698155000 0x7f0698147340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0690009d90 tx=0x7f069000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.751+0000 7f069cdaa640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06980caf40 0x7f06980c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.751+0000 7f06957fa640 1 -- 172.21.15.2:0/1679494634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690018020 con 0x7f0698155000 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.751+0000 7f06957fa640 1 -- 172.21.15.2:0/1679494634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0690014480 con 0x7f0698155000 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.751+0000 7f06957fa640 1 -- 172.21.15.2:0/1679494634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690014870 con 0x7f0698155000 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.751+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06980cf5b0 con 0x7f0698155000 2024-09-19T05:43:42.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.751+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0698146600 con 0x7f0698155000 2024-09-19T05:43:42.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.753+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06980c5770 con 0x7f0698155000 2024-09-19T05:43:42.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.753+0000 7f06957fa640 1 -- 172.21.15.2:0/1679494634 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0690005d10 con 0x7f0698155000 2024-09-19T05:43:42.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.754+0000 7f06957fa640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f067c06c2b0 0x7f067c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.754+0000 7f06957fa640 1 -- 172.21.15.2:0/1679494634 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f069008bfd0 con 0x7f0698155000 2024-09-19T05:43:42.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.754+0000 7f069cdaa640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f067c06c2b0 0x7f067c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.755+0000 7f069cdaa640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f067c06c2b0 0x7f067c06e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f068c009560 tx=0x7f068c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.758+0000 7f06957fa640 1 -- 172.21.15.2:0/1679494634 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0690022030 con 0x7f0698155000 2024-09-19T05:43:42.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:42.811 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:42.810+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:42.855 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:42.855 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:42.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.864+0000 7fc93884c640 1 Processor -- start 2024-09-19T05:43:42.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.865+0000 7fc93884c640 1 -- start start 2024-09-19T05:43:42.868 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.866+0000 7fc93884c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc93406e950 0x7fc9340c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.866+0000 7fc93884c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc9340c7fa0 0x7fc93406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.866+0000 7fc93884c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc934071a10 con 0x7fc93406f270 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.866+0000 7fc93884c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fc934071cf0 con 0x7fc93406e950 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.866+0000 7fc93884c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc93406fad0 con 0x7fc9340c7fa0 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.866+0000 7fc93259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc93406e950 0x7fc9340c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.867+0000 7fc93259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc93406e950 0x7fc9340c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59172/0 (socket says 172.21.15.2:59172) 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.867+0000 7fc93259c640 1 -- 172.21.15.2:0/3579384032 learned_addr learned my addr 172.21.15.2:0/3579384032 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.867+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3579384032 >> v2:172.21.15.123:3300/0 conn(0x7fc9340c7fa0 0x7fc93406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.867+0000 7fc931d9b640 1 -- 172.21.15.2:0/3579384032 >> v2:172.21.15.2:3300/0 conn(0x7fc93406e950 msgr2=0x7fc9340c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:42.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.867+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3579384032 >> v2:172.21.15.2:3300/0 conn(0x7fc93406e950 0x7fc9340c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.867+0000 7fc931d9b640 1 -- 172.21.15.2:0/3579384032 >> v1:172.21.15.2:6789/0 conn(0x7fc93406f270 legacy=0x7fc93415ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.867+0000 7fc931d9b640 1 -- 172.21.15.2:0/3579384032 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93406fdb0 con 0x7fc9340c7fa0 2024-09-19T05:43:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.868+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3579384032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9340c7fa0 0x7fc93406ed30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc93406b060 tx=0x7fc92802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0291aa694bccf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.868+0000 7fc93159a640 1 -- 172.21.15.2:0/3579384032 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928037070 con 0x7fc9340c7fa0 2024-09-19T05:43:42.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.869+0000 7fc93159a640 1 -- 172.21.15.2:0/3579384032 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc928032440 con 0x7fc9340c7fa0 2024-09-19T05:43:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.869+0000 7fc93884c640 1 -- 172.21.15.2:0/3579384032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9340c7fa0 msgr2=0x7fc93406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.869+0000 7fc93884c640 1 --2- 172.21.15.2:0/3579384032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9340c7fa0 0x7fc93406ed30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc93406b060 tx=0x7fc92802a1a0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.870+0000 7fc93884c640 1 -- 172.21.15.2:0/3579384032 shutdown_connections 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.870+0000 7fc93884c640 1 --2- 172.21.15.2:0/3579384032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9340c7fa0 0x7fc93406ed30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.870+0000 7fc93884c640 1 --2- 172.21.15.2:0/3579384032 >> v2:172.21.15.2:3300/0 conn(0x7fc93406e950 0x7fc9340c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.870+0000 7fc93884c640 1 -- 172.21.15.2:0/3579384032 >> 172.21.15.2:0/3579384032 conn(0x7fc9340cc0d0 msgr2=0x7fc9340cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.870+0000 7fc93884c640 1 -- 172.21.15.2:0/3579384032 shutdown_connections 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.870+0000 7fc93884c640 1 -- 172.21.15.2:0/3579384032 wait complete. 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.872+0000 7fc93884c640 1 Processor -- start 2024-09-19T05:43:42.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.872+0000 7fc93884c640 1 -- start start 2024-09-19T05:43:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.873+0000 7fc93884c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc93406e950 0x7fc9340723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.873+0000 7fc93884c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc93406f270 0x7fc9341f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.873+0000 7fc93884c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9340c7fa0 0x7fc93414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.873+0000 7fc93884c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc934155140 con 0x7fc9340c7fa0 2024-09-19T05:43:42.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.873+0000 7fc93884c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc934154fc0 con 0x7fc93406f270 2024-09-19T05:43:42.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.873+0000 7fc93884c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc93415b530 con 0x7fc93406e950 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc93406f270 0x7fc9341f0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc93406f270 0x7fc9341f0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36390/0 (socket says 172.21.15.2:36390) 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 -- 172.21.15.2:0/3585077862 learned_addr learned my addr 172.21.15.2:0/3585077862 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc932d9d640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9340c7fa0 0x7fc93414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc93259c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc93406e950 0x7fc9340723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc93406e950 msgr2=0x7fc9340723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc93406e950 0x7fc9340723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9340c7fa0 msgr2=0x7fc93414f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9340c7fa0 0x7fc93414f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9341f0660 con 0x7fc93406f270 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc93406e950 0x7fc9340723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:42.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.874+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc93406f270 0x7fc9341f0e20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc928009890 tx=0x7fc9280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.875+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928037040 con 0x7fc93406f270 2024-09-19T05:43:42.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.875+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9341f4910 con 0x7fc93406f270 2024-09-19T05:43:42.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.875+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc928030a70 con 0x7fc93406f270 2024-09-19T05:43:42.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.875+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93415a4a0 con 0x7fc93406f270 2024-09-19T05:43:42.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.875+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc928030e40 con 0x7fc93406f270 2024-09-19T05:43:42.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.876+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc918005270 con 0x7fc93406f270 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.877+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc928044020 con 0x7fc93406f270 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.878+0000 7fc922ffd640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc91406c2b0 0x7fc91406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.878+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fc9280ab8b0 con 0x7fc93406f270 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.878+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc91406c2b0 0x7fc91406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.878+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v48) ==== 54+0+16 (secure 0 0 0) 0x7fc9280b0220 con 0x7fc93406f270 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.878+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc918000fa0 0x7fc9180045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.878+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc918004b20 con 0x7fc918000fa0 2024-09-19T05:43:42.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.879+0000 7fc932d9d640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc918000fa0 0x7fc9180045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.879+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc91406c2b0 0x7fc91406e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc924009c20 tx=0x7fc924008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.879+0000 7fc932d9d640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc918000fa0 0x7fc9180045e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.880+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc918004b20 con 0x7fc918000fa0 2024-09-19T05:43:42.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.896+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc918005090 con 0x7fc918000fa0 2024-09-19T05:43:42.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.898+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc918005090 con 0x7fc918000fa0 2024-09-19T05:43:42.901 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.899+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc918001c70 0x7fc918002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.899+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc9180022b0 con 0x7fc918001c70 2024-09-19T05:43:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.899+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc918001c70 0x7fc918002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.900+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc918001c70 0x7fc918002990 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.901+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9180022b0 con 0x7fc918001c70 2024-09-19T05:43:42.912 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:42.910+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:42.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.931+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc918003200 con 0x7fc918001c70 2024-09-19T05:43:42.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.935+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc918003200 con 0x7fc918001c70 2024-09-19T05:43:42.937 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:42.937 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:42.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.935+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc918009cd0 0x7fc91800c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.935+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc91800ee90 con 0x7fc918009cd0 2024-09-19T05:43:42.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.935+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc918009cd0 0x7fc91800c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.936+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc918009cd0 0x7fc91800c0c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.937+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc91800ee90 con 0x7fc918009cd0 2024-09-19T05:43:42.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.967+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc91800c8d0 con 0x7fc918009cd0 2024-09-19T05:43:42.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.970+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc91800c8d0 con 0x7fc918009cd0 2024-09-19T05:43:42.972 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:42.973 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:42.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.970+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc91800ca80 0x7fc91800cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:42.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.970+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc91800d420 con 0x7fc91800ca80 2024-09-19T05:43:42.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.970+0000 7fc932d9d640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc91800ca80 0x7fc91800cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:42.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.971+0000 7fc932d9d640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc91800ca80 0x7fc91800cee0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:42.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:42.972+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc91800d420 con 0x7fc91800ca80 2024-09-19T05:43:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.005+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc91800d930 con 0x7fc91800ca80 2024-09-19T05:43:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.007+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc91800d930 con 0x7fc91800ca80 2024-09-19T05:43:43.010 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:43.010 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:43.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.008+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc918011050 0x7fc918013440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:43.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.008+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc91800e050 con 0x7fc918011050 2024-09-19T05:43:43.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.008+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc918011050 0x7fc918013440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:43.011 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.009+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc918011050 0x7fc918013440 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:43.012 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.010+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc91800e050 con 0x7fc918011050 2024-09-19T05:43:43.012 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:43.010+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:43.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.037+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc91800e050 con 0x7fc918011050 2024-09-19T05:43:43.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.039+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f069814ea70 con 0x7f0698155000 2024-09-19T05:43:43.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.039+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc91800e050 con 0x7fc918011050 2024-09-19T05:43:43.042 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:43.042 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:43.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.040+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc918013e90 0x7fc9180142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:43.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.040+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc918019450 con 0x7fc918013e90 2024-09-19T05:43:43.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.040+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc918013e90 0x7fc9180142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:43.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.040+0000 7fc931d9b640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc918013e90 0x7fc9180142d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:43.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.041+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc918019450 con 0x7fc918013e90 2024-09-19T05:43:43.046 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:43.046 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:43.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.069139952, 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002950785, 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:43.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975062+0000", 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975069+0000", 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975082+0000", 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3139000000000001e-05 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975096+0000", 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4250000000000001e-05 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975106+0000", 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4949999999999997e-06 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975244+0000", 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013790100000000001 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122292 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975481+0000", 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011483499999999999 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977502+0000", 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020215849999999998 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977932+0000", 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000429877 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5309000000000003e-05 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978013+0000", 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5796000000000001e-05 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.069061599999998, 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033652420000000001, 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975141+0000", 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-07 2024-09-19T05:43:43.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975145+0000", 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975154+0000", 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0219999999999993e-06 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975161+0000", 2024-09-19T05:43:43.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9870000000000002e-06 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975720+0000", 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055893000000000002 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975804+0000", 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4025999999999998e-05 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975919+0000", 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115009 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978071+0000", 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002151563 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978433+0000", 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036205599999999998 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7867999999999997e-05 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978506+0000", 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5397e-05 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.061 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:28 2.b 2:d0630e4c:::mds0_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.069030776999995, 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030203629999999999, 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975172+0000", 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2099999999999997e-07 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975175+0000", 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1109999999999999e-06 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975182+0000", 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8859999999999998e-06 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975188+0000", 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4090000000000002e-06 2024-09-19T05:43:43.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975243+0000", 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5392000000000001e-05 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112596 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:43.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975488+0000", 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013297499999999999 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977490+0000", 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020015580000000001 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978100+0000", 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00060999199999999996 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978150+0000", 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0287000000000002e-05 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978192+0000", 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1635999999999999e-05 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:34 2.b 2:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.068770142000005, 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028916179999999999, 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975432+0000", 2024-09-19T05:43:43.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975433+0000", 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9599999999999999e-07 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975438+0000", 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8210000000000001e-06 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975448+0000", 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0145e-05 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975455+0000", 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9149999999999999e-06 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975979+0000", 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00052412100000000001 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976090+0000", 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111307 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978324+0000", 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022336130000000002 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.072 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.068709502000004, 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026148299999999998, 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975493+0000", 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975496+0000", 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:43.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975504+0000", 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5009999999999998e-06 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975510+0000", 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975620+0000", 2024-09-19T05:43:43.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109602 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975717+0000", 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7676999999999993e-05 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978108+0000", 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023904070000000002 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:4 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.068412753000004, 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022744829999999999, 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975790+0000", 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6600000000000005e-07 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975797+0000", 2024-09-19T05:43:43.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3790000000000003e-06 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975805+0000", 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975812+0000", 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.029e-06 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975861+0000", 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8266999999999997e-05 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975882+0000", 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0896000000000001e-05 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978029+0000", 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021471490000000001 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978064+0000", 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:5 2.1f e22/19 2:ff5b34d6:::1.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.067141207000006, 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001037481, 2024-09-19T05:43:43.081 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977061+0000", 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977066+0000", 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2250000000000002e-06 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977087+0000", 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977095+0000", 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7889999999999995e-06 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977159+0000", 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4196999999999995e-05 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977190+0000", 2024-09-19T05:43:43.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1526999999999998e-05 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000881224 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978099+0000", 2024-09-19T05:43:43.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6962999999999999e-05 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.06655963, 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046917399999999999, 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977643+0000", 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2399999999999997e-07 2024-09-19T05:43:43.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977647+0000", 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977671+0000", 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3926000000000001e-05 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977679+0000", 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.593e-06 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977789+0000", 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109969 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978112+0000", 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032325799999999999 2024-09-19T05:43:43.089 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.066481280000005, 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029909400000000002, 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977721+0000", 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977722+0000", 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.89e-07 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977725+0000", 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3830000000000001e-06 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977737+0000", 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2194999999999999e-05 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-19T05:43:43.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977920+0000", 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017706699999999999 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977927+0000", 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978020+0000", 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3358999999999998e-05 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:28 2.b e22/19 ondisk, result = 0)", 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.066247868999994, 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024615999999999999, 2024-09-19T05:43:43.094 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977955+0000", 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2699999999999999e-07 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977960+0000", 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3630000000000004e-06 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977970+0000", 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:43:43.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977978+0000", 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0209999999999999e-06 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978088+0000", 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109578 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9630000000000001e-06 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978201+0000", 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106018 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing_reply(2.b epoch 22/19 rep_tid 4 lcod 22'2)", 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:43.098 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.066211464999995, 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033492, 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:43.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977991+0000", 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0699999999999998e-07 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977999+0000", 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8690000000000004e-06 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978007+0000", 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4259999999999997e-06 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978012+0000", 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3929999999999999e-06 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978223+0000", 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000211051 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978326+0000", 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102874 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:25 2.7 e22/19 ondisk, result = 0)", 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.066020468999994, 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033417400000000002, 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978182+0000", 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0500000000000004e-07 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978186+0000", 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6349999999999999e-06 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978195+0000", 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7870000000000007e-06 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978367+0000", 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017261800000000001 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978420+0000", 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2339999999999997e-05 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978427+0000", 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.25e-06 2024-09-19T05:43:43.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978516+0000", 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9038999999999999e-05 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:26 2.1f e22/19 2:fe07bba1:::100.00000000.inode:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.065458735000007, 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047866199999999998, 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978744+0000", 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978750+0000", 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4809999999999997e-06 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978759+0000", 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1300000000000007e-06 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978763+0000", 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3290000000000004e-06 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978801+0000", 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8127999999999998e-05 2024-09-19T05:43:43.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978815+0000", 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3523000000000001e-05 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979197+0000", 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038223899999999998 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979222+0000", 2024-09-19T05:43:43.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5219999999999999e-05 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1f epoch 22/19 rep_tid 9 entries 22'4 (0'0) error 2:f83d65d8:::200.00000004:head by mds.0.4:35 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.065016310999994, 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033723899999999997, 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:43.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979186+0000", 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979187+0000", 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8999999999999996e-07 2024-09-19T05:43:43.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4270000000000002e-06 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979206+0000", 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313000000000001e-05 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979214+0000", 2024-09-19T05:43:43.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4420000000000001e-06 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979253+0000", 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.892e-05 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.979523+0000", 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000270547 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:43.115 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:43.115 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:43.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.055+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc918014920 con 0x7fc918013e90 2024-09-19T05:43:43.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:43.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.058+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc918014920 con 0x7fc918013e90 2024-09-19T05:43:43.117 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:43.118 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:43.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.058+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc918014fa0 0x7fc9180153e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:43.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.058+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc918015b00 con 0x7fc918014fa0 2024-09-19T05:43:43.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.058+0000 7fc932d9d640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc918014fa0 0x7fc9180153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:43.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.059+0000 7fc932d9d640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc918014fa0 0x7fc9180153e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:43.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.059+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc918015b00 con 0x7fc918014fa0 2024-09-19T05:43:43.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.079+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc918000950 con 0x7fc918014fa0 2024-09-19T05:43:43.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.081+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc918000950 con 0x7fc918014fa0 2024-09-19T05:43:43.119 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.081+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc91801a7f0 0x7fc91801cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.081+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc91801f9b0 con 0x7fc91801a7f0 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.081+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc91801a7f0 0x7fc91801cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.082+0000 7fc93259c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc91801a7f0 0x7fc91801cbe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.083+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc91801f9b0 con 0x7fc91801a7f0 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.096+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc91801d120 con 0x7fc91801a7f0 2024-09-19T05:43:43.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc922ffd640 1 -- 172.21.15.2:0/3585077862 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc91801d120 con 0x7fc91801a7f0 2024-09-19T05:43:43.121 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:43.122 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:43.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc918000fa0 msgr2=0x7fc9180045e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc918000fa0 0x7fc9180045e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc918001c70 msgr2=0x7fc918002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc918001c70 0x7fc918002990 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc918009cd0 msgr2=0x7fc91800c0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc918009cd0 0x7fc91800c0c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc91800ca80 msgr2=0x7fc91800cee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc91800ca80 0x7fc91800cee0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc918011050 msgr2=0x7fc918013440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc918011050 0x7fc918013440 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc918013e90 msgr2=0x7fc9180142d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.098+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc918013e90 0x7fc9180142d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc918014fa0 msgr2=0x7fc9180153e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc918014fa0 0x7fc9180153e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc91801a7f0 msgr2=0x7fc91801cbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc91801a7f0 0x7fc91801cbe0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc932d9d640 1 -- 172.21.15.2:0/3585077862 reap_dead start 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc91406c2b0 msgr2=0x7fc91406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc91406c2b0 0x7fc91406e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc924009c20 tx=0x7fc924008040 comp rx=0 tx=0).stop 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc932d9d640 1 -- 172.21.15.2:0/3585077862 reap_dead start 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc93406f270 msgr2=0x7fc9341f0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 --2- 172.21.15.2:0/3585077862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc93406f270 0x7fc9341f0e20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc928009890 tx=0x7fc9280046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc932d9d640 1 -- 172.21.15.2:0/3585077862 reap_dead start 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc932d9d640 1 -- 172.21.15.2:0/3585077862 reap_dead start 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc932d9d640 1 -- 172.21.15.2:0/3585077862 reap_dead start 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 shutdown_connections 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 >> 172.21.15.2:0/3585077862 conn(0x7fc9340cc0d0 msgr2=0x7fc9340bde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 shutdown_connections 2024-09-19T05:43:43.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.099+0000 7fc93884c640 1 -- 172.21.15.2:0/3585077862 wait complete. 2024-09-19T05:43:43.125 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:43.114+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:43.216 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:43.214+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:43.244 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:43.244 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:43.244 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:43.244 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:43.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:43.317 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:43.315+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:43.359 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:43.359 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:43.360 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:43.368 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:43.451 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:43.450+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:43.513 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.536039746, 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046763919999999997, 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.514 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974995+0000", 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975001+0000", 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4580000000000001e-06 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975020+0000", 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9562e-05 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975039+0000", 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9242e-05 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975050+0000", 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0178e-05 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975222+0000", 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172423 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975356+0000", 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134237 2024-09-19T05:43:43.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975566+0000", 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000209841 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979194+0000", 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036279139999999999 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979569+0000", 2024-09-19T05:43:43.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037515299999999998 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979624+0000", 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5265e-05 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979672+0000", 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7119000000000001e-05 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.535164089000006, 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041776799999999996, 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:43.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975871+0000", 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975872+0000", 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7300000000000004e-07 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975878+0000", 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9490000000000001e-06 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975889+0000", 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0795000000000001e-05 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975948+0000", 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9605999999999998e-05 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976010+0000", 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976083+0000", 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3405000000000006e-05 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037922299999999998 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979550+0000", 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030876250000000001 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979944+0000", 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00039340499999999999 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980004+0000", 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0449999999999999e-05 2024-09-19T05:43:43.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980049+0000", 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4401999999999999e-05 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534539909000003, 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038019949999999999, 2024-09-19T05:43:43.525 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976495+0000", 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976496+0000", 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2759999999999999e-06 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976520+0000", 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3349e-05 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976534+0000", 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4810000000000001e-05 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976543+0000", 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5420000000000005e-06 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976614+0000", 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1037000000000004e-05 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976639+0000", 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5080999999999999e-05 2024-09-19T05:43:43.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980251+0000", 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036122490000000001 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980297+0000", 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5651e-05 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.529 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534419170999996, 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037084309999999999, 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976616+0000", 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6200000000000005e-06 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976635+0000", 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7359999999999996e-06 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976641+0000", 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3520000000000003e-06 2024-09-19T05:43:43.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976879+0000", 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023839299999999999 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976897+0000", 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7354e-05 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980300+0000", 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034028980000000001 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980324+0000", 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4499e-05 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:43.534 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534379881999996, 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039957810000000003, 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:43.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976655+0000", 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6699999999999999e-07 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976668+0000", 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2649999999999999e-05 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976675+0000", 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1539999999999996e-06 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976682+0000", 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1879999999999996e-06 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977053+0000", 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00037080099999999998 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977067+0000", 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3719e-05 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980609+0000", 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035425719999999999 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980651+0000", 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534315382000003, 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044593430000000002, 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976720+0000", 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976721+0000", 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0559999999999999e-06 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976728+0000", 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6590000000000007e-06 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976743+0000", 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5043e-05 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976752+0000", 2024-09-19T05:43:43.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0019999999999995e-06 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976839+0000", 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6273000000000006e-05 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976976+0000", 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013742100000000001 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977093+0000", 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116916 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980036+0000", 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029435939999999999 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981094+0000", 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001057211 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981136+0000", 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2299999999999998e-05 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981179+0000", 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2867999999999997e-05 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.544 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534251964000006, 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050925320000000003, 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976783+0000", 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976784+0000", 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000003e-07 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976789+0000", 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3040000000000004e-06 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976800+0000", 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0562e-05 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976806+0000", 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3740000000000003e-06 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977402+0000", 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00059561100000000001 2024-09-19T05:43:43.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977478+0000", 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6049000000000003e-05 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977612+0000", 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000134244 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981208+0000", 2024-09-19T05:43:43.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003595708 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981773+0000", 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00056561800000000005 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981828+0000", 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5152000000000003e-05 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981875+0000", 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7205e-05 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534202308999994, 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051966069999999998, 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.550 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976833+0000", 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976851+0000", 2024-09-19T05:43:43.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8423e-05 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976857+0000", 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976867+0000", 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0086e-05 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976875+0000", 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5250000000000005e-06 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977857+0000", 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00098190899999999999 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978002+0000", 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145041 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978168+0000", 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000165577 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981217+0000", 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030489620000000001 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981963+0000", 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00074650400000000005 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982002+0000", 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8266000000000003e-05 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982029+0000", 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7515e-05 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.555 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534146140000004, 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052458110000000004, 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976889+0000", 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1200000000000003e-07 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976894+0000", 2024-09-19T05:43:43.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2799999999999997e-06 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976901+0000", 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2590000000000003e-06 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976907+0000", 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.567e-06 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978471+0000", 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015634360000000001 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978545+0000", 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4258000000000006e-05 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978667+0000", 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121397 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981221+0000", 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025547930000000001 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982075+0000", 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00085312799999999998 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982107+0000", 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2212000000000003e-05 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982135+0000", 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7869e-05 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534119149000006, 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053302410000000003, 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.561 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.06e-07 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976960+0000", 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3954e-05 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976973+0000", 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2335e-05 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976979+0000", 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8709999999999999e-06 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978919+0000", 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001939311 2024-09-19T05:43:43.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979088+0000", 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016881399999999999 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979211+0000", 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000123039 2024-09-19T05:43:43.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981225+0000", 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020147860000000002 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982190+0000", 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00096418399999999998 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982219+0000", 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9830999999999999e-05 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982246+0000", 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6610000000000001e-05 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:43.567 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534040701999999, 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060464159999999998, 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976994+0000", 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976995+0000", 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0399999999999995e-07 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977006+0000", 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1107999999999999e-05 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977014+0000", 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.807e-06 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977020+0000", 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7300000000000002e-06 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025226020000000001 2024-09-19T05:43:43.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979622+0000", 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9579999999999994e-05 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979740+0000", 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011852 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982302+0000", 2024-09-19T05:43:43.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0025617880000000002 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982942+0000", 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00063959100000000001 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982996+0000", 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4304e-05 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983041+0000", 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4681999999999997e-05 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.534007020999994, 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085412860000000004, 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.573 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977028+0000", 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.37e-07 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977032+0000", 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977037+0000", 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7069999999999997e-06 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977043+0000", 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4759999999999997e-06 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980074+0000", 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030308190000000001 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980196+0000", 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121908 2024-09-19T05:43:43.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980317+0000", 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012128 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982364+0000", 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020474099999999999 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985476+0000", 2024-09-19T05:43:43.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031114129999999999 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985531+0000", 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5146999999999999e-05 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985569+0000", 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8309999999999997e-05 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.578 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.533984009999998, 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086236330000000003, 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:43.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977051+0000", 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977055+0000", 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7689999999999998e-06 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977062+0000", 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4459999999999998e-06 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977076+0000", 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151000000000001e-05 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980661+0000", 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003585721 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980740+0000", 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8227999999999997e-05 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980869+0000", 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129135 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982884+0000", 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002015116 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985582+0000", 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026977279999999999 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985632+0000", 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9931e-05 2024-09-19T05:43:43.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.985675+0000", 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2979000000000003e-05 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.533950614999995, 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035181660000000001, 2024-09-19T05:43:43.584 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977084+0000", 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977085+0000", 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5899999999999996e-07 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977089+0000", 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0820000000000001e-06 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977096+0000", 2024-09-19T05:43:43.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3089999999999998e-06 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977101+0000", 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1000000000000003e-06 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977165+0000", 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4097999999999995e-05 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977293+0000", 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012744899999999999 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977399+0000", 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979355+0000", 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019564410000000002 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980522+0000", 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001166299 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980559+0000", 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7465000000000001e-05 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980602+0000", 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3364e-05 2024-09-19T05:43:43.589 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.533907936000006, 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004157371, 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977127+0000", 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977128+0000", 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7000000000000005e-07 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977132+0000", 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5850000000000001e-06 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977140+0000", 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9990000000000008e-06 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977156+0000", 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5797000000000001e-05 2024-09-19T05:43:43.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977876+0000", 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00072030100000000001 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977984+0000", 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107543 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978980+0000", 2024-09-19T05:43:43.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099578800000000006 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980952+0000", 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019726819999999999 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000168295 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981171+0000", 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0182999999999997e-05 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981284+0000", 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113628 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:43.595 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.533868771000002, 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003662013, 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977166+0000", 2024-09-19T05:43:43.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977167+0000", 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977171+0000", 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2549999999999999e-06 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977178+0000", 2024-09-19T05:43:43.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.939e-06 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977183+0000", 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4129999999999997e-06 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00031271499999999998 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977664+0000", 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016795100000000001 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977813+0000", 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149054 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979723+0000", 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001909633 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980736+0000", 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010134370000000001 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980787+0000", 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0464000000000003e-05 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980828+0000", 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1526999999999997e-05 2024-09-19T05:43:43.600 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.533836743999998, 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037051749999999998, 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977198+0000", 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977199+0000", 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5799999999999998e-07 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977210+0000", 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0642000000000001e-05 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977218+0000", 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6070000000000008e-06 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977225+0000", 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3829999999999997e-06 2024-09-19T05:43:43.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978766+0000", 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015413740000000001 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978879+0000", 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113365 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980903+0000", 2024-09-19T05:43:43.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020240459999999998 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.533638210999996, 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035664300000000002, 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977397+0000", 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977398+0000", 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1519999999999999e-06 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977405+0000", 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4189999999999996e-06 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977417+0000", 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2068e-05 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977426+0000", 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1239999999999994e-06 2024-09-19T05:43:43.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978334+0000", 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00090739599999999998 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978395+0000", 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1569000000000006e-05 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980916+0000", 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002520156 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980963+0000", 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 4 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:43.609 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.533567032999997, 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054677559999999998, 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977468+0000", 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977477+0000", 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.456e-06 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977496+0000", 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8753000000000002e-05 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977509+0000", 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3152e-05 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979786+0000", 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022774119999999999 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982936+0000", 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031495099999999999 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 65.527350874999996, 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004335541, 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:43.613 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983684+0000", 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:43.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983685+0000", 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3500000000000005e-07 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983692+0000", 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6390000000000003e-06 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983704+0000", 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2466999999999999e-05 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983710+0000", 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.733e-06 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983761+0000", 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1384999999999997e-05 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.983777+0000", 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6239000000000002e-05 2024-09-19T05:43:43.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.987977+0000", 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041994440000000001 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.988020+0000", 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2698999999999999e-05 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:43.617 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:43.618 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:43.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.614+0000 7f06957fa640 1 -- 172.21.15.2:0/1679494634 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v49) ==== 91+0+0 (secure 0 0 0) 0x7f0690010040 con 0x7f0698155000 2024-09-19T05:43:43.619 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:43:43.619 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:43.620 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:43.617+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:43.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f067c06c2b0 msgr2=0x7f067c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f067c06c2b0 0x7f067c06e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f068c009560 tx=0x7f068c02e000 comp rx=0 tx=0).stop 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698155000 msgr2=0x7f0698147340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698155000 0x7f0698147340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0690009d90 tx=0x7f069000c490 comp rx=0 tx=0).stop 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 shutdown_connections 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f067c06c2b0 0x7f067c06e770 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0698154c20 0x7f0698146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0698155000 0x7f0698147340 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 --2- 172.21.15.2:0/1679494634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06980caf40 0x7f06980c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.618+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 >> 172.21.15.2:0/1679494634 conn(0x7f069806d0e0 msgr2=0x7f0698151990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.619+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 shutdown_connections 2024-09-19T05:43:43.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:43.619+0000 7f069f00e640 1 -- 172.21.15.2:0/1679494634 wait complete. 2024-09-19T05:43:43.720 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:43.718+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:43.764 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:43.765 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:43.765 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:43.765 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:43.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:43.857 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:43.856+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:43.917 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:43.917 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:43.921 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:43.920+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:43.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:44.022 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:44.020+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:44.070 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.549684181000003, 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:43:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:43:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:43:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.549326213000001, 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:43:44.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:43:44.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.548825821999998, 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:43:44.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.541839589000006, 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:43:44.087 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:43:44.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:43:44.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.541381780999998, 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.518668160999994, 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.096 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 80.518173215999994, 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:43:44.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:43:44.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.95637026, 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.105 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:43:44.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:43:44.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:43:44.109 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.955612484, 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:43:44.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.955031437000002, 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:43:44.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:43:44.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.954396391000003, 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:43:44.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.948589885000004, 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.127 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.947933712999998, 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:43:44.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:43:44.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.471385038999998, 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.136 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.471016724999998, 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:43:44.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.470637815000003, 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.141 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:43:44.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.145 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.469940961999995, 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:44.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:43:44.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 79.464156348000003, 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:43:44.150 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:43:44.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.154 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.093041334000006, 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:44.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:43:44.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:43:44.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.092998250999997, 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:43:44.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:43:44.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:43:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:43:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.163 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.163 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:44.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:44.165 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:44.162+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:44.265 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:44.264+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:44.268 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:44.268 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:44.268 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:44.269 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:44.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:44.366 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:44.364+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:44.421 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:44.421 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:44.421 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:44.421 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:44.421 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:44.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:44.467 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:44.465+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:44.533 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.556267227000006, 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.538 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:44.542 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:44.568 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:44.566+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:44.649 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:44.649 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:44.649 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:44.649 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:44.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:44.668 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:44.666+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:44.762 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:44.762 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:44.768 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:44.767+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:44.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:44.869 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:44.867+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:44.916 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.916 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.939492853000004, 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062186530000000002, 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974705+0000", 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1270000000000002e-06 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974716+0000", 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1306e-05 2024-09-19T05:43:44.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974736+0000", 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9293e-05 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974749+0000", 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3059e-05 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:44.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022932900000000001 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975163+0000", 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018497500000000001 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975314+0000", 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151328 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:44.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979256+0000", 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039412179999999998 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571296 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980877+0000", 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0387999999999999e-05 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980919+0000", 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1334000000000001e-05 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.939370045000004, 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062599709999999996, 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.922 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974824+0000", 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2500000000000004e-07 2024-09-19T05:43:44.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974829+0000", 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.66e-06 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974844+0000", 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5160000000000001e-05 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974852+0000", 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9570000000000002e-06 2024-09-19T05:43:44.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975748+0000", 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089547100000000001 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975835+0000", 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7190999999999999e-05 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:44.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975997+0000", 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016230800000000001 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979261+0000", 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032635310000000001 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980973+0000", 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017117199999999999 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981027+0000", 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4732999999999999e-05 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981083+0000", 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5414999999999997e-05 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:30 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:44.927 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.939309514000001, 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040710980000000004, 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974884+0000", 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6400000000000001e-07 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974899+0000", 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0584000000000001e-05 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974917+0000", 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9799999999999998e-06 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974994+0000", 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6612000000000004e-05 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975176+0000", 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018201400000000001 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975321+0000", 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014440899999999999 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977845+0000", 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002524409 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:44.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978841+0000", 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099560399999999998 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978896+0000", 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5075999999999999e-05 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978954+0000", 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8743999999999999e-05 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.932 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.939241569000004, 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040869110000000004, 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974952+0000", 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974957+0000", 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7260000000000002e-06 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974966+0000", 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974972+0000", 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0970000000000002e-06 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975780+0000", 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080748199999999999 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975894+0000", 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113867 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976037+0000", 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014333 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978068+0000", 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020306439999999999 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978965+0000", 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089710699999999996 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979003+0000", 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8797999999999997e-05 2024-09-19T05:43:44.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979038+0000", 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4668000000000001e-05 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.939209851000001, 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062404959999999999, 2024-09-19T05:43:44.938 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3900000000000005e-07 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974999+0000", 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5817000000000001e-05 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975006+0000", 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5509999999999996e-06 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975012+0000", 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8189999999999997e-06 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976240+0000", 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012283870000000001 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976367+0000", 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126465 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976487+0000", 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120548 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980396+0000", 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003908702 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981127+0000", 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073081199999999995 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981182+0000", 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5921999999999998e-05 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981223+0000", 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0933999999999998e-05 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1e epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:7bf7e7b5:::200.00000006:head by mds.0.4:37 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:44.943 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.938084396999997, 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028842130000000001, 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976110+0000", 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2869999999999999e-06 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976123+0000", 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3224000000000001e-05 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976140+0000", 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.755e-05 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976157+0000", 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6464e-05 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976275+0000", 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117602 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978993+0000", 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027180860000000002 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.936877519000006, 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001691115, 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.947 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977316+0000", 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209999999999999e-06 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977337+0000", 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0890000000000002e-05 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977356+0000", 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032999999999999e-05 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977365+0000", 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7649999999999999e-06 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977425+0000", 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0044999999999999e-05 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977449+0000", 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4006999999999999e-05 2024-09-19T05:43:44.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978959+0000", 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015099379999999999 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979006+0000", 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7315999999999999e-05 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:30 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.936457411999996, 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118336, 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977736+0000", 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.096e-06 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0979999999999999e-06 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977756+0000", 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3891e-05 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977765+0000", 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7579999999999998e-06 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977830+0000", 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5184999999999993e-05 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977839+0000", 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4770000000000003e-06 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977854+0000", 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4831000000000001e-05 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:27 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:44.955 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.936219860999998, 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101439, 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977974+0000", 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6899999999999996e-07 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977979+0000", 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1399999999999999e-06 2024-09-19T05:43:44.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977993+0000", 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4215e-05 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978001+0000", 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978055+0000", 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3711000000000001e-05 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978062+0000", 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978074+0000", 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2238999999999999e-05 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.934975812000005, 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024064609999999999, 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979218+0000", 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979227+0000", 2024-09-19T05:43:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979239+0000", 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979247+0000", 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8369999999999997e-06 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979307+0000", 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0438e-05 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979326+0000", 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8938999999999999e-05 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981594+0000", 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022679710000000001 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981623+0000", 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9329999999999999e-05 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.934675131999995, 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021274990000000001, 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.964 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7800000000000001e-07 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979524+0000", 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8019999999999997e-06 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979534+0000", 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8859999999999999e-06 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8189999999999992e-06 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979610+0000", 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8433e-05 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979641+0000", 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1233999999999999e-05 2024-09-19T05:43:44.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981625+0000", 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001983964 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981645+0000", 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9783e-05 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.968 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.934614159999995, 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020817100000000001, 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979598+0000", 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9072e-05 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979609+0000", 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0360000000000001e-05 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979615+0000", 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8700000000000003e-06 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979866+0000", 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000250454 2024-09-19T05:43:44.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979884+0000", 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7725000000000001e-05 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981647+0000", 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017632629999999999 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981660+0000", 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3502999999999999e-05 2024-09-19T05:43:44.972 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.934127740999998, 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016104190000000001, 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.973 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980066+0000", 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8400000000000005e-07 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980070+0000", 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8010000000000003e-06 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980077+0000", 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.08e-06 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980082+0000", 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8450000000000002e-06 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980146+0000", 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3589000000000004e-05 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980166+0000", 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0514e-05 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981661+0000", 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014949539999999999 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981675+0000", 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:44.977 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.934077526999999, 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018022209999999999, 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980116+0000", 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980126+0000", 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980143+0000", 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6407999999999999e-05 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980150+0000", 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980385+0000", 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023545400000000001 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980403+0000", 2024-09-19T05:43:44.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7652999999999999e-05 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981869+0000", 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014661679999999999 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981917+0000", 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319000000000003e-05 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.981 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.933478574999995, 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000222196, 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:44.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980715+0000", 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980721+0000", 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2620000000000004e-06 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980735+0000", 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3756e-05 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980759+0000", 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980813+0000", 2024-09-19T05:43:44.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4249999999999997e-05 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980821+0000", 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7540000000000006e-06 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980936+0000", 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115574 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.933410512999998, 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030787300000000001, 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980783+0000", 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980786+0000", 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5279999999999999e-06 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980795+0000", 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0790000000000007e-06 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980802+0000", 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4910000000000002e-06 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980962+0000", 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015999100000000001 2024-09-19T05:43:44.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980968+0000", 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.874e-06 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981090+0000", 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012244099999999999 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:29 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.933380607000004, 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041785999999999999, 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1600000000000002e-07 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980816+0000", 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6660000000000001e-06 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980823+0000", 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6789999999999999e-06 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6480000000000002e-06 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981115+0000", 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028752699999999999 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981230+0000", 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108546 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:44.993 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.933357975999996, 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011264039999999999, 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980840+0000", 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1540000000000001e-06 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980848+0000", 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:43:44.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980853+0000", 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9030000000000003e-06 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980907+0000", 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4224999999999997e-05 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:44.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980937+0000", 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9852000000000001e-05 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981931+0000", 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099441099999999995 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981961+0000", 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0144000000000001e-05 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:44.997 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 66.932839732999994, 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013984010000000001, 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:44.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981354+0000", 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0160000000000001e-06 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981361+0000", 2024-09-19T05:43:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981373+0000", 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1439e-05 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981385+0000", 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2041999999999999e-05 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:45.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981446+0000", 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1446999999999996e-05 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981469+0000", 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982708+0000", 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012396359999999999 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:45.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982751+0000", 2024-09-19T05:43:45.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3198000000000002e-05 2024-09-19T05:43:45.002 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:45.002 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:45.002 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:45.002 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:45.002 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:45.002 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:45.003 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:45.004 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:45.001+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:45.104 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:45.103+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:45.115 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:45.115 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:45.115 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:45.115 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:45.124 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:45.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:45.205 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:45.204+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:45.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.274+0000 7f3f9f4dd640 1 Processor -- start 2024-09-19T05:43:45.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.274+0000 7f3f9f4dd640 1 -- start start 2024-09-19T05:43:45.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.275+0000 7f3f9f4dd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f980cd4e0 0x7f3f980b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.275+0000 7f3f9f4dd640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9815ebe0 con 0x7f3f9815aa90 2024-09-19T05:43:45.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.275+0000 7f3f9f4dd640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f980b2e00 con 0x7f3f980cd8c0 2024-09-19T05:43:45.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.275+0000 7f3f9f4dd640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9815eec0 con 0x7f3f980cd4e0 2024-09-19T05:43:45.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f9ca78640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3f9815aa90 0x7f3f9815df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49174/0 (socket says 172.21.15.2:49174) 2024-09-19T05:43:45.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f9d279640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f980cd4e0 0x7f3f980b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f9ca78640 1 -- 172.21.15.2:0/4207305410 learned_addr learned my addr 172.21.15.2:0/4207305410 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:45.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f9da7a640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3f980cd8c0 0x7f3f980c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:53076/0 (socket says 172.21.15.2:53076) 2024-09-19T05:43:45.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f9d279640 1 -- 172.21.15.2:0/4207305410 >> v1:172.21.15.123:6789/0 conn(0x7f3f980cd8c0 legacy=0x7f3f980c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f9d279640 1 -- 172.21.15.2:0/4207305410 >> v1:172.21.15.2:6789/0 conn(0x7f3f9815aa90 legacy=0x7f3f9815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f8ffff640 1 -- 172.21.15.2:0/4207305410 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122117873 0 0) 0x7f3f9815ebe0 con 0x7f3f9815aa90 2024-09-19T05:43:45.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.276+0000 7f3f9d279640 1 -- 172.21.15.2:0/4207305410 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f980b30e0 con 0x7f3f980cd4e0 2024-09-19T05:43:45.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.277+0000 7f3f9d279640 1 --2- 172.21.15.2:0/4207305410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 0x7f3f980b3bc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f940097a0 tx=0x7f3f9402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dd153ad1c550d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.277+0000 7f3f8ffff640 1 -- 172.21.15.2:0/4207305410 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94037070 con 0x7f3f980cd4e0 2024-09-19T05:43:45.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.277+0000 7f3f8ffff640 1 -- 172.21.15.2:0/4207305410 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94032440 con 0x7f3f980cd4e0 2024-09-19T05:43:45.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.277+0000 7f3f8ffff640 1 -- 172.21.15.2:0/4207305410 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94032810 con 0x7f3f980cd4e0 2024-09-19T05:43:45.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.278+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/4207305410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 msgr2=0x7f3f980b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.278+0000 7f3f9f4dd640 1 --2- 172.21.15.2:0/4207305410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 0x7f3f980b3bc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f940097a0 tx=0x7f3f9402a230 comp rx=0 tx=0).stop 2024-09-19T05:43:45.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.278+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/4207305410 shutdown_connections 2024-09-19T05:43:45.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.278+0000 7f3f9f4dd640 1 --2- 172.21.15.2:0/4207305410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 0x7f3f980b3bc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.278+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/4207305410 >> 172.21.15.2:0/4207305410 conn(0x7f3f98067b30 msgr2=0x7f3f98067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:45.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.278+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/4207305410 shutdown_connections 2024-09-19T05:43:45.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.278+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/4207305410 wait complete. 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.279+0000 7f3f9f4dd640 1 Processor -- start 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.279+0000 7f3f9f4dd640 1 -- start start 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9f4dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 0x7f3f980cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9f4dd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f980cd8c0 0x7f3f981f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9f4dd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f9815aa90 0x7f3f981f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9f4dd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9815e640 con 0x7f3f9815aa90 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9f4dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f980b36a0 con 0x7f3f980cd4e0 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9f4dd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9815e7c0 con 0x7f3f980cd8c0 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9da7a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f9815aa90 0x7f3f981f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9da7a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f9815aa90 0x7f3f981f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59204/0 (socket says 172.21.15.2:59204) 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9da7a640 1 -- 172.21.15.2:0/223134327 learned_addr learned my addr 172.21.15.2:0/223134327 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 0x7f3f980cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.280+0000 7f3f9ca78640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f980cd8c0 0x7f3f981f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9da7a640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f980cd8c0 msgr2=0x7f3f981f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9da7a640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f980cd8c0 0x7f3f981f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9da7a640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 msgr2=0x7f3f980cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9da7a640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 0x7f3f980cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9da7a640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981f3f60 con 0x7f3f9815aa90 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f980cd4e0 0x7f3f980cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9da7a640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f9815aa90 0x7f3f981f0c00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c7a0 tx=0x7f3f9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f90017070 con 0x7f3f9815aa90 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9000a5e0 con 0x7f3f9815aa90 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000a990 con 0x7f3f9815aa90 2024-09-19T05:43:45.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.281+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981f44b0 con 0x7f3f9815aa90 2024-09-19T05:43:45.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.282+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9815c280 con 0x7f3f9815aa90 2024-09-19T05:43:45.286 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:45.286 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:45.287 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:45.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.283+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3f9001a070 con 0x7f3f9815aa90 2024-09-19T05:43:45.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.283+0000 7f3f9f4dd640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f3f9815df30 con 0x7f3f9815aa90 2024-09-19T05:43:45.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.284+0000 7f3f8dffb640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3f7c06c2b0 0x7f3f7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.284+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f3f9008a930 con 0x7f3f9815aa90 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.284+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v49) ==== 54+0+16 (secure 0 0 0) 0x7f3f900589b0 con 0x7f3f9815aa90 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.284+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3f7c06c2b0 0x7f3f7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.284+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f9814e910 0x7f3f981509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.284+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f981f03c0 con 0x7f3f9814e910 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.285+0000 7f3f9ca78640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f9814e910 0x7f3f981509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.285+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3f7c06c2b0 0x7f3f7c06e770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f94034000 tx=0x7f3f9402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.285+0000 7f3f9ca78640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f9814e910 0x7f3f981509f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.286+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f880047e0 con 0x7f3f9814e910 2024-09-19T05:43:45.297 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:45.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.301+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f98150f30 con 0x7f3f9814e910 2024-09-19T05:43:45.307 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:45.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.304+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f98150f30 con 0x7f3f9814e910 2024-09-19T05:43:45.308 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:45.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.304+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f9814d830 0x7f3f9814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.304+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f3f9814e150 con 0x7f3f9814d830 2024-09-19T05:43:45.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.304+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f9814d830 0x7f3f9814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.308 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:45.304+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:45.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.305+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f9814d830 0x7f3f9814dc10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.306+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f9814e150 con 0x7f3f9814d830 2024-09-19T05:43:45.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.323+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f9815a3d0 con 0x7f3f9814d830 2024-09-19T05:43:45.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.325+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f9815a3d0 con 0x7f3f9814d830 2024-09-19T05:43:45.327 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:45.327 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:45.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.325+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f980cfba0 0x7f3f98062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.325+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f3f980cffa0 con 0x7f3f980cfba0 2024-09-19T05:43:45.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.325+0000 7f3f9ca78640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f980cfba0 0x7f3f98062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.325+0000 7f3f9ca78640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f980cfba0 0x7f3f98062630 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.328 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.327+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f980cffa0 con 0x7f3f980cfba0 2024-09-19T05:43:45.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.341+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f98062e40 con 0x7f3f980cfba0 2024-09-19T05:43:45.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.342+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f98062e40 con 0x7f3f980cfba0 2024-09-19T05:43:45.345 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:45.345 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:45.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.343+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f98151dd0 0x7f3f98152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.343+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f3f98152770 con 0x7f3f98151dd0 2024-09-19T05:43:45.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.343+0000 7f3f9da7a640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f98151dd0 0x7f3f98152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.343+0000 7f3f9da7a640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f98151dd0 0x7f3f98152230 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.344+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f98152770 con 0x7f3f98151dd0 2024-09-19T05:43:45.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.358+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f98152770 con 0x7f3f98151dd0 2024-09-19T05:43:45.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.360+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f98152770 con 0x7f3f98151dd0 2024-09-19T05:43:45.362 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.360+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f982030b0 0x7f3f982054a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.361+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f3f98208270 con 0x7f3f982030b0 2024-09-19T05:43:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.361+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f982030b0 0x7f3f982054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.361+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f982030b0 0x7f3f982054a0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.362+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f98208270 con 0x7f3f982030b0 2024-09-19T05:43:45.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.376+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f982059e0 con 0x7f3f982030b0 2024-09-19T05:43:45.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.378+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f982059e0 con 0x7f3f982030b0 2024-09-19T05:43:45.381 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:45.381 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:45.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.379+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f3f98206060 0x7f3f98208450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.379+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f3f9820b4b0 con 0x7f3f98206060 2024-09-19T05:43:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.379+0000 7f3f9ca78640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f3f98206060 0x7f3f98208450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.379+0000 7f3f9ca78640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f3f98206060 0x7f3f98208450 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.380+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f9820b4b0 con 0x7f3f98206060 2024-09-19T05:43:45.393 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:45.393 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:45.394 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-19T05:43:45.394 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T05:43:45.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.394+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f9820b480 con 0x7f3f98206060 2024-09-19T05:43:45.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.396+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f9820b480 con 0x7f3f98206060 2024-09-19T05:43:45.398 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:45.399 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:45.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.397+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f9820a700 0x7f3f9820ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.397+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f3f9820e8d0 con 0x7f3f9820a700 2024-09-19T05:43:45.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.397+0000 7f3f9da7a640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f9820a700 0x7f3f9820ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.397+0000 7f3f9da7a640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f9820a700 0x7f3f9820ab40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.398+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f9820e8d0 con 0x7f3f9820a700 2024-09-19T05:43:45.407 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:45.405+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:45.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.416+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f98208ab0 con 0x7f3f9820a700 2024-09-19T05:43:45.420 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.418+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f98208ab0 con 0x7f3f9820a700 2024-09-19T05:43:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.418+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f98208fc0 0x7f3f98209400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.418+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f3f98209940 con 0x7f3f98208fc0 2024-09-19T05:43:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.419+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f98208fc0 0x7f3f98209400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.419+0000 7f3f9d279640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f98208fc0 0x7f3f98209400 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.420+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f98209940 con 0x7f3f98208fc0 2024-09-19T05:43:45.443 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:45.444 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:45.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.446+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f98209940 con 0x7f3f98208fc0 2024-09-19T05:43:45.450 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.448+0000 7f3f8dffb640 1 -- 172.21.15.2:0/223134327 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f98209940 con 0x7f3f98208fc0 2024-09-19T05:43:45.451 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:45.454 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:45.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f9814e910 msgr2=0x7f3f981509f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f9814e910 0x7f3f981509f0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f9814d830 msgr2=0x7f3f9814dc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f9814d830 0x7f3f9814dc10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f980cfba0 msgr2=0x7f3f98062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f980cfba0 0x7f3f98062630 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f98151dd0 msgr2=0x7f3f98152230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f9da7a640 1 -- 172.21.15.2:0/223134327 reap_dead start 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.449+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f98151dd0 0x7f3f98152230 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f982030b0 msgr2=0x7f3f982054a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f982030b0 0x7f3f982054a0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f3f98206060 msgr2=0x7f3f98208450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f3f98206060 0x7f3f98208450 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f9820a700 msgr2=0x7f3f9820ab40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f9820a700 0x7f3f9820ab40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f98208fc0 msgr2=0x7f3f98209400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f98208fc0 0x7f3f98209400 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f9da7a640 1 -- 172.21.15.2:0/223134327 reap_dead start 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3f7c06c2b0 msgr2=0x7f3f7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3f7c06c2b0 0x7f3f7c06e770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f94034000 tx=0x7f3f9402e000 comp rx=0 tx=0).stop 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f9815aa90 msgr2=0x7f3f981f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f9815aa90 0x7f3f981f0c00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c7a0 tx=0x7f3f9000cc60 comp rx=0 tx=0).stop 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 shutdown_connections 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3f7c06c2b0 0x7f3f7c06e770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.450+0000 7f3f7b7fe640 1 --2- 172.21.15.2:0/223134327 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f9815aa90 0x7f3f981f0c00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.451+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 >> 172.21.15.2:0/223134327 conn(0x7f3f98067b30 msgr2=0x7f3f981585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:45.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.451+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 shutdown_connections 2024-09-19T05:43:45.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.451+0000 7f3f7b7fe640 1 -- 172.21.15.2:0/223134327 wait complete. 2024-09-19T05:43:45.458 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:45.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.507+0000 7fac5f76d640 1 Processor -- start 2024-09-19T05:43:45.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.508+0000 7fac5f76d640 1 -- start start 2024-09-19T05:43:45.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5f76d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5f76d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fac580c3430 0x7fac580c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5f76d640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac580c42c0 0x7fac580ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5f76d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fac58159700 con 0x7fac580c3430 2024-09-19T05:43:45.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5f76d640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fac5815bea0 con 0x7fac580c42c0 2024-09-19T05:43:45.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5f76d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac5815c180 con 0x7fac58152ab0 2024-09-19T05:43:45.512 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:45.509+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:45.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5d509640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5dd0a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac580c42c0 0x7fac580ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5dd0a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac580c42c0 0x7fac580ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36412/0 (socket says 172.21.15.2:36412) 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5dd0a640 1 -- 172.21.15.2:0/1147510887 learned_addr learned my addr 172.21.15.2:0/1147510887 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5cd08640 1 --2- 172.21.15.2:0/1147510887 >> v2:172.21.15.2:3300/0 conn(0x7fac580c3430 0x7fac580c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5dd0a640 1 -- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 msgr2=0x7fac581591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5dd0a640 1 --2- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5dd0a640 1 -- 172.21.15.2:0/1147510887 >> v2:172.21.15.2:3300/0 conn(0x7fac580c3430 msgr2=0x7fac580c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.509+0000 7fac5dd0a640 1 --2- 172.21.15.2:0/1147510887 >> v2:172.21.15.2:3300/0 conn(0x7fac580c3430 0x7fac580c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.510+0000 7fac5dd0a640 1 -- 172.21.15.2:0/1147510887 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5815c460 con 0x7fac580c42c0 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.510+0000 7fac5d509640 1 --2- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.511+0000 7fac5dd0a640 1 --2- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 0x7fac580ca0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fac580cbae0 tx=0x7fac5002a6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5ec89020094e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.511+0000 7fac4ffff640 1 -- 172.21.15.2:0/1147510887 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50036070 con 0x7fac580c42c0 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.511+0000 7fac4ffff640 1 -- 172.21.15.2:0/1147510887 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac50031600 con 0x7fac580c42c0 2024-09-19T05:43:45.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.511+0000 7fac4ffff640 1 -- 172.21.15.2:0/1147510887 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac500319d0 con 0x7fac580c42c0 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 -- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 msgr2=0x7fac580ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 --2- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 0x7fac580ca0e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fac580cbae0 tx=0x7fac5002a6b0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 -- 172.21.15.2:0/1147510887 shutdown_connections 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 --2- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 0x7fac580ca0e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 --2- 172.21.15.2:0/1147510887 >> v2:172.21.15.2:3300/0 conn(0x7fac580c3430 0x7fac580c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 --2- 172.21.15.2:0/1147510887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 -- 172.21.15.2:0/1147510887 >> 172.21.15.2:0/1147510887 conn(0x7fac580716e0 msgr2=0x7fac58071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:45.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 -- 172.21.15.2:0/1147510887 shutdown_connections 2024-09-19T05:43:45.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.512+0000 7fac5f76d640 1 -- 172.21.15.2:0/1147510887 wait complete. 2024-09-19T05:43:45.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.514+0000 7fac5f76d640 1 Processor -- start 2024-09-19T05:43:45.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.514+0000 7fac5f76d640 1 -- start start 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5f76d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac580c3430 0x7fac58149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5f76d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 0x7fac581f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5f76d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5f76d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac58152e90 con 0x7fac580c3430 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5f76d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac5815ca20 con 0x7fac580c42c0 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5f76d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac580c8e10 con 0x7fac58152ab0 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5d509640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac580c3430 0x7fac58149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5d509640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac580c3430 0x7fac58149120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59220/0 (socket says 172.21.15.2:59220) 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5d509640 1 -- 172.21.15.2:0/3684903173 learned_addr learned my addr 172.21.15.2:0/3684903173 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5dd0a640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.515+0000 7fac5cd08640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 0x7fac581f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac5d509640 1 -- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 msgr2=0x7fac581f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac5d509640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac5d509640 1 -- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 msgr2=0x7fac581f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac5d509640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 0x7fac581f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac5d509640 1 -- 172.21.15.2:0/3684903173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac581e1a80 con 0x7fac580c3430 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac5d509640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac580c3430 0x7fac58149120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e810 tx=0x7fac5400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac4dffb640 1 -- 172.21.15.2:0/3684903173 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5400f040 con 0x7fac580c3430 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac4dffb640 1 -- 172.21.15.2:0/3684903173 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac54004030 con 0x7fac580c3430 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac4dffb640 1 -- 172.21.15.2:0/3684903173 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54004400 con 0x7fac580c3430 2024-09-19T05:43:45.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.516+0000 7fac5f76d640 1 -- 172.21.15.2:0/3684903173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac581e1d20 con 0x7fac580c3430 2024-09-19T05:43:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.517+0000 7fac5f76d640 1 -- 172.21.15.2:0/3684903173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5806e2e0 con 0x7fac580c3430 2024-09-19T05:43:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.518+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac34005290 con 0x7fac580c3430 2024-09-19T05:43:45.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.519+0000 7fac4dffb640 1 -- 172.21.15.2:0/3684903173 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fac5400c6d0 con 0x7fac580c3430 2024-09-19T05:43:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.519+0000 7fac4dffb640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fac3c06c2e0 0x7fac3c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.520+0000 7fac4dffb640 1 -- 172.21.15.2:0/3684903173 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fac5408b2d0 con 0x7fac580c3430 2024-09-19T05:43:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.520+0000 7fac5cd08640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fac3c06c2e0 0x7fac3c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.521+0000 7fac5cd08640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fac3c06c2e0 0x7fac3c06e7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fac58149a20 tx=0x7fac48008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.529+0000 7fac4dffb640 1 -- 172.21.15.2:0/3684903173 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fac54017020 con 0x7fac580c3430 2024-09-19T05:43:45.609 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:45.607+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:45.625 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:45.625 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:45.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:45.638 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:43:45.720 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:45.718+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:45.810 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:45.809+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:45.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.835+0000 7fb9852a5640 1 Processor -- start 2024-09-19T05:43:45.838 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.836+0000 7fb9852a5640 1 -- start start 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb9852a5640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb9800c6de0 0x7fb9800c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb9852a5640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9800c71c0 0x7fb9801527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb9852a5640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb980152d30 0x7fb98015c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb9852a5640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fb980153180 con 0x7fb980152d30 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb9852a5640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb9800c85a0 con 0x7fb9800c6de0 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb9852a5640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb9800737c0 con 0x7fb9800c71c0 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb97f7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb980152d30 0x7fb98015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb97f7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb980152d30 0x7fb98015c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59226/0 (socket says 172.21.15.2:59226) 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb97f7fe640 1 -- 172.21.15.2:0/4021415392 learned_addr learned my addr 172.21.15.2:0/4021415392 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.837+0000 7fb97effd640 1 --2- 172.21.15.2:0/4021415392 >> v2:172.21.15.123:3300/0 conn(0x7fb9800c6de0 0x7fb9800c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.838+0000 7fb97e7fc640 1 --2- 172.21.15.2:0/4021415392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9800c71c0 0x7fb9801527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.838+0000 7fb97f7fe640 1 -- 172.21.15.2:0/4021415392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9800c71c0 msgr2=0x7fb9801527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:45.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.838+0000 7fb97f7fe640 1 --2- 172.21.15.2:0/4021415392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9800c71c0 0x7fb9801527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.838+0000 7fb97f7fe640 1 -- 172.21.15.2:0/4021415392 >> v2:172.21.15.123:3300/0 conn(0x7fb9800c6de0 msgr2=0x7fb9800c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:45.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.838+0000 7fb97f7fe640 1 --2- 172.21.15.2:0/4021415392 >> v2:172.21.15.123:3300/0 conn(0x7fb9800c6de0 0x7fb9800c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.838+0000 7fb97f7fe640 1 -- 172.21.15.2:0/4021415392 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb980073aa0 con 0x7fb980152d30 2024-09-19T05:43:45.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.838+0000 7fb97e7fc640 1 --2- 172.21.15.2:0/4021415392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9800c71c0 0x7fb9801527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:45.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.839+0000 7fb97f7fe640 1 --2- 172.21.15.2:0/4021415392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980152d30 0x7fb98015c470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb9800cb000 tx=0x7fb97802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3a58418209c85d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.839+0000 7fb97dffb640 1 -- 172.21.15.2:0/4021415392 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978036070 con 0x7fb980152d30 2024-09-19T05:43:45.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.840+0000 7fb97dffb640 1 -- 172.21.15.2:0/4021415392 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb978031600 con 0x7fb980152d30 2024-09-19T05:43:45.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.840+0000 7fb97dffb640 1 -- 172.21.15.2:0/4021415392 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780319d0 con 0x7fb980152d30 2024-09-19T05:43:45.843 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:45.843 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.840+0000 7fb9852a5640 1 -- 172.21.15.2:0/4021415392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980152d30 msgr2=0x7fb98015c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.840+0000 7fb9852a5640 1 --2- 172.21.15.2:0/4021415392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980152d30 0x7fb98015c470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb9800cb000 tx=0x7fb97802a690 comp rx=0 tx=0).stop 2024-09-19T05:43:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.841+0000 7fb9852a5640 1 -- 172.21.15.2:0/4021415392 shutdown_connections 2024-09-19T05:43:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.841+0000 7fb9852a5640 1 --2- 172.21.15.2:0/4021415392 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb980152d30 0x7fb98015c470 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.841+0000 7fb9852a5640 1 --2- 172.21.15.2:0/4021415392 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb9800c71c0 0x7fb9801527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.841+0000 7fb9852a5640 1 --2- 172.21.15.2:0/4021415392 >> v2:172.21.15.123:3300/0 conn(0x7fb9800c6de0 0x7fb9800c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.841+0000 7fb9852a5640 1 -- 172.21.15.2:0/4021415392 >> 172.21.15.2:0/4021415392 conn(0x7fb98006f3d0 msgr2=0x7fb9800707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.841+0000 7fb9852a5640 1 -- 172.21.15.2:0/4021415392 shutdown_connections 2024-09-19T05:43:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.841+0000 7fb9852a5640 1 -- 172.21.15.2:0/4021415392 wait complete. 2024-09-19T05:43:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.843+0000 7fb9852a5640 1 Processor -- start 2024-09-19T05:43:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.843+0000 7fb9852a5640 1 -- start start 2024-09-19T05:43:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.844+0000 7fb9852a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9800c6de0 0x7fb98013c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.844+0000 7fb9852a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9800c71c0 0x7fb9801ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.844+0000 7fb97effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9800c6de0 0x7fb98013c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9800c71c0 0x7fb9801ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9800c71c0 0x7fb9801ec2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59236/0 (socket says 172.21.15.2:59236) 2024-09-19T05:43:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9800c6de0 0x7fb98013c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36448/0 (socket says 172.21.15.2:36448) 2024-09-19T05:43:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 -- 172.21.15.2:0/3619390168 learned_addr learned my addr 172.21.15.2:0/3619390168 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb9852a5640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb980152d30 0x7fb9801ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb9852a5640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb98015bf50 con 0x7fb9800c71c0 2024-09-19T05:43:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb9852a5640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb980074060 con 0x7fb9800c6de0 2024-09-19T05:43:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb9852a5640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb98006dc50 con 0x7fb980152d30 2024-09-19T05:43:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97f7fe640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb980152d30 0x7fb9801ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 -- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb980152d30 msgr2=0x7fb9801ec7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb980152d30 0x7fb9801ec7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 -- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9800c6de0 msgr2=0x7fb98013c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9800c6de0 0x7fb98013c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.845+0000 7fb97e7fc640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9800ca990 con 0x7fb9800c71c0 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.846+0000 7fb97e7fc640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9800c71c0 0x7fb9801ec2a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e810 tx=0x7fb97400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.846+0000 7fb9677fe640 1 -- 172.21.15.2:0/3619390168 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97400f040 con 0x7fb9800c71c0 2024-09-19T05:43:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.846+0000 7fb9852a5640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9801ef9b0 con 0x7fb9800c71c0 2024-09-19T05:43:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.846+0000 7fb9852a5640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9801eff70 con 0x7fb9800c71c0 2024-09-19T05:43:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.848+0000 7fb9677fe640 1 -- 172.21.15.2:0/3619390168 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb974004030 con 0x7fb9800c71c0 2024-09-19T05:43:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.848+0000 7fb9677fe640 1 -- 172.21.15.2:0/3619390168 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974004400 con 0x7fb9800c71c0 2024-09-19T05:43:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.849+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9800c8890 con 0x7fb9800c71c0 2024-09-19T05:43:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.849+0000 7fb9677fe640 1 -- 172.21.15.2:0/3619390168 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb97400c6d0 con 0x7fb9800c71c0 2024-09-19T05:43:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.850+0000 7fb9677fe640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb96c06c2b0 0x7fb96c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.850+0000 7fb9677fe640 1 -- 172.21.15.2:0/3619390168 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fb97408b110 con 0x7fb9800c71c0 2024-09-19T05:43:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.850+0000 7fb97effd640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb96c06c2b0 0x7fb96c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:45.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.851+0000 7fb97effd640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb96c06c2b0 0x7fb96c06e770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb970016f90 tx=0x7fb970018040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:45.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.856+0000 7fb9677fe640 1 -- 172.21.15.2:0/3619390168 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb974017020 con 0x7fb9800c71c0 2024-09-19T05:43:45.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.888+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fac34005580 con 0x7fac580c3430 2024-09-19T05:43:45.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.890+0000 7fac4dffb640 1 -- 172.21.15.2:0/3684903173 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) ==== 81+0+34189 (secure 0 0 0) 0x7fac54059910 con 0x7fac580c3430 2024-09-19T05:43:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:43:45.893 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 49, 2024-09-19T05:43:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "896e162c-4658-4743-b70e-7ec2b5bce950", 2024-09-19T05:43:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-19T05:42:09.734568+0000", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-19T05:43:43.611765+0000", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-19T05:42:20.842106+0000", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-19T05:43:19.394170+0000", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5804032, 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "nodeep-scrub", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-19T05:43:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "jewel", 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-19T05:43:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:21.863894+0000", 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "18", 2024-09-19T05:43:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:45.898 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:45.899 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:45.900 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_metadata", 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:30.912746+0000", 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:45.902 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:45.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:45.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:45.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_bias": 4, 2024-09-19T05:43:45.906 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 16, 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_priority": 5 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": "cephfs" 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 2.25, 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 2.25, 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2.25, 2024-09-19T05:43:45.907 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2.25, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_data", 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:31.312255+0000", 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:43:45.908 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 64, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 64, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 64, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 64, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 64, 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:43:45.909 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "22", 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:43:45.910 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:43:45.911 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:43:45.912 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 64, 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "target_size_ratio": 0.89999997615814209 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "data": "cephfs" 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:43:45.913 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.5, 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.5, 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5, 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5, 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.914 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "665c2ac5-a964-4f38-8fa6-f4e95cac9272", 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.917 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/288510134", 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/288510134", 2024-09-19T05:43:45.919 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/288510134", 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/288510134", 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a4443f63-2c47-4c6f-a2df-6765b78afcc2", 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.920 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 41, 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.921 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.922 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-19T05:43:45.923 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-19T05:43:45.924 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2074594312", 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2074594312", 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2074594312", 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2074594312", 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.925 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "61465313-1ba1-4f26-b488-6ea73981419b", 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 40, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2644191985", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2644191985", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2644191985", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2644191985", 2024-09-19T05:43:45.930 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fa9d27d5-68d4-4467-a2a5-79f767a071aa", 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.931 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/3985782072", 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/3985782072", 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/3985782072", 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/3985782072", 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.936 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "85a040ff-ad8b-44ff-b82e-a36b05d83087", 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:45.937 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6816", 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6817", 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6818", 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6819", 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6822", 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6823", 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6820", 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6821", 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6817/2654957182", 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6819/2654957182", 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6823/2654957182", 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6821/2654957182", 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "f90f8fa4-4afc-4272-89eb-5e90123155d9", 2024-09-19T05:43:45.942 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6800", 2024-09-19T05:43:45.943 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6801", 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6802", 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6803", 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6806", 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6807", 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6804", 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6805", 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6801/3690741328", 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6803/3690741328", 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6807/3690741328", 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6805/3690741328", 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.947 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "48561580-ba9a-4096-9722-16c308f414ee", 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:43:45.948 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6824", 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6825", 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.949 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6826", 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6827", 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.950 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6830", 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6831", 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.951 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6828", 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6829", 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.952 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6825/593730548", 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6827/593730548", 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6831/593730548", 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6829/593730548", 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "d1ee7357-25a5-432c-b96a-e808139a1cf3", 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:43:45.953 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 38, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6808", 2024-09-19T05:43:45.954 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6809", 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6810", 2024-09-19T05:43:45.955 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6811", 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6814", 2024-09-19T05:43:45.956 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6815", 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6812", 2024-09-19T05:43:45.957 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6813", 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6809/1657801827", 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6811/1657801827", 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6815/1657801827", 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6813/1657801827", 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:43:45.958 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.148620+0000", 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.921396+0000", 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.960 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.980898+0000", 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.184322+0000", 2024-09-19T05:43:45.961 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.139258+0000", 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:43:45.962 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.479047+0000", 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.963 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.432111+0000", 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.402960+0000", 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.964 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-19T05:43:45.965 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-19T05:43:45.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:45.967 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:45.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.893+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fac3c06c2e0 msgr2=0x7fac3c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.893+0000 7fac3b7fe640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fac3c06c2e0 0x7fac3c06e7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fac58149a20 tx=0x7fac48008040 comp rx=0 tx=0).stop 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.893+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac580c3430 msgr2=0x7fac58149120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.893+0000 7fac3b7fe640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac580c3430 0x7fac58149120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fac5400e810 tx=0x7fac5400ecd0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.893+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 shutdown_connections 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.893+0000 7fac3b7fe640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fac3c06c2e0 0x7fac3c06e7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.893+0000 7fac3b7fe640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac580c42c0 0x7fac581f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.894+0000 7fac3b7fe640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac58152ab0 0x7fac581f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.894+0000 7fac3b7fe640 1 --2- 172.21.15.2:0/3684903173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac580c3430 0x7fac58149120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.894+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 >> 172.21.15.2:0/3684903173 conn(0x7fac580716e0 msgr2=0x7fac5814b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.894+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 shutdown_connections 2024-09-19T05:43:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:45.894+0000 7fac3b7fe640 1 -- 172.21.15.2:0/3684903173 wait complete. 2024-09-19T05:43:45.970 INFO:tasks.thrashosds.thrasher:j is {'epoch': 49, 'fsid': '896e162c-4658-4743-b70e-7ec2b5bce950', 'created': '2024-09-19T05:42:09.734568+0000', 'modified': '2024-09-19T05:43:43.611765+0000', 'last_up_change': '2024-09-19T05:42:20.842106+0000', 'last_in_change': '2024-09-19T05:43:19.394170+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:42:21.863894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:42:30.912746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:42:31.312255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '665c2ac5-a964-4f38-8fa6-f4e95cac9272', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 288510134}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 288510134}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 288510134}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 288510134}]}, 'public_addr': '172.21.15.2:6809/288510134', 'cluster_addr': '172.21.15.2:6811/288510134', 'heartbeat_back_addr': '172.21.15.2:6815/288510134', 'heartbeat_front_addr': '172.21.15.2:6813/288510134', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a4443f63-2c47-4c6f-a2df-6765b78afcc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2074594312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2074594312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2074594312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2074594312}]}, 'public_addr': '172.21.15.2:6817/2074594312', 'cluster_addr': '172.21.15.2:6819/2074594312', 'heartbeat_back_addr': '172.21.15.2:6823/2074594312', 'heartbeat_front_addr': '172.21.15.2:6821/2074594312', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61465313-1ba1-4f26-b488-6ea73981419b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2644191985}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2644191985}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2644191985}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2644191985}]}, 'public_addr': '172.21.15.2:6825/2644191985', 'cluster_addr': '172.21.15.2:6827/2644191985', 'heartbeat_back_addr': '172.21.15.2:6831/2644191985', 'heartbeat_front_addr': '172.21.15.2:6829/2644191985', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fa9d27d5-68d4-4467-a2a5-79f767a071aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 3985782072}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 3985782072}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 3985782072}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 3985782072}]}, 'public_addr': '172.21.15.2:6801/3985782072', 'cluster_addr': '172.21.15.2:6803/3985782072', 'heartbeat_back_addr': '172.21.15.2:6807/3985782072', 'heartbeat_front_addr': '172.21.15.2:6805/3985782072', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85a040ff-ad8b-44ff-b82e-a36b05d83087', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 2654957182}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 2654957182}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 2654957182}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 2654957182}]}, 'public_addr': '172.21.15.123:6817/2654957182', 'cluster_addr': '172.21.15.123:6819/2654957182', 'heartbeat_back_addr': '172.21.15.123:6823/2654957182', 'heartbeat_front_addr': '172.21.15.123:6821/2654957182', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f90f8fa4-4afc-4272-89eb-5e90123155d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 3690741328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 3690741328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 3690741328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 3690741328}]}, 'public_addr': '172.21.15.123:6801/3690741328', 'cluster_addr': '172.21.15.123:6803/3690741328', 'heartbeat_back_addr': '172.21.15.123:6807/3690741328', 'heartbeat_front_addr': '172.21.15.123:6805/3690741328', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '48561580-ba9a-4096-9722-16c308f414ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 593730548}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 593730548}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 593730548}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 593730548}]}, 'public_addr': '172.21.15.123:6825/593730548', 'cluster_addr': '172.21.15.123:6827/593730548', 'heartbeat_back_addr': '172.21.15.123:6831/593730548', 'heartbeat_front_addr': '172.21.15.123:6829/593730548', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd1ee7357-25a5-432c-b96a-e808139a1cf3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 1657801827}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 1657801827}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 1657801827}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 1657801827}]}, 'public_addr': '172.21.15.123:6809/1657801827', 'cluster_addr': '172.21.15.123:6811/1657801827', 'heartbeat_back_addr': '172.21.15.123:6815/1657801827', 'heartbeat_front_addr': '172.21.15.123:6813/1657801827', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.148620+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.921396+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.980898+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.184322+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.139258+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.479047+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.432111+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.402960+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T05:43:45.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:43:45.971 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:45.966+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:46.037 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:46.037 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:46.051 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:46.092 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:46.090+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:46.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.130+0000 7f6a88b42640 1 Processor -- start 2024-09-19T05:43:46.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.130+0000 7f6a88b42640 1 -- start start 2024-09-19T05:43:46.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.132+0000 7f6a88b42640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8406e6c0 0x7f6a840c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.132+0000 7f6a88b42640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6a84151340 0x7f6a84157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.132+0000 7f6a81d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6a8406eaa0 0x7f6a84150c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49176/0 (socket says 172.21.15.2:49176) 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a88b42640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a8415a5c0 con 0x7f6a8406eaa0 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a82d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6a84151340 0x7f6a84157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a82d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6a84151340 0x7f6a84157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59238/0 (socket says 172.21.15.2:59238) 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a82d9d640 1 -- 172.21.15.2:0/3099541887 learned_addr learned my addr 172.21.15.2:0/3099541887 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a88b42640 1 -- 172.21.15.2:0/3099541887 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f6a8415a860 con 0x7f6a84151340 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a88b42640 1 -- 172.21.15.2:0/3099541887 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8415ab40 con 0x7f6a8406e6c0 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1129825483 0 0) 0x7f6a8415a5c0 con 0x7f6a8406eaa0 2024-09-19T05:43:46.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.133+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a6c003520 con 0x7f6a8406eaa0 2024-09-19T05:43:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8259c640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8406e6c0 0x7f6a840c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2988244191 0 0) 0x7f6a6c003520 con 0x7f6a8406eaa0 2024-09-19T05:43:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8406e6c0 msgr2=0x7f6a840c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a8406e6c0 0x7f6a840c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 >> v2:172.21.15.2:3300/0 conn(0x7f6a84151340 msgr2=0x7f6a84157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 --2- 172.21.15.2:0/3099541887 >> v2:172.21.15.2:3300/0 conn(0x7f6a84151340 0x7f6a84157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8415ae20 con 0x7f6a8406eaa0 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f6a78001670 con 0x7f6a8406eaa0 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 >> v1:172.21.15.2:6789/0 conn(0x7f6a8406eaa0 legacy=0x7f6a84150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c003a20 0x7f6a6c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a6c004800 0x7f6a6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a6c0051c0 0x7f6a6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8415a860 con 0x7f6a6c004800 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8415ab40 con 0x7f6a6c003a20 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.134+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6a6c0148a0 con 0x7f6a6c0051c0 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.135+0000 7f6a82d9d640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a6c0051c0 0x7f6a6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.135+0000 7f6a81d9b640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a6c004800 0x7f6a6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.135+0000 7f6a81d9b640 1 -- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a6c0051c0 msgr2=0x7f6a6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:46.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.135+0000 7f6a81d9b640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a6c0051c0 0x7f6a6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.135+0000 7f6a81d9b640 1 -- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c003a20 msgr2=0x7f6a6c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.135+0000 7f6a81d9b640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a6c003a20 0x7f6a6c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.135+0000 7f6a81d9b640 1 -- 172.21.15.2:0/3099541887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8415ae20 con 0x7f6a6c004800 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.136+0000 7f6a81d9b640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a6c004800 0x7f6a6c004c80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a8406da00 tx=0x7f6a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9525a1835e895f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.137+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78007cf0 con 0x7f6a6c004800 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.137+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a78007270 con 0x7f6a6c004800 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.137+0000 7f6a8159a640 1 -- 172.21.15.2:0/3099541887 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78007660 con 0x7f6a6c004800 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.137+0000 7f6a88b42640 1 -- 172.21.15.2:0/3099541887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a6c004800 msgr2=0x7f6a6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.138+0000 7f6a88b42640 1 --2- 172.21.15.2:0/3099541887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a6c004800 0x7f6a6c004c80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a8406da00 tx=0x7f6a6c005830 comp rx=0 tx=0).stop 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.138+0000 7f6a82d9d640 1 -- 172.21.15.2:0/3099541887 reap_dead start 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.138+0000 7f6a88b42640 1 -- 172.21.15.2:0/3099541887 shutdown_connections 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.138+0000 7f6a88b42640 1 -- 172.21.15.2:0/3099541887 >> 172.21.15.2:0/3099541887 conn(0x7f6a840ccd70 msgr2=0x7f6a840cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:46.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.138+0000 7f6a88b42640 1 -- 172.21.15.2:0/3099541887 shutdown_connections 2024-09-19T05:43:46.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.138+0000 7f6a88b42640 1 -- 172.21.15.2:0/3099541887 wait complete. 2024-09-19T05:43:46.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.140+0000 7f6a88b42640 1 Processor -- start 2024-09-19T05:43:46.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.140+0000 7f6a88b42640 1 -- start start 2024-09-19T05:43:46.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.141+0000 7f6a88b42640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a840cbe30 0x7f6a841ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.141+0000 7f6a88b42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a840cc210 0x7f6a841523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a81d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a840cc210 0x7f6a841523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a81d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a840cc210 0x7f6a841523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59242/0 (socket says 172.21.15.2:59242) 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a81d9b640 1 -- 172.21.15.2:0/2445540849 learned_addr learned my addr 172.21.15.2:0/2445540849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a88b42640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a841ec650 0x7f6a841ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a8259c640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a840cbe30 0x7f6a841ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a88b42640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6a840c92a0 con 0x7f6a840cc210 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a88b42640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a840c9120 con 0x7f6a840cbe30 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a88b42640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8406d1b0 con 0x7f6a841ec650 2024-09-19T05:43:46.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.142+0000 7f6a82d9d640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a841ec650 0x7f6a841ecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a81d9b640 1 -- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a841ec650 msgr2=0x7f6a841ecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a81d9b640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a841ec650 0x7f6a841ecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a81d9b640 1 -- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a840cbe30 msgr2=0x7f6a841ef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a81d9b640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a840cbe30 0x7f6a841ef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a81d9b640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a84154d40 con 0x7f6a840cc210 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a82d9d640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a841ec650 0x7f6a841ecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a81d9b640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a840cc210 0x7f6a841523b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6a7802ae00 tx=0x7f6a78001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a72ffd640 1 -- 172.21.15.2:0/2445540849 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78044070 con 0x7f6a840cc210 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a72ffd640 1 -- 172.21.15.2:0/2445540849 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a78039040 con 0x7f6a840cc210 2024-09-19T05:43:46.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a72ffd640 1 -- 172.21.15.2:0/2445540849 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004bf0 con 0x7f6a840cc210 2024-09-19T05:43:46.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a88b42640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a84155020 con 0x7f6a840cc210 2024-09-19T05:43:46.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.143+0000 7f6a88b42640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a841555e0 con 0x7f6a840cc210 2024-09-19T05:43:46.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.145+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a840c8130 con 0x7f6a840cc210 2024-09-19T05:43:46.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.147+0000 7f6a72ffd640 1 -- 172.21.15.2:0/2445540849 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6a780333f0 con 0x7f6a840cc210 2024-09-19T05:43:46.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.147+0000 7f6a72ffd640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6a6406c2e0 0x7f6a6406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.147+0000 7f6a72ffd640 1 -- 172.21.15.2:0/2445540849 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f6a780aba40 con 0x7f6a840cc210 2024-09-19T05:43:46.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.149+0000 7f6a8259c640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6a6406c2e0 0x7f6a6406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.149+0000 7f6a8259c640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6a6406c2e0 0x7f6a6406e7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400a3e0 tx=0x7f6a7400a370 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:46.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.155+0000 7f6a72ffd640 1 -- 172.21.15.2:0/2445540849 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6a78079b40 con 0x7f6a840cc210 2024-09-19T05:43:46.169 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:46.167+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:46.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.205+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fb9801442d0 con 0x7fb9800c71c0 2024-09-19T05:43:46.223 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:46.223 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:46.233 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:46.270 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:46.269+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:46.371 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:46.370+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:46.384 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:46.385 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:46.394 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:46.472 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:46.470+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:46.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.488+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a841ed040 con 0x7f6a6406c2e0 2024-09-19T05:43:46.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.494+0000 7f6a72ffd640 1 -- 172.21.15.2:0/2445540849 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252222 (secure 0 0 0) 0x7f6a841ed040 con 0x7f6a6406c2e0 2024-09-19T05:43:46.500 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:43:46.501 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:43:46.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6a6406c2e0 msgr2=0x7f6a6406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6a6406c2e0 0x7f6a6406e7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400a3e0 tx=0x7f6a7400a370 comp rx=0 tx=0).stop 2024-09-19T05:43:46.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a840cc210 msgr2=0x7f6a841523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a840cc210 0x7f6a841523b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6a7802ae00 tx=0x7f6a78001970 comp rx=0 tx=0).stop 2024-09-19T05:43:46.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 shutdown_connections 2024-09-19T05:43:46.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6a6406c2e0 0x7f6a6406e7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6a841ec650 0x7f6a841ecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6a840cc210 0x7f6a841523b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 --2- 172.21.15.2:0/2445540849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a840cbe30 0x7f6a841ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 >> 172.21.15.2:0/2445540849 conn(0x7f6a840ccd70 msgr2=0x7f6a8406c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:46.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.498+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 shutdown_connections 2024-09-19T05:43:46.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.499+0000 7f6a70ff9640 1 -- 172.21.15.2:0/2445540849 wait complete. 2024-09-19T05:43:46.505 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:46.505 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:46.515 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:46.518 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-19T05:43:45.832999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597828,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":215,"num_write_kb":2440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":590508,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":163,"ondisk_log_size":163,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237376,"kb_used_data":4512,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737344,"statfs":{"total":773094113280,"available":772851040256,"internally_reserved":0,"allocated":4620288,"data_stored":3103814,"data_compressed":66785,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001013"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619233+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:43:43.619233+0000","last_peered":"2024-09-19T05:43:43.619233+0000","last_clean":"2024-09-19T05:43:43.619233+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:43:43.619233+0000","last_undegraded":"2024-09-19T05:43:43.619233+0000","last_fullsized":"2024-09-19T05:43:43.619233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268256+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:43:44.268256+0000","last_peered":"2024-09-19T05:43:44.268256+0000","last_clean":"2024-09-19T05:43:44.268256+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:43:44.268256+0000","last_undegraded":"2024-09-19T05:43:44.268256+0000","last_fullsized":"2024-09-19T05:43:44.268256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268804+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:43:44.268804+0000","last_peered":"2024-09-19T05:43:44.268804+0000","last_clean":"2024-09-19T05:43:44.268804+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:43:44.268804+0000","last_undegraded":"2024-09-19T05:43:44.268804+0000","last_fullsized":"2024-09-19T05:43:44.268804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.474546+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:43:41.474546+0000","last_peered":"2024-09-19T05:43:41.474546+0000","last_clean":"2024-09-19T05:43:41.474546+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:43:41.474546+0000","last_undegraded":"2024-09-19T05:43:41.474546+0000","last_fullsized":"2024-09-19T05:43:41.474546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619172+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:43:43.619172+0000","last_peered":"2024-09-19T05:43:43.619172+0000","last_clean":"2024-09-19T05:43:43.619172+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:43:43.619172+0000","last_undegraded":"2024-09-19T05:43:43.619172+0000","last_fullsized":"2024-09-19T05:43:43.619172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.474166+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:43:41.474166+0000","last_peered":"2024-09-19T05:43:41.474166+0000","last_clean":"2024-09-19T05:43:41.474166+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:43:41.474166+0000","last_undegraded":"2024-09-19T05:43:41.474166+0000","last_fullsized":"2024-09-19T05:43:41.474166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.618618+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:43:43.618618+0000","last_peered":"2024-09-19T05:43:43.618618+0000","last_clean":"2024-09-19T05:43:43.618618+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:43:43.618618+0000","last_undegraded":"2024-09-19T05:43:43.618618+0000","last_fullsized":"2024-09-19T05:43:43.618618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268098+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:43:44.268098+0000","last_peered":"2024-09-19T05:43:44.268098+0000","last_clean":"2024-09-19T05:43:44.268098+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:43:44.268098+0000","last_undegraded":"2024-09-19T05:43:44.268098+0000","last_fullsized":"2024-09-19T05:43:44.268098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":30,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.269312+0000","last_change":"2024-09-19T05:43:19.548002+0000","last_active":"2024-09-19T05:43:44.269312+0000","last_peered":"2024-09-19T05:43:44.269312+0000","last_clean":"2024-09-19T05:43:44.269312+0000","last_became_active":"2024-09-19T05:43:19.547710+0000","last_became_peered":"2024-09-19T05:43:19.547710+0000","last_unstale":"2024-09-19T05:43:44.269312+0000","last_undegraded":"2024-09-19T05:43:44.269312+0000","last_fullsized":"2024-09-19T05:43:44.269312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619604+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:43:43.619604+0000","last_peered":"2024-09-19T05:43:43.619604+0000","last_clean":"2024-09-19T05:43:43.619604+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:43:43.619604+0000","last_undegraded":"2024-09-19T05:43:43.619604+0000","last_fullsized":"2024-09-19T05:43:43.619604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619490+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:43:43.619490+0000","last_peered":"2024-09-19T05:43:43.619490+0000","last_clean":"2024-09-19T05:43:43.619490+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:43:43.619490+0000","last_undegraded":"2024-09-19T05:43:43.619490+0000","last_fullsized":"2024-09-19T05:43:43.619490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619090+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:43:43.619090+0000","last_peered":"2024-09-19T05:43:43.619090+0000","last_clean":"2024-09-19T05:43:43.619090+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:43:43.619090+0000","last_undegraded":"2024-09-19T05:43:43.619090+0000","last_fullsized":"2024-09-19T05:43:43.619090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:54.111389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268446+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:43:44.268446+0000","last_peered":"2024-09-19T05:43:44.268446+0000","last_clean":"2024-09-19T05:43:44.268446+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:43:44.268446+0000","last_undegraded":"2024-09-19T05:43:44.268446+0000","last_fullsized":"2024-09-19T05:43:44.268446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268321+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:43:44.268321+0000","last_peered":"2024-09-19T05:43:44.268321+0000","last_clean":"2024-09-19T05:43:44.268321+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:43:44.268321+0000","last_undegraded":"2024-09-19T05:43:44.268321+0000","last_fullsized":"2024-09-19T05:43:44.268321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.267900+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:43:44.267900+0000","last_peered":"2024-09-19T05:43:44.267900+0000","last_clean":"2024-09-19T05:43:44.267900+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:43:44.267900+0000","last_undegraded":"2024-09-19T05:43:44.267900+0000","last_fullsized":"2024-09-19T05:43:44.267900+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234420+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:43:44.234420+0000","last_peered":"2024-09-19T05:43:44.234420+0000","last_clean":"2024-09-19T05:43:44.234420+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:43:44.234420+0000","last_undegraded":"2024-09-19T05:43:44.234420+0000","last_fullsized":"2024-09-19T05:43:44.234420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234739+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:43:44.234739+0000","last_peered":"2024-09-19T05:43:44.234739+0000","last_clean":"2024-09-19T05:43:44.234739+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:43:44.234739+0000","last_undegraded":"2024-09-19T05:43:44.234739+0000","last_fullsized":"2024-09-19T05:43:44.234739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620773+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:43:43.620773+0000","last_peered":"2024-09-19T05:43:43.620773+0000","last_clean":"2024-09-19T05:43:43.620773+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:43:43.620773+0000","last_undegraded":"2024-09-19T05:43:43.620773+0000","last_fullsized":"2024-09-19T05:43:43.620773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234194+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:43:44.234194+0000","last_peered":"2024-09-19T05:43:44.234194+0000","last_clean":"2024-09-19T05:43:44.234194+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:43:44.234194+0000","last_undegraded":"2024-09-19T05:43:44.234194+0000","last_fullsized":"2024-09-19T05:43:44.234194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619178+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:43:43.619178+0000","last_peered":"2024-09-19T05:43:43.619178+0000","last_clean":"2024-09-19T05:43:43.619178+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:43:43.619178+0000","last_undegraded":"2024-09-19T05:43:43.619178+0000","last_fullsized":"2024-09-19T05:43:43.619178+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:17.766409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620070+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:43:43.620070+0000","last_peered":"2024-09-19T05:43:43.620070+0000","last_clean":"2024-09-19T05:43:43.620070+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:43:43.620070+0000","last_undegraded":"2024-09-19T05:43:43.620070+0000","last_fullsized":"2024-09-19T05:43:43.620070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":30,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268745+0000","last_change":"2024-09-19T05:43:19.550823+0000","last_active":"2024-09-19T05:43:44.268745+0000","last_peered":"2024-09-19T05:43:44.268745+0000","last_clean":"2024-09-19T05:43:44.268745+0000","last_became_active":"2024-09-19T05:43:19.550475+0000","last_became_peered":"2024-09-19T05:43:19.550475+0000","last_unstale":"2024-09-19T05:43:44.268745+0000","last_undegraded":"2024-09-19T05:43:44.268745+0000","last_fullsized":"2024-09-19T05:43:44.268745+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:00.949531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268003+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:43:44.268003+0000","last_peered":"2024-09-19T05:43:44.268003+0000","last_clean":"2024-09-19T05:43:44.268003+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:43:44.268003+0000","last_undegraded":"2024-09-19T05:43:44.268003+0000","last_fullsized":"2024-09-19T05:43:44.268003+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234447+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:43:44.234447+0000","last_peered":"2024-09-19T05:43:44.234447+0000","last_clean":"2024-09-19T05:43:44.234447+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:43:44.234447+0000","last_undegraded":"2024-09-19T05:43:44.234447+0000","last_fullsized":"2024-09-19T05:43:44.234447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268532+0000","last_change":"2024-09-19T05:43:19.551649+0000","last_active":"2024-09-19T05:43:44.268532+0000","last_peered":"2024-09-19T05:43:44.268532+0000","last_clean":"2024-09-19T05:43:44.268532+0000","last_became_active":"2024-09-19T05:43:19.551457+0000","last_became_peered":"2024-09-19T05:43:19.551457+0000","last_unstale":"2024-09-19T05:43:44.268532+0000","last_undegraded":"2024-09-19T05:43:44.268532+0000","last_fullsized":"2024-09-19T05:43:44.268532+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:23.453548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.474959+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:43:41.474959+0000","last_peered":"2024-09-19T05:43:41.474959+0000","last_clean":"2024-09-19T05:43:41.474959+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:43:41.474959+0000","last_undegraded":"2024-09-19T05:43:41.474959+0000","last_fullsized":"2024-09-19T05:43:41.474959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268204+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:43:44.268204+0000","last_peered":"2024-09-19T05:43:44.268204+0000","last_clean":"2024-09-19T05:43:44.268204+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:43:44.268204+0000","last_undegraded":"2024-09-19T05:43:44.268204+0000","last_fullsized":"2024-09-19T05:43:44.268204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.267957+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:43:44.267957+0000","last_peered":"2024-09-19T05:43:44.267957+0000","last_clean":"2024-09-19T05:43:44.267957+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:43:44.267957+0000","last_undegraded":"2024-09-19T05:43:44.267957+0000","last_fullsized":"2024-09-19T05:43:44.267957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268641+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:43:44.268641+0000","last_peered":"2024-09-19T05:43:44.268641+0000","last_clean":"2024-09-19T05:43:44.268641+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:43:44.268641+0000","last_undegraded":"2024-09-19T05:43:44.268641+0000","last_fullsized":"2024-09-19T05:43:44.268641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.233919+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:43:44.233919+0000","last_peered":"2024-09-19T05:43:44.233919+0000","last_clean":"2024-09-19T05:43:44.233919+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:43:44.233919+0000","last_undegraded":"2024-09-19T05:43:44.233919+0000","last_fullsized":"2024-09-19T05:43:44.233919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620270+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:43:43.620270+0000","last_peered":"2024-09-19T05:43:43.620270+0000","last_clean":"2024-09-19T05:43:43.620270+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:43:43.620270+0000","last_undegraded":"2024-09-19T05:43:43.620270+0000","last_fullsized":"2024-09-19T05:43:43.620270+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.473845+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:43:41.473845+0000","last_peered":"2024-09-19T05:43:41.473845+0000","last_clean":"2024-09-19T05:43:41.473845+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:43:41.473845+0000","last_undegraded":"2024-09-19T05:43:41.473845+0000","last_fullsized":"2024-09-19T05:43:41.473845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268664+0000","last_change":"2024-09-19T05:43:19.548033+0000","last_active":"2024-09-19T05:43:44.268664+0000","last_peered":"2024-09-19T05:43:44.268664+0000","last_clean":"2024-09-19T05:43:44.268664+0000","last_became_active":"2024-09-19T05:43:19.547770+0000","last_became_peered":"2024-09-19T05:43:19.547770+0000","last_unstale":"2024-09-19T05:43:44.268664+0000","last_undegraded":"2024-09-19T05:43:44.268664+0000","last_fullsized":"2024-09-19T05:43:44.268664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268684+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:43:44.268684+0000","last_peered":"2024-09-19T05:43:44.268684+0000","last_clean":"2024-09-19T05:43:44.268684+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:43:44.268684+0000","last_undegraded":"2024-09-19T05:43:44.268684+0000","last_fullsized":"2024-09-19T05:43:44.268684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620159+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:43:43.620159+0000","last_peered":"2024-09-19T05:43:43.620159+0000","last_clean":"2024-09-19T05:43:43.620159+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:43:43.620159+0000","last_undegraded":"2024-09-19T05:43:43.620159+0000","last_fullsized":"2024-09-19T05:43:43.620159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.233752+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:43:44.233752+0000","last_peered":"2024-09-19T05:43:44.233752+0000","last_clean":"2024-09-19T05:43:44.233752+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:43:44.233752+0000","last_undegraded":"2024-09-19T05:43:44.233752+0000","last_fullsized":"2024-09-19T05:43:44.233752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":67,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.267773+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:43:44.267773+0000","last_peered":"2024-09-19T05:43:44.267773+0000","last_clean":"2024-09-19T05:43:44.267773+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:43:44.267773+0000","last_undegraded":"2024-09-19T05:43:44.267773+0000","last_fullsized":"2024-09-19T05:43:44.267773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268326+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:43:44.268326+0000","last_peered":"2024-09-19T05:43:44.268326+0000","last_clean":"2024-09-19T05:43:44.268326+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:43:44.268326+0000","last_undegraded":"2024-09-19T05:43:44.268326+0000","last_fullsized":"2024-09-19T05:43:44.268326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":78,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234506+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:43:44.234506+0000","last_peered":"2024-09-19T05:43:44.234506+0000","last_clean":"2024-09-19T05:43:44.234506+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:43:44.234506+0000","last_undegraded":"2024-09-19T05:43:44.234506+0000","last_fullsized":"2024-09-19T05:43:44.234506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620147+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:43:43.620147+0000","last_peered":"2024-09-19T05:43:43.620147+0000","last_clean":"2024-09-19T05:43:43.620147+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:43:43.620147+0000","last_undegraded":"2024-09-19T05:43:43.620147+0000","last_fullsized":"2024-09-19T05:43:43.620147+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":30,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.269270+0000","last_change":"2024-09-19T05:43:19.550087+0000","last_active":"2024-09-19T05:43:44.269270+0000","last_peered":"2024-09-19T05:43:44.269270+0000","last_clean":"2024-09-19T05:43:44.269270+0000","last_became_active":"2024-09-19T05:43:19.549808+0000","last_became_peered":"2024-09-19T05:43:19.549808+0000","last_unstale":"2024-09-19T05:43:44.269270+0000","last_undegraded":"2024-09-19T05:43:44.269270+0000","last_fullsized":"2024-09-19T05:43:44.269270+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:34.469388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.475044+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:43:41.475044+0000","last_peered":"2024-09-19T05:43:41.475044+0000","last_clean":"2024-09-19T05:43:41.475044+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:43:41.475044+0000","last_undegraded":"2024-09-19T05:43:41.475044+0000","last_fullsized":"2024-09-19T05:43:41.475044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":78,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619688+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:43:43.619688+0000","last_peered":"2024-09-19T05:43:43.619688+0000","last_clean":"2024-09-19T05:43:43.619688+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:43:43.619688+0000","last_undegraded":"2024-09-19T05:43:43.619688+0000","last_fullsized":"2024-09-19T05:43:43.619688+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:39.063921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268691+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:43:44.268691+0000","last_peered":"2024-09-19T05:43:44.268691+0000","last_clean":"2024-09-19T05:43:44.268691+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:43:44.268691+0000","last_undegraded":"2024-09-19T05:43:44.268691+0000","last_fullsized":"2024-09-19T05:43:44.268691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620630+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:43:43.620630+0000","last_peered":"2024-09-19T05:43:43.620630+0000","last_clean":"2024-09-19T05:43:43.620630+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:43:43.620630+0000","last_undegraded":"2024-09-19T05:43:43.620630+0000","last_fullsized":"2024-09-19T05:43:43.620630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.233957+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:43:44.233957+0000","last_peered":"2024-09-19T05:43:44.233957+0000","last_clean":"2024-09-19T05:43:44.233957+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:43:44.233957+0000","last_undegraded":"2024-09-19T05:43:44.233957+0000","last_fullsized":"2024-09-19T05:43:44.233957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"42'10","reported_seq":76,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.474636+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:43:41.474636+0000","last_peered":"2024-09-19T05:43:41.474636+0000","last_clean":"2024-09-19T05:43:41.474636+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:43:41.474636+0000","last_undegraded":"2024-09-19T05:43:41.474636+0000","last_fullsized":"2024-09-19T05:43:41.474636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6662,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619406+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:43:43.619406+0000","last_peered":"2024-09-19T05:43:43.619406+0000","last_clean":"2024-09-19T05:43:43.619406+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:43:43.619406+0000","last_undegraded":"2024-09-19T05:43:43.619406+0000","last_fullsized":"2024-09-19T05:43:43.619406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620451+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:43:43.620451+0000","last_peered":"2024-09-19T05:43:43.620451+0000","last_clean":"2024-09-19T05:43:43.620451+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:43:43.620451+0000","last_undegraded":"2024-09-19T05:43:43.620451+0000","last_fullsized":"2024-09-19T05:43:43.620451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268409+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:43:44.268409+0000","last_peered":"2024-09-19T05:43:44.268409+0000","last_clean":"2024-09-19T05:43:44.268409+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:43:44.268409+0000","last_undegraded":"2024-09-19T05:43:44.268409+0000","last_fullsized":"2024-09-19T05:43:44.268409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.233628+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:43:44.233628+0000","last_peered":"2024-09-19T05:43:44.233628+0000","last_clean":"2024-09-19T05:43:44.233628+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:43:44.233628+0000","last_undegraded":"2024-09-19T05:43:44.233628+0000","last_fullsized":"2024-09-19T05:43:44.233628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234605+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:43:44.234605+0000","last_peered":"2024-09-19T05:43:44.234605+0000","last_clean":"2024-09-19T05:43:44.234605+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:43:44.234605+0000","last_undegraded":"2024-09-19T05:43:44.234605+0000","last_fullsized":"2024-09-19T05:43:44.234605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.618704+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:43:43.618704+0000","last_peered":"2024-09-19T05:43:43.618704+0000","last_clean":"2024-09-19T05:43:43.618704+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:43:43.618704+0000","last_undegraded":"2024-09-19T05:43:43.618704+0000","last_fullsized":"2024-09-19T05:43:43.618704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268241+0000","last_change":"2024-09-19T05:43:19.546195+0000","last_active":"2024-09-19T05:43:44.268241+0000","last_peered":"2024-09-19T05:43:44.268241+0000","last_clean":"2024-09-19T05:43:44.268241+0000","last_became_active":"2024-09-19T05:43:19.545797+0000","last_became_peered":"2024-09-19T05:43:19.545797+0000","last_unstale":"2024-09-19T05:43:44.268241+0000","last_undegraded":"2024-09-19T05:43:44.268241+0000","last_fullsized":"2024-09-19T05:43:44.268241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620259+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:43:43.620259+0000","last_peered":"2024-09-19T05:43:43.620259+0000","last_clean":"2024-09-19T05:43:43.620259+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:43:43.620259+0000","last_undegraded":"2024-09-19T05:43:43.620259+0000","last_fullsized":"2024-09-19T05:43:43.620259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.233766+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:43:44.233766+0000","last_peered":"2024-09-19T05:43:44.233766+0000","last_clean":"2024-09-19T05:43:44.233766+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:43:44.233766+0000","last_undegraded":"2024-09-19T05:43:44.233766+0000","last_fullsized":"2024-09-19T05:43:44.233766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619431+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:43:43.619431+0000","last_peered":"2024-09-19T05:43:43.619431+0000","last_clean":"2024-09-19T05:43:43.619431+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:43:43.619431+0000","last_undegraded":"2024-09-19T05:43:43.619431+0000","last_fullsized":"2024-09-19T05:43:43.619431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.475128+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:43:41.475128+0000","last_peered":"2024-09-19T05:43:41.475128+0000","last_clean":"2024-09-19T05:43:41.475128+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:43:41.475128+0000","last_undegraded":"2024-09-19T05:43:41.475128+0000","last_fullsized":"2024-09-19T05:43:41.475128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620483+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:43:43.620483+0000","last_peered":"2024-09-19T05:43:43.620483+0000","last_clean":"2024-09-19T05:43:43.620483+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:43:43.620483+0000","last_undegraded":"2024-09-19T05:43:43.620483+0000","last_fullsized":"2024-09-19T05:43:43.620483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619337+0000","last_change":"2024-09-19T05:43:19.552783+0000","last_active":"2024-09-19T05:43:43.619337+0000","last_peered":"2024-09-19T05:43:43.619337+0000","last_clean":"2024-09-19T05:43:43.619337+0000","last_became_active":"2024-09-19T05:43:19.552078+0000","last_became_peered":"2024-09-19T05:43:19.552078+0000","last_unstale":"2024-09-19T05:43:43.619337+0000","last_undegraded":"2024-09-19T05:43:43.619337+0000","last_fullsized":"2024-09-19T05:43:43.619337+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":87,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268812+0000","last_change":"2024-09-19T05:43:19.556467+0000","last_active":"2024-09-19T05:43:44.268812+0000","last_peered":"2024-09-19T05:43:44.268812+0000","last_clean":"2024-09-19T05:43:44.268812+0000","last_became_active":"2024-09-19T05:43:19.553271+0000","last_became_peered":"2024-09-19T05:43:19.553271+0000","last_unstale":"2024-09-19T05:43:44.268812+0000","last_undegraded":"2024-09-19T05:43:44.268812+0000","last_fullsized":"2024-09-19T05:43:44.268812+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:05.953596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":140,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619399+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:43:43.619399+0000","last_peered":"2024-09-19T05:43:43.619399+0000","last_clean":"2024-09-19T05:43:43.619399+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:43:43.619399+0000","last_undegraded":"2024-09-19T05:43:43.619399+0000","last_fullsized":"2024-09-19T05:43:43.619399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":67,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.267775+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:43:44.267775+0000","last_peered":"2024-09-19T05:43:44.267775+0000","last_clean":"2024-09-19T05:43:44.267775+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:43:44.267775+0000","last_undegraded":"2024-09-19T05:43:44.267775+0000","last_fullsized":"2024-09-19T05:43:44.267775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234110+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:43:44.234110+0000","last_peered":"2024-09-19T05:43:44.234110+0000","last_clean":"2024-09-19T05:43:44.234110+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:43:44.234110+0000","last_undegraded":"2024-09-19T05:43:44.234110+0000","last_fullsized":"2024-09-19T05:43:44.234110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268479+0000","last_change":"2024-09-19T05:43:22.550909+0000","last_active":"2024-09-19T05:43:44.268479+0000","last_peered":"2024-09-19T05:43:44.268479+0000","last_clean":"2024-09-19T05:43:44.268479+0000","last_became_active":"2024-09-19T05:43:22.550525+0000","last_became_peered":"2024-09-19T05:43:22.550525+0000","last_unstale":"2024-09-19T05:43:44.268479+0000","last_undegraded":"2024-09-19T05:43:44.268479+0000","last_fullsized":"2024-09-19T05:43:44.268479+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:54.373647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234781+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:43:44.234781+0000","last_peered":"2024-09-19T05:43:44.234781+0000","last_clean":"2024-09-19T05:43:44.234781+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:43:44.234781+0000","last_undegraded":"2024-09-19T05:43:44.234781+0000","last_fullsized":"2024-09-19T05:43:44.234781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620057+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:43:43.620057+0000","last_peered":"2024-09-19T05:43:43.620057+0000","last_clean":"2024-09-19T05:43:43.620057+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:43:43.620057+0000","last_undegraded":"2024-09-19T05:43:43.620057+0000","last_fullsized":"2024-09-19T05:43:43.620057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234543+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:43:44.234543+0000","last_peered":"2024-09-19T05:43:44.234543+0000","last_clean":"2024-09-19T05:43:44.234543+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:43:44.234543+0000","last_undegraded":"2024-09-19T05:43:44.234543+0000","last_fullsized":"2024-09-19T05:43:44.234543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.267892+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:43:44.267892+0000","last_peered":"2024-09-19T05:43:44.267892+0000","last_clean":"2024-09-19T05:43:44.267892+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:43:44.267892+0000","last_undegraded":"2024-09-19T05:43:44.267892+0000","last_fullsized":"2024-09-19T05:43:44.267892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620372+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:43:43.620372+0000","last_peered":"2024-09-19T05:43:43.620372+0000","last_clean":"2024-09-19T05:43:43.620372+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:43:43.620372+0000","last_undegraded":"2024-09-19T05:43:43.620372+0000","last_fullsized":"2024-09-19T05:43:43.620372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268185+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:43:44.268185+0000","last_peered":"2024-09-19T05:43:44.268185+0000","last_clean":"2024-09-19T05:43:44.268185+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:43:44.268185+0000","last_undegraded":"2024-09-19T05:43:44.268185+0000","last_fullsized":"2024-09-19T05:43:44.268185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234120+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:43:44.234120+0000","last_peered":"2024-09-19T05:43:44.234120+0000","last_clean":"2024-09-19T05:43:44.234120+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:43:44.234120+0000","last_undegraded":"2024-09-19T05:43:44.234120+0000","last_fullsized":"2024-09-19T05:43:44.234120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619086+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:43:43.619086+0000","last_peered":"2024-09-19T05:43:43.619086+0000","last_clean":"2024-09-19T05:43:43.619086+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:43:43.619086+0000","last_undegraded":"2024-09-19T05:43:43.619086+0000","last_fullsized":"2024-09-19T05:43:43.619086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268688+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:43:44.268688+0000","last_peered":"2024-09-19T05:43:44.268688+0000","last_clean":"2024-09-19T05:43:44.268688+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:43:44.268688+0000","last_undegraded":"2024-09-19T05:43:44.268688+0000","last_fullsized":"2024-09-19T05:43:44.268688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-19T05:43:41.474747+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:43:41.474747+0000","last_peered":"2024-09-19T05:43:41.474747+0000","last_clean":"2024-09-19T05:43:41.474747+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:43:41.474747+0000","last_undegraded":"2024-09-19T05:43:41.474747+0000","last_fullsized":"2024-09-19T05:43:41.474747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.233858+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:43:44.233858+0000","last_peered":"2024-09-19T05:43:44.233858+0000","last_clean":"2024-09-19T05:43:44.233858+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:43:44.233858+0000","last_undegraded":"2024-09-19T05:43:44.233858+0000","last_fullsized":"2024-09-19T05:43:44.233858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268039+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:43:44.268039+0000","last_peered":"2024-09-19T05:43:44.268039+0000","last_clean":"2024-09-19T05:43:44.268039+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:43:44.268039+0000","last_undegraded":"2024-09-19T05:43:44.268039+0000","last_fullsized":"2024-09-19T05:43:44.268039+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"44'4","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620569+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:43:43.620569+0000","last_peered":"2024-09-19T05:43:43.620569+0000","last_clean":"2024-09-19T05:43:43.620569+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:43:43.620569+0000","last_undegraded":"2024-09-19T05:43:43.620569+0000","last_fullsized":"2024-09-19T05:43:43.620569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620665+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:43:43.620665+0000","last_peered":"2024-09-19T05:43:43.620665+0000","last_clean":"2024-09-19T05:43:43.620665+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:43:43.620665+0000","last_undegraded":"2024-09-19T05:43:43.620665+0000","last_fullsized":"2024-09-19T05:43:43.620665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234306+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:43:44.234306+0000","last_peered":"2024-09-19T05:43:44.234306+0000","last_clean":"2024-09-19T05:43:44.234306+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:43:44.234306+0000","last_undegraded":"2024-09-19T05:43:44.234306+0000","last_fullsized":"2024-09-19T05:43:44.234306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620398+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:43:43.620398+0000","last_peered":"2024-09-19T05:43:43.620398+0000","last_clean":"2024-09-19T05:43:43.620398+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:43:43.620398+0000","last_undegraded":"2024-09-19T05:43:43.620398+0000","last_fullsized":"2024-09-19T05:43:43.620398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.618434+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:43:43.618434+0000","last_peered":"2024-09-19T05:43:43.618434+0000","last_clean":"2024-09-19T05:43:43.618434+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:43:43.618434+0000","last_undegraded":"2024-09-19T05:43:43.618434+0000","last_fullsized":"2024-09-19T05:43:43.618434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.233861+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:43:44.233861+0000","last_peered":"2024-09-19T05:43:44.233861+0000","last_clean":"2024-09-19T05:43:44.233861+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:43:44.233861+0000","last_undegraded":"2024-09-19T05:43:44.233861+0000","last_fullsized":"2024-09-19T05:43:44.233861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":67,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620228+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:43:43.620228+0000","last_peered":"2024-09-19T05:43:43.620228+0000","last_clean":"2024-09-19T05:43:43.620228+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:43:43.620228+0000","last_undegraded":"2024-09-19T05:43:43.620228+0000","last_fullsized":"2024-09-19T05:43:43.620228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619931+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:43:43.619931+0000","last_peered":"2024-09-19T05:43:43.619931+0000","last_clean":"2024-09-19T05:43:43.619931+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:43:43.619931+0000","last_undegraded":"2024-09-19T05:43:43.619931+0000","last_fullsized":"2024-09-19T05:43:43.619931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":67,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268495+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:43:44.268495+0000","last_peered":"2024-09-19T05:43:44.268495+0000","last_clean":"2024-09-19T05:43:44.268495+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:43:44.268495+0000","last_undegraded":"2024-09-19T05:43:44.268495+0000","last_fullsized":"2024-09-19T05:43:44.268495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620363+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:43:43.620363+0000","last_peered":"2024-09-19T05:43:43.620363+0000","last_clean":"2024-09-19T05:43:43.620363+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:43:43.620363+0000","last_undegraded":"2024-09-19T05:43:43.620363+0000","last_fullsized":"2024-09-19T05:43:43.620363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":70,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234692+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:43:44.234692+0000","last_peered":"2024-09-19T05:43:44.234692+0000","last_clean":"2024-09-19T05:43:44.234692+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:43:44.234692+0000","last_undegraded":"2024-09-19T05:43:44.234692+0000","last_fullsized":"2024-09-19T05:43:44.234692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234010+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:43:44.234010+0000","last_peered":"2024-09-19T05:43:44.234010+0000","last_clean":"2024-09-19T05:43:44.234010+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:43:44.234010+0000","last_undegraded":"2024-09-19T05:43:44.234010+0000","last_fullsized":"2024-09-19T05:43:44.234010+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.234024+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:43:44.234024+0000","last_peered":"2024-09-19T05:43:44.234024+0000","last_clean":"2024-09-19T05:43:44.234024+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:43:44.234024+0000","last_undegraded":"2024-09-19T05:43:44.234024+0000","last_fullsized":"2024-09-19T05:43:44.234024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619384+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:43:43.619384+0000","last_peered":"2024-09-19T05:43:43.619384+0000","last_clean":"2024-09-19T05:43:43.619384+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:43:43.619384+0000","last_undegraded":"2024-09-19T05:43:43.619384+0000","last_fullsized":"2024-09-19T05:43:43.619384+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:59.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.269025+0000","last_change":"2024-09-19T05:43:19.549176+0000","last_active":"2024-09-19T05:43:44.269025+0000","last_peered":"2024-09-19T05:43:44.269025+0000","last_clean":"2024-09-19T05:43:44.269025+0000","last_became_active":"2024-09-19T05:43:19.548893+0000","last_became_peered":"2024-09-19T05:43:19.548893+0000","last_unstale":"2024-09-19T05:43:44.269025+0000","last_undegraded":"2024-09-19T05:43:44.269025+0000","last_fullsized":"2024-09-19T05:43:44.269025+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.619500+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:43:43.619500+0000","last_peered":"2024-09-19T05:43:43.619500+0000","last_clean":"2024-09-19T05:43:43.619500+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:43:43.619500+0000","last_undegraded":"2024-09-19T05:43:43.619500+0000","last_fullsized":"2024-09-19T05:43:43.619500+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:52.607339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":76,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268610+0000","last_change":"2024-09-19T05:43:19.550136+0000","last_active":"2024-09-19T05:43:44.268610+0000","last_peered":"2024-09-19T05:43:44.268610+0000","last_clean":"2024-09-19T05:43:44.268610+0000","last_became_active":"2024-09-19T05:43:19.549913+0000","last_became_peered":"2024-09-19T05:43:19.549913+0000","last_unstale":"2024-09-19T05:43:44.268610+0000","last_undegraded":"2024-09-19T05:43:44.268610+0000","last_fullsized":"2024-09-19T05:43:44.268610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:58.288341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":69,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268023+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:43:44.268023+0000","last_peered":"2024-09-19T05:43:44.268023+0000","last_clean":"2024-09-19T05:43:44.268023+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:43:44.268023+0000","last_undegraded":"2024-09-19T05:43:44.268023+0000","last_fullsized":"2024-09-19T05:43:44.268023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:44.268347+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:43:44.268347+0000","last_peered":"2024-09-19T05:43:44.268347+0000","last_clean":"2024-09-19T05:43:44.268347+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:43:44.268347+0000","last_undegraded":"2024-09-19T05:43:44.268347+0000","last_fullsized":"2024-09-19T05:43:44.268347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":71,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T05:43:43.620754+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:43:43.620754+0000","last_peered":"2024-09-19T05:43:43.620754+0000","last_clean":"2024-09-19T05:43:43.620754+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:43:43.620754+0000","last_undegraded":"2024-09-19T05:43:43.620754+0000","last_fullsized":"2024-09-19T05:43:43.620754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7460,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":140,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":14920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46,"ondisk_log_size":46,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":561152,"data_stored":240750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":240121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":239997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":830457,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":247331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":234670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":610304,"data_stored":830435,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":565248,"data_stored":240053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":7334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:43:46.522 INFO:tasks.thrashosds.thrasher:Setting 3.6 to [1, 3, 2, 7] 2024-09-19T05:43:46.522 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.6', '1', '3', '2', '7'] 2024-09-19T05:43:46.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.6 1 3 2 7 2024-09-19T05:43:46.573 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:46.572+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:46.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.624+0000 7fb9677fe640 1 -- 172.21.15.2:0/3619390168 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v50) ==== 101+0+0 (secure 0 0 0) 0x7fb974059190 con 0x7fb9800c71c0 2024-09-19T05:43:46.627 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb96c06c2b0 msgr2=0x7fb96c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb96c06c2b0 0x7fb96c06e770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb970016f90 tx=0x7fb970018040 comp rx=0 tx=0).stop 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9800c71c0 msgr2=0x7fb9801ec2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9800c71c0 0x7fb9801ec2a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb97400e810 tx=0x7fb97400ecd0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 shutdown_connections 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb96c06c2b0 0x7fb96c06e770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb980152d30 0x7fb9801ec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9800c71c0 0x7fb9801ec2a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 --2- 172.21.15.2:0/3619390168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9800c6de0 0x7fb98013c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 >> 172.21.15.2:0/3619390168 conn(0x7fb98006f3d0 msgr2=0x7fb98014d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.629+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 shutdown_connections 2024-09-19T05:43:46.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.630+0000 7fb9657fa640 1 -- 172.21.15.2:0/3619390168 wait complete. 2024-09-19T05:43:46.686 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:46.684+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.705+0000 7f795268d640 1 Processor -- start 2024-09-19T05:43:46.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.705+0000 7f795268d640 1 -- start start 2024-09-19T05:43:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f795268d640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f794c152950 0x7f794c0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f795268d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f794c15c880 0x7f794c06f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f795268d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f794c0ce850 con 0x7f794c0c5400 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f7950c2a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f794c15c880 0x7f794c06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f7950c2a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f794c15c880 0x7f794c06f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59248/0 (socket says 172.21.15.2:59248) 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f7950c2a640 1 -- 172.21.15.2:0/3121469162 learned_addr learned my addr 172.21.15.2:0/3121469162 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f795268d640 1 -- 172.21.15.2:0/3121469162 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f794c0ceb30 con 0x7f794c15c880 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f795268d640 1 -- 172.21.15.2:0/3121469162 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f794c0cee10 con 0x7f794c152950 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.707+0000 7f794bfff640 1 --2- 172.21.15.2:0/3121469162 >> v2:172.21.15.123:3300/0 conn(0x7f794c152950 0x7f794c0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.708+0000 7f7950c2a640 1 -- 172.21.15.2:0/3121469162 >> v2:172.21.15.123:3300/0 conn(0x7f794c152950 msgr2=0x7f794c0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.708+0000 7f7950c2a640 1 --2- 172.21.15.2:0/3121469162 >> v2:172.21.15.123:3300/0 conn(0x7f794c152950 0x7f794c0ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.708+0000 7f7950c2a640 1 -- 172.21.15.2:0/3121469162 >> v1:172.21.15.123:6789/0 conn(0x7f794c0c5400 legacy=0x7f794c0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.708+0000 7f7950c2a640 1 -- 172.21.15.2:0/3121469162 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c15bda0 con 0x7f794c15c880 2024-09-19T05:43:46.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.708+0000 7f794bfff640 1 --2- 172.21.15.2:0/3121469162 >> v2:172.21.15.123:3300/0 conn(0x7f794c152950 0x7f794c0ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:46.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.709+0000 7f7950c2a640 1 --2- 172.21.15.2:0/3121469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c15c880 0x7f794c06f970 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f794c06f900 tx=0x7f794402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802a21329b2dc5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:46.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.709+0000 7f794affd640 1 -- 172.21.15.2:0/3121469162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944036070 con 0x7f794c15c880 2024-09-19T05:43:46.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.709+0000 7f794affd640 1 -- 172.21.15.2:0/3121469162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7944031600 con 0x7f794c15c880 2024-09-19T05:43:46.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.709+0000 7f794affd640 1 -- 172.21.15.2:0/3121469162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79440319f0 con 0x7f794c15c880 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.710+0000 7f795268d640 1 -- 172.21.15.2:0/3121469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c15c880 msgr2=0x7f794c06f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.710+0000 7f795268d640 1 --2- 172.21.15.2:0/3121469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c15c880 0x7f794c06f970 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f794c06f900 tx=0x7f794402a7c0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.710+0000 7f795268d640 1 -- 172.21.15.2:0/3121469162 shutdown_connections 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.710+0000 7f795268d640 1 --2- 172.21.15.2:0/3121469162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c15c880 0x7f794c06f970 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.710+0000 7f795268d640 1 --2- 172.21.15.2:0/3121469162 >> v2:172.21.15.123:3300/0 conn(0x7f794c152950 0x7f794c0ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.710+0000 7f795268d640 1 -- 172.21.15.2:0/3121469162 >> 172.21.15.2:0/3121469162 conn(0x7f794c06c240 msgr2=0x7f794c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.710+0000 7f795268d640 1 -- 172.21.15.2:0/3121469162 shutdown_connections 2024-09-19T05:43:46.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.711+0000 7f795268d640 1 -- 172.21.15.2:0/3121469162 wait complete. 2024-09-19T05:43:46.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.713+0000 7f795268d640 1 Processor -- start 2024-09-19T05:43:46.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.713+0000 7f795268d640 1 -- start start 2024-09-19T05:43:46.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.714+0000 7f795268d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f794c0c5400 0x7f794c1ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.714+0000 7f795268d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c152950 0x7f794c1e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f794b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c152950 0x7f794c1e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f794bfff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f794c0c5400 0x7f794c1ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.717 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f794bfff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f794c0c5400 0x7f794c1ee950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:43816/0 (socket says 172.21.15.2:43816) 2024-09-19T05:43:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f794b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c152950 0x7f794c1e4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59264/0 (socket says 172.21.15.2:59264) 2024-09-19T05:43:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f794bfff640 1 -- 172.21.15.2:0/3453270533 learned_addr learned my addr 172.21.15.2:0/3453270533 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f795268d640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c15c880 0x7f794c1e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f794c0ce4f0 con 0x7f794c152950 2024-09-19T05:43:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f794c15c360 con 0x7f794c15c880 2024-09-19T05:43:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f794c152d30 con 0x7f794c0c5400 2024-09-19T05:43:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.715+0000 7f7950c2a640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c15c880 0x7f794c1e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f794bfff640 1 -- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c15c880 msgr2=0x7f794c1e3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f794bfff640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c15c880 0x7f794c1e3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f794bfff640 1 -- 172.21.15.2:0/3453270533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c152950 msgr2=0x7f794c1e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:46.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f794bfff640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c152950 0x7f794c1e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:46.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f794bfff640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c1f2460 con 0x7f794c0c5400 2024-09-19T05:43:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f794b7fe640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c152950 0x7f794c1e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f794bfff640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f794c0c5400 0x7f794c1ee950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f793c009e50 tx=0x7f793c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f7948ff9640 1 -- 172.21.15.2:0/3453270533 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c01f020 con 0x7f794c0c5400 2024-09-19T05:43:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f7948ff9640 1 -- 172.21.15.2:0/3453270533 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793c012230 con 0x7f794c0c5400 2024-09-19T05:43:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c1ee190 con 0x7f794c0c5400 2024-09-19T05:43:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f7948ff9640 1 -- 172.21.15.2:0/3453270533 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00cea0 con 0x7f794c0c5400 2024-09-19T05:43:46.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.716+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c1f2180 con 0x7f794c0c5400 2024-09-19T05:43:46.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.718+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794c1edfb0 con 0x7f794c0c5400 2024-09-19T05:43:46.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.719+0000 7f7948ff9640 1 -- 172.21.15.2:0/3453270533 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f793c022020 con 0x7f794c0c5400 2024-09-19T05:43:46.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.719+0000 7f7948ff9640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f793806c2b0 0x7f793806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:46.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.719+0000 7f7948ff9640 1 -- 172.21.15.2:0/3453270533 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f793c093b50 con 0x7f794c0c5400 2024-09-19T05:43:46.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.720+0000 7f794b7fe640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f793806c2b0 0x7f793806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:46.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.720+0000 7f794b7fe640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f793806c2b0 0x7f793806e770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f794c069570 tx=0x7f7940009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:46.725 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:46.725 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:46.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:46.724+0000 7f7948ff9640 1 -- 172.21.15.2:0/3453270533 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f794c1edfb0 con 0x7f794c0c5400 2024-09-19T05:43:46.738 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:46.774 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:46.772+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:46.874 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:46.873+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:46.895 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:46.895 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:46.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:46.975 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:46.973+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:47.031 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:47.031 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:47.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:47.087 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:47.085+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:47.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.102+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "3.6", "id": [1, 3, 2, 7]} v 0) -- 0x7f794c070540 con 0x7f794c0c5400 2024-09-19T05:43:47.161 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:47.162 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:47.171 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:47.176 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:47.174+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:47.277 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:47.276+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:47.318 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:47.318 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:47.327 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:47.412 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:47.410+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:47.470 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:47.471 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:47.471 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:47.478 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:47.477+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:47.479 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:47.579 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:47.577+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:47.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.620+0000 7face3f31640 1 Processor -- start 2024-09-19T05:43:47.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.620+0000 7face3f31640 1 -- start start 2024-09-19T05:43:47.623 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:47.623 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:47.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face3f31640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7facdc0cd4e0 0x7facdc0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face3f31640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd8c0 0x7facdc0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face3f31640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc15ebe0 con 0x7facdc15aa90 2024-09-19T05:43:47.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face3f31640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7facdc15eec0 con 0x7facdc0cd4e0 2024-09-19T05:43:47.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face3f31640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7facdc0b2e00 con 0x7facdc0cd8c0 2024-09-19T05:43:47.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face14cc640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7facdc15aa90 0x7facdc15df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49178/0 (socket says 172.21.15.2:49178) 2024-09-19T05:43:47.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face14cc640 1 -- 172.21.15.2:0/66062671 learned_addr learned my addr 172.21.15.2:0/66062671 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:47.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face1ccd640 1 --2- 172.21.15.2:0/66062671 >> v2:172.21.15.2:3300/0 conn(0x7facdc0cd4e0 0x7facdc0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face24ce640 1 --2- 172.21.15.2:0/66062671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd8c0 0x7facdc0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face24ce640 1 -- 172.21.15.2:0/66062671 >> v2:172.21.15.2:3300/0 conn(0x7facdc0cd4e0 msgr2=0x7facdc0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face24ce640 1 --2- 172.21.15.2:0/66062671 >> v2:172.21.15.2:3300/0 conn(0x7facdc0cd4e0 0x7facdc0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face24ce640 1 -- 172.21.15.2:0/66062671 >> v1:172.21.15.2:6789/0 conn(0x7facdc15aa90 legacy=0x7facdc15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face24ce640 1 -- 172.21.15.2:0/66062671 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0b30e0 con 0x7facdc0cd8c0 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.622+0000 7face1ccd640 1 --2- 172.21.15.2:0/66062671 >> v2:172.21.15.2:3300/0 conn(0x7facdc0cd4e0 0x7facdc0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.623+0000 7face24ce640 1 --2- 172.21.15.2:0/66062671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd8c0 0x7facdc0b4100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facd4009d90 tx=0x7facd402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0aa1f3b7f36b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.624+0000 7face0ccb640 1 -- 172.21.15.2:0/66062671 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4007740 con 0x7facdc0cd8c0 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.624+0000 7face0ccb640 1 -- 172.21.15.2:0/66062671 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd40078e0 con 0x7facdc0cd8c0 2024-09-19T05:43:47.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.624+0000 7face0ccb640 1 -- 172.21.15.2:0/66062671 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4007cb0 con 0x7facdc0cd8c0 2024-09-19T05:43:47.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.624+0000 7face3f31640 1 -- 172.21.15.2:0/66062671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd8c0 msgr2=0x7facdc0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.624+0000 7face3f31640 1 --2- 172.21.15.2:0/66062671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd8c0 0x7facdc0b4100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facd4009d90 tx=0x7facd402a660 comp rx=0 tx=0).stop 2024-09-19T05:43:47.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.625+0000 7face3f31640 1 -- 172.21.15.2:0/66062671 shutdown_connections 2024-09-19T05:43:47.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.625+0000 7face3f31640 1 --2- 172.21.15.2:0/66062671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd8c0 0x7facdc0b4100 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.625+0000 7face3f31640 1 --2- 172.21.15.2:0/66062671 >> v2:172.21.15.2:3300/0 conn(0x7facdc0cd4e0 0x7facdc0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.625+0000 7face3f31640 1 -- 172.21.15.2:0/66062671 >> 172.21.15.2:0/66062671 conn(0x7facdc067b30 msgr2=0x7facdc067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:47.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.625+0000 7face3f31640 1 -- 172.21.15.2:0/66062671 shutdown_connections 2024-09-19T05:43:47.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.625+0000 7face3f31640 1 -- 172.21.15.2:0/66062671 wait complete. 2024-09-19T05:43:47.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.627+0000 7face3f31640 1 Processor -- start 2024-09-19T05:43:47.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.627+0000 7face3f31640 1 -- start start 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face3f31640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd4e0 0x7facdc1f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face3f31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facdc0cd8c0 0x7facdc14d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face3f31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facdc15aa90 0x7facdc14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face3f31640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7facdc15e640 con 0x7facdc0cd8c0 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face3f31640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facdc0b36a0 con 0x7facdc15aa90 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face3f31640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7facdc15e7c0 con 0x7facdc0cd4e0 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face1ccd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd4e0 0x7facdc1f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face1ccd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd4e0 0x7facdc1f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:43830/0 (socket says 172.21.15.2:43830) 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face1ccd640 1 -- 172.21.15.2:0/2684282097 learned_addr learned my addr 172.21.15.2:0/2684282097 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:47.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face24ce640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facdc15aa90 0x7facdc14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.628+0000 7face14cc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facdc0cd8c0 0x7facdc14d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7face1ccd640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facdc15aa90 msgr2=0x7facdc14d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facdc15aa90 0x7facdc14d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7face1ccd640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facdc0cd8c0 msgr2=0x7facdc14d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facdc0cd8c0 0x7facdc14d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7face1ccd640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc15d400 con 0x7facdc0cd4e0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd4e0 0x7facdc1f3c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7facd800ec10 tx=0x7facd800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8017070 con 0x7facdc0cd4e0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd800cc30 con 0x7facdc0cd4e0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.629+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8013470 con 0x7facdc0cd4e0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.630+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc0ceb10 con 0x7facdc0cd4e0 2024-09-19T05:43:47.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.630+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc1584e0 con 0x7facdc0cd4e0 2024-09-19T05:43:47.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.631+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7facdc15df30 con 0x7facdc0cd4e0 2024-09-19T05:43:47.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:47.635 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.632+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7facd801a070 con 0x7facdc0cd4e0 2024-09-19T05:43:47.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.633+0000 7facd27fc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7facc006c2b0 0x7facc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.633+0000 7face14cc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7facc006c2b0 0x7facc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.634+0000 7f7948ff9640 1 -- 172.21.15.2:0/3453270533 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "3.6", "id": [1, 3, 2, 7]}]=0 set 3.6 pg_upmap_items mapping to [1->3,2->7] v51) ==== 146+0+0 (secure 0 0 0) 0x7f793c029030 con 0x7f794c0c5400 2024-09-19T05:43:47.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.634+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(51..51 src has 1..51) ==== 5709+0+0 (secure 0 0 0) 0x7facd8057810 con 0x7facdc0cd4e0 2024-09-19T05:43:47.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.634+0000 7face14cc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7facc006c2b0 0x7facc006e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7facdc153ec0 tx=0x7faccc007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.637 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.6 pg_upmap_items mapping to [1->3,2->7] 2024-09-19T05:43:47.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.635+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v51) ==== 54+0+16 (secure 0 0 0) 0x7facdc15d400 con 0x7facdc0cd4e0 2024-09-19T05:43:47.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.635+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7facdc0cfdf0 0x7facdc1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.635+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facdc0d01d0 con 0x7facdc0cfdf0 2024-09-19T05:43:47.638 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.635+0000 7face24ce640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7facdc0cfdf0 0x7facdc1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.636+0000 7face24ce640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7facdc0cfdf0 0x7facdc1f4150 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc0d01d0 con 0x7facdc0cfdf0 2024-09-19T05:43:47.639 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f793806c2b0 msgr2=0x7f793806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f793806c2b0 0x7f793806e770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f794c069570 tx=0x7f7940009040 comp rx=0 tx=0).stop 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f794c0c5400 msgr2=0x7f794c1ee950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f794c0c5400 0x7f794c1ee950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f793c009e50 tx=0x7f793c00c520 comp rx=0 tx=0).stop 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 shutdown_connections 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f793806c2b0 0x7f793806e770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f794c15c880 0x7f794c1e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f794c152950 0x7f794c1e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 --2- 172.21.15.2:0/3453270533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f794c0c5400 0x7f794c1ee950 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 >> 172.21.15.2:0/3453270533 conn(0x7f794c06c240 msgr2=0x7f794c0cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 shutdown_connections 2024-09-19T05:43:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.637+0000 7f795268d640 1 -- 172.21.15.2:0/3453270533 wait complete. 2024-09-19T05:43:47.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.652+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc1e6bb0 con 0x7facdc0cfdf0 2024-09-19T05:43:47.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.653+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc1e6bb0 con 0x7facdc0cfdf0 2024-09-19T05:43:47.655 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.654+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7facdc062630 0x7facdc062a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.654+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7facdc154640 con 0x7facdc062630 2024-09-19T05:43:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.654+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7facdc062630 0x7facdc062a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.654+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7facdc062630 0x7facdc062a10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.655+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc154640 con 0x7facdc062630 2024-09-19T05:43:47.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.672+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc0c7030 con 0x7facdc062630 2024-09-19T05:43:47.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.673+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc0c7030 con 0x7facdc062630 2024-09-19T05:43:47.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:47.676 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:47.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.674+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7facdc063310 0x7facdc1e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.674+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7facdc1e78e0 con 0x7facdc063310 2024-09-19T05:43:47.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.674+0000 7face14cc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7facdc063310 0x7facdc1e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.674+0000 7face14cc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7facdc063310 0x7facdc1e73a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.675+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc1e78e0 con 0x7facdc063310 2024-09-19T05:43:47.680 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:47.678+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:47.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.689+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc1e7d60 con 0x7facdc063310 2024-09-19T05:43:47.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.690+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc1e7d60 con 0x7facdc063310 2024-09-19T05:43:47.692 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.690+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7facdc02cf10 0x7facdc02d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.690+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7facdc02d890 con 0x7facdc02cf10 2024-09-19T05:43:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.691+0000 7face24ce640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7facdc02cf10 0x7facdc02d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.691+0000 7face24ce640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7facdc02cf10 0x7facdc02d350 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.692+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc02d890 con 0x7facdc02cf10 2024-09-19T05:43:47.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.705+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc1e3d90 con 0x7facdc02cf10 2024-09-19T05:43:47.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.707+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc1e3d90 con 0x7facdc02cf10 2024-09-19T05:43:47.709 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:47.710 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:47.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.707+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7facdc1e44b0 0x7facdc1e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.707+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7facdc1e3d90 con 0x7facdc1e44b0 2024-09-19T05:43:47.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.708+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7facdc1e44b0 0x7facdc1e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.708+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7facdc1e44b0 0x7facdc1e48b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.709+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc1e3d90 con 0x7facdc1e44b0 2024-09-19T05:43:47.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.722+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc1e3d90 con 0x7facdc1e44b0 2024-09-19T05:43:47.726 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.724+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc1e3d90 con 0x7facdc1e44b0 2024-09-19T05:43:47.726 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.724+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7facdc1e5300 0x7facdc206fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.724+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7facdc1e5850 con 0x7facdc1e5300 2024-09-19T05:43:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.724+0000 7face14cc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7facdc1e5300 0x7facdc206fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.725+0000 7face14cc640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7facdc1e5300 0x7facdc206fb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.725+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc1e5850 con 0x7facdc1e5300 2024-09-19T05:43:47.739 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:47.740 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:47.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.739+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc1e58b0 con 0x7facdc1e5300 2024-09-19T05:43:47.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.741+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc1e58b0 con 0x7facdc1e5300 2024-09-19T05:43:47.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:47.743 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:47.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.741+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7facdc207a60 0x7facdc207ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.741+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7facdc20cfc0 con 0x7facdc207a60 2024-09-19T05:43:47.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.741+0000 7face24ce640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7facdc207a60 0x7facdc207ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.742+0000 7face24ce640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7facdc207a60 0x7facdc207ec0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.743+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc20cfc0 con 0x7facdc207a60 2024-09-19T05:43:47.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:47.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.756+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc2084c0 con 0x7facdc207a60 2024-09-19T05:43:47.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.758+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc2084c0 con 0x7facdc207a60 2024-09-19T05:43:47.760 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.758+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7facdc208b40 0x7facdc208fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.758+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7facdc209640 con 0x7facdc208b40 2024-09-19T05:43:47.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.759+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7facdc208b40 0x7facdc208fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:47.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.759+0000 7face1ccd640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7facdc208b40 0x7facdc208fa0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:47.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.760+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facdc209640 con 0x7facdc208b40 2024-09-19T05:43:47.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.774+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facdc075590 con 0x7facdc208b40 2024-09-19T05:43:47.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.775+0000 7facd27fc640 1 -- 172.21.15.2:0/2684282097 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facdc075590 con 0x7facdc208b40 2024-09-19T05:43:47.777 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7facdc0cfdf0 msgr2=0x7facdc1f4150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7facdc0cfdf0 0x7facdc1f4150 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7facdc062630 msgr2=0x7facdc062a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7facdc062630 0x7facdc062a10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7facdc063310 msgr2=0x7facdc1e73a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7facdc063310 0x7facdc1e73a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7facdc02cf10 msgr2=0x7facdc02d350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7facdc02cf10 0x7facdc02d350 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7facdc1e44b0 msgr2=0x7facdc1e48b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7facdc1e44b0 0x7facdc1e48b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7facdc1e5300 msgr2=0x7facdc206fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7facdc1e5300 0x7facdc206fb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7facdc207a60 msgr2=0x7facdc207ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7facdc207a60 0x7facdc207ec0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7facdc208b40 msgr2=0x7facdc208fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7facdc208b40 0x7facdc208fa0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face24ce640 1 -- 172.21.15.2:0/2684282097 reap_dead start 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face24ce640 1 -- 172.21.15.2:0/2684282097 reap_dead start 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7facc006c2b0 msgr2=0x7facc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7facc006c2b0 0x7facc006e770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7facdc153ec0 tx=0x7faccc007e80 comp rx=0 tx=0).stop 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd4e0 msgr2=0x7facdc1f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 --2- 172.21.15.2:0/2684282097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facdc0cd4e0 0x7facdc1f3c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7facd800ec10 tx=0x7facd800c520 comp rx=0 tx=0).stop 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face24ce640 1 -- 172.21.15.2:0/2684282097 reap_dead start 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face24ce640 1 -- 172.21.15.2:0/2684282097 reap_dead start 2024-09-19T05:43:47.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face24ce640 1 -- 172.21.15.2:0/2684282097 reap_dead start 2024-09-19T05:43:47.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 shutdown_connections 2024-09-19T05:43:47.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 >> 172.21.15.2:0/2684282097 conn(0x7facdc067b30 msgr2=0x7facdc159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:47.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 shutdown_connections 2024-09-19T05:43:47.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:47.776+0000 7face3f31640 1 -- 172.21.15.2:0/2684282097 wait complete. 2024-09-19T05:43:47.782 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:47.780+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:54 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'29)", 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.333865838999998, 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058827250000000001, 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.855 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518676+0000", 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518677+0000", 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6799999999999999e-07 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518683+0000", 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.502e-06 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518695+0000", 2024-09-19T05:43:47.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2123000000000001e-05 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518702+0000", 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2110000000000001e-06 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518746+0000", 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4263e-05 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.858 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.518759+0000", 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2986999999999999e-05 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524518+0000", 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057588040000000002 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524559+0000", 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1066999999999998e-05 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.859 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:55 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'29)", 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.333507870999995, 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059035429999999998, 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.860 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519034+0000", 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519035+0000", 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.861 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519099+0000", 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4177999999999994e-05 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519108+0000", 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6950000000000006e-06 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519117+0000", 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6409999999999997e-06 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.862 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519159+0000", 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1961e-05 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519172+0000", 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2867000000000001e-05 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524891+0000", 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057190560000000001 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.863 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524938+0000", 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6551999999999997e-05 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:56 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'29)", 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.333007480000006, 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054228260000000004, 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.864 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.865 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519535+0000", 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1900000000000002e-07 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519550+0000", 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4711e-05 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519558+0000", 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2360000000000004e-06 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.866 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.519562+0000", 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0249999999999996e-06 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521502+0000", 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0019400929999999999 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.521515+0000", 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2578000000000001e-05 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.867 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524940+0000", 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034254070000000001 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.524957+0000", 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7156999999999999e-05 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-19T05:43:47.868 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.326021247, 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042132890000000003, 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.869 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526521+0000", 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8800000000000002e-07 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526604+0000", 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2996000000000003e-05 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.870 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526615+0000", 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0511e-05 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526621+0000", 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8590000000000001e-06 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526658+0000", 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7280999999999998e-05 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.871 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526670+0000", 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1997000000000001e-05 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530695+0000", 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040249789999999997 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530734+0000", 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9078000000000001e-05 2024-09-19T05:43:47.872 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.325563439000007, 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037742489999999999, 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.873 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526979+0000", 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1799999999999995e-07 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526984+0000", 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6210000000000003e-06 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.526992+0000", 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527006+0000", 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:43:47.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527049+0000", 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2713999999999998e-05 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.527063+0000", 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4076e-05 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530737+0000", 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003673707 2024-09-19T05:43:47.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.530753+0000", 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6226999999999999e-05 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:87 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'57, mlcod=17'53)", 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.302849819000002, 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050352469999999996, 2024-09-19T05:43:47.877 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549692+0000", 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549693+0000", 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0200000000000002e-07 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549752+0000", 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9267e-05 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549760+0000", 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4440000000000008e-06 2024-09-19T05:43:47.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549765+0000", 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7690000000000004e-06 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549810+0000", 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4690000000000001e-05 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.549831+0000", 2024-09-19T05:43:47.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000001e-05 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554690+0000", 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048590450000000002 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554728+0000", 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7370000000000003e-05 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:88 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'58, mlcod=17'53)", 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 84.302354874000002, 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045598629999999999, 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550187+0000", 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550188+0000", 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1699999999999998e-07 2024-09-19T05:43:47.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550192+0000", 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.172e-06 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550198+0000", 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550201+0000", 2024-09-19T05:43:47.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550267+0000", 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5769e-05 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.550279+0000", 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2242000000000001e-05 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554729+0000", 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004450168 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:23.554747+0000", 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7591000000000001e-05 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:138 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'96, mlcod=17'90)", 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:47.886 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.740551917999994, 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047245919999999997, 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111990+0000", 2024-09-19T05:43:47.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111991+0000", 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8500000000000002e-07 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111993+0000", 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8669999999999998e-06 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.111998+0000", 2024-09-19T05:43:47.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0409999999999997e-06 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112002+0000", 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112038+0000", 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6437999999999999e-05 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112055+0000", 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915999999999999e-05 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116651+0000", 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045961149999999996 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116715+0000", 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3349000000000006e-05 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.890 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:139 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'97, mlcod=17'92)", 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.739794141999994, 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004002607, 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112748+0000", 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112755+0000", 2024-09-19T05:43:47.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0569999999999998e-06 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112764+0000", 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7620000000000007e-06 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112769+0000", 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6789999999999996e-06 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112803+0000", 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3949999999999999e-05 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.112815+0000", 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.205e-05 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116734+0000", 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039194770000000002 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116751+0000", 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.611e-05 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:140 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'98, mlcod=17'93)", 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.739213094999997, 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037906519999999998, 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.895 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113329+0000", 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113330+0000", 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8500000000000001e-07 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113389+0000", 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9429000000000003e-05 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113398+0000", 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4690000000000003e-06 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113403+0000", 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7849999999999999e-06 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113439+0000", 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6199999999999999e-05 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113451+0000", 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1909000000000001e-05 2024-09-19T05:43:47.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117080+0000", 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036287260000000001 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117120+0000", 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9548999999999997e-05 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:141 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'99, mlcod=17'93)", 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.738578048999997, 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039428980000000002, 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113964+0000", 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113965+0000", 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0900000000000001e-07 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113970+0000", 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4700000000000001e-06 2024-09-19T05:43:47.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113980+0000", 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0056e-05 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.113985+0000", 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7400000000000004e-06 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116773+0000", 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027877129999999998 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.116785+0000", 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2792999999999999e-05 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117882+0000", 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0010963380000000001 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.117907+0000", 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5179000000000001e-05 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:144 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.732771542999998, 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050709850000000001, 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.904 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119771+0000", 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119832+0000", 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0609000000000002e-05 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119838+0000", 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7179999999999999e-06 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119842+0000", 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.557e-06 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119881+0000", 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8651e-05 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.119900+0000", 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9621999999999999e-05 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124812+0000", 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049117240000000001 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124842+0000", 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9641e-05 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.908 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:145 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'103, mlcod=17'99)", 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.732115371000006, 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044299500000000002, 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120427+0000", 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0299999999999999e-07 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120497+0000", 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9641000000000006e-05 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120505+0000", 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1070000000000005e-06 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120509+0000", 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.579e-06 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120539+0000", 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0880999999999998e-05 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.120552+0000", 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2462e-05 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124843+0000", 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042911620000000003 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.124857+0000", 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3614999999999999e-05 2024-09-19T05:43:47.912 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:156 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'110, mlcod=17'105)", 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.255566697000006, 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055850830000000002, 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596975+0000", 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596976+0000", 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7399999999999998e-07 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596980+0000", 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1370000000000004e-06 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596986+0000", 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1920000000000002e-06 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.596990+0000", 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.399e-06 2024-09-19T05:43:47.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597103+0000", 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113806 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597115+0000", 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1528e-05 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602533+0000", 2024-09-19T05:43:47.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054181530000000002 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602560+0000", 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7393999999999999e-05 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:157 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'111, mlcod=17'105)", 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.255198383000007, 2024-09-19T05:43:47.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052319310000000004, 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597344+0000", 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5299999999999999e-07 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597351+0000", 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4269999999999996e-06 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597357+0000", 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7220000000000002e-06 2024-09-19T05:43:47.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597363+0000", 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5470000000000004e-06 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597390+0000", 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7623e-05 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597401+0000", 2024-09-19T05:43:47.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0767e-05 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602562+0000", 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051604349999999997 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3957e-05 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:158 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'112, mlcod=17'105)", 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.254819472999998, 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048646619999999996, 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597723+0000", 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1e-07 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597797+0000", 2024-09-19T05:43:47.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4029999999999994e-05 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597802+0000", 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3170000000000001e-06 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597805+0000", 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9510000000000002e-06 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597833+0000", 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8082e-05 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.597845+0000", 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602576+0000", 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047312439999999999 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602587+0000", 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1009000000000001e-05 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:159 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'113, mlcod=18'107)", 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.254122620000004, 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041827160000000004, 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.926 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598419+0000", 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598420+0000", 2024-09-19T05:43:47.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3000000000000001e-07 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598423+0000", 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5549999999999998e-06 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598430+0000", 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.64e-06 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598433+0000", 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598462+0000", 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8274999999999999e-05 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.598474+0000", 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2224e-05 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602588+0000", 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041140810000000003 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.602602+0000", 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4336e-05 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4172.0:161 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'115, mlcod=18'112)", 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:47.930 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 83.248338005999997, 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003753424, 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604204+0000", 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604272+0000", 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7458000000000004e-05 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604279+0000", 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.63e-06 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604282+0000", 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.298e-06 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604314+0000", 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1844000000000002e-05 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.604327+0000", 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2847000000000001e-05 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607928+0000", 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036014160000000001 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:24.607958+0000", 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9522e-05 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.934 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 69.877222992, 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044897909999999999, 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975319+0000", 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975320+0000", 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7300000000000005e-07 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975325+0000", 2024-09-19T05:43:47.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7500000000000003e-06 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975335+0000", 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0533999999999999e-05 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975342+0000", 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1330000000000001e-06 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975457+0000", 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011431599999999999 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975588+0000", 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013170200000000001 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979809+0000", 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004220583 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.938 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 69.877179909000006, 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040519329999999998, 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:47.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975362+0000", 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975363+0000", 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4700000000000001e-07 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:47.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975366+0000", 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6119999999999999e-06 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975375+0000", 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5990000000000002e-06 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975381+0000", 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1249999999999998e-06 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975453+0000", 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1382000000000002e-05 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975589+0000", 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013633699999999999 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979414+0000", 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038252310000000001 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.942 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:47.943 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:47.943 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:47.944 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:47.945 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:47.943+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:48.045 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:48.044+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:48.089 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:48.089 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:48.097 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:48.181 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:48.180+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:48.241 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:48.241 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:48.241 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:48.246 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:48.245+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:48.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:48.348 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:48.346+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:48.393 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:48.393 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:48.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:48.393 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:48.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.393 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1b epoch 22/19 rep_tid 1 entries 22'1 (0'0) error 2:ddb4e4d7:::200.00000002:head by mds.0.4:33 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:48.393 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.415784044000006, 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001043324, 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976263+0000", 2024-09-19T05:43:48.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976265+0000", 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6980000000000001e-06 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976289+0000", 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4139999999999999e-05 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976340+0000", 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0810000000000003e-05 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3818000000000001e-05 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976463+0000", 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9477999999999999e-05 2024-09-19T05:43:48.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977306+0000", 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00084338000000000002 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.398 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:48.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:48.448 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:48.446+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:48.545 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:48.545 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:48.545 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:48.545 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:48.550 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:48.548+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:48.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:48.645 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:43:48.658 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:43:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:43:48.658 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:48.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:48.689 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:48.687+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:48.751 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:48.749+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:48.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.750+0000 7f524f801640 1 Processor -- start 2024-09-19T05:43:48.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.750+0000 7f524f801640 1 -- start start 2024-09-19T05:43:48.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.751+0000 7f524f801640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5248073660 0x7f52480c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:48.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.751+0000 7f524f801640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52480725c0 con 0x7f52480c86f0 2024-09-19T05:43:48.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.751+0000 7f524f801640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52480728a0 con 0x7f52481584c0 2024-09-19T05:43:48.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.751+0000 7f524f801640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5248072370 con 0x7f5248073660 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.751+0000 7f524dd9e640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f52480c86f0 0x7f524815e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49182/0 (socket says 172.21.15.2:49182) 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.751+0000 7f524dd9e640 1 -- 172.21.15.2:0/2599892950 learned_addr learned my addr 172.21.15.2:0/2599892950 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.751+0000 7f524cd9c640 1 --2- 172.21.15.2:0/2599892950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5248073660 0x7f52480c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.752+0000 7f523ffff640 1 -- 172.21.15.2:0/2599892950 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3909607231 0 0) 0x7f52480725c0 con 0x7f52480c86f0 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.752+0000 7f523ffff640 1 -- 172.21.15.2:0/2599892950 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5234003520 con 0x7f52480c86f0 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.752+0000 7f523ffff640 1 -- 172.21.15.2:0/2599892950 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394627249 0 0) 0x7f52480728a0 con 0x7f52481584c0 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.752+0000 7f523ffff640 1 -- 172.21.15.2:0/2599892950 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52480725c0 con 0x7f52481584c0 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.752+0000 7f524cd9c640 1 -- 172.21.15.2:0/2599892950 >> v1:172.21.15.123:6789/0 conn(0x7f52481584c0 legacy=0x7f524806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.752+0000 7f524cd9c640 1 -- 172.21.15.2:0/2599892950 >> v1:172.21.15.2:6789/0 conn(0x7f52480c86f0 legacy=0x7f524815e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:48.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.752+0000 7f524cd9c640 1 -- 172.21.15.2:0/2599892950 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5248072b80 con 0x7f5248073660 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.753+0000 7f524cd9c640 1 --2- 172.21.15.2:0/2599892950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5248073660 0x7f52480c4400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52380097d0 tx=0x7f523802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96240e5fd9fd38b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.753+0000 7f523ffff640 1 -- 172.21.15.2:0/2599892950 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238037070 con 0x7f5248073660 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.753+0000 7f523ffff640 1 -- 172.21.15.2:0/2599892950 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523802ae50 con 0x7f5248073660 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.753+0000 7f523ffff640 1 -- 172.21.15.2:0/2599892950 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238032640 con 0x7f5248073660 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.753+0000 7f524f801640 1 -- 172.21.15.2:0/2599892950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5248073660 msgr2=0x7f52480c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.754+0000 7f524f801640 1 --2- 172.21.15.2:0/2599892950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5248073660 0x7f52480c4400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f52380097d0 tx=0x7f523802a100 comp rx=0 tx=0).stop 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.754+0000 7f524f801640 1 -- 172.21.15.2:0/2599892950 shutdown_connections 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.754+0000 7f524f801640 1 --2- 172.21.15.2:0/2599892950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5248073660 0x7f52480c4400 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.754+0000 7f524f801640 1 -- 172.21.15.2:0/2599892950 >> 172.21.15.2:0/2599892950 conn(0x7f52480cd470 msgr2=0x7f52480c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:48.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.754+0000 7f524f801640 1 -- 172.21.15.2:0/2599892950 shutdown_connections 2024-09-19T05:43:48.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.754+0000 7f524f801640 1 -- 172.21.15.2:0/2599892950 wait complete. 2024-09-19T05:43:48.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.755+0000 7f524f801640 1 Processor -- start 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.755+0000 7f524f801640 1 -- start start 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524f801640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5248073660 0x7f52481f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524f801640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52480c86f0 0x7f52481f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524f801640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52481584c0 0x7f52480c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524f801640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52481588a0 con 0x7f5248073660 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524f801640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5248074120 con 0x7f52481584c0 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524f801640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5248158a20 con 0x7f52480c86f0 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5248073660 0x7f52481f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5248073660 0x7f52481f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59298/0 (socket says 172.21.15.2:59298) 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 -- 172.21.15.2:0/1339414507 learned_addr learned my addr 172.21.15.2:0/1339414507 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524dd9e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52481584c0 0x7f52480c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524cd9c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52480c86f0 0x7f52481f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 -- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52480c86f0 msgr2=0x7f52481f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52480c86f0 0x7f52481f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 -- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52481584c0 msgr2=0x7f52480c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52481584c0 0x7f52480c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524d59d640 1 -- 172.21.15.2:0/1339414507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5248151b10 con 0x7f5248073660 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.756+0000 7f524dd9e640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52481584c0 0x7f52480c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.757+0000 7f524d59d640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5248073660 0x7f52481f4ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f524400c490 tx=0x7f524400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.757+0000 7f523dffb640 1 -- 172.21.15.2:0/1339414507 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52440052d0 con 0x7f5248073660 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.757+0000 7f523dffb640 1 -- 172.21.15.2:0/1339414507 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5244005470 con 0x7f5248073660 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.757+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5248151df0 con 0x7f5248073660 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.757+0000 7f523dffb640 1 -- 172.21.15.2:0/1339414507 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244012850 con 0x7f5248073660 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.757+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52481f40c0 con 0x7f5248073660 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.758+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52480694c0 con 0x7f5248073660 2024-09-19T05:43:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.758+0000 7f523dffb640 1 -- 172.21.15.2:0/1339414507 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f52440129f0 con 0x7f5248073660 2024-09-19T05:43:48.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.759+0000 7f523dffb640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f522c06c2b0 0x7f522c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:48.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.759+0000 7f523dffb640 1 -- 172.21.15.2:0/1339414507 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5709+0+0 (secure 0 0 0) 0x7f524408b020 con 0x7f5248073660 2024-09-19T05:43:48.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.759+0000 7f524cd9c640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f522c06c2b0 0x7f522c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:48.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.760+0000 7f524cd9c640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f522c06c2b0 0x7f522c06e770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5238034000 tx=0x7f523802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:48.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.763+0000 7f523dffb640 1 -- 172.21.15.2:0/1339414507 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f52480694c0 con 0x7f5248073660 2024-09-19T05:43:48.772 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:43:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:43:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:43:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.772 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.795440838000005, 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062186530000000002, 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974700+0000", 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974705+0000", 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1270000000000002e-06 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974716+0000", 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1306e-05 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974736+0000", 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9293e-05 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974749+0000", 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3059e-05 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974978+0000", 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022932900000000001 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975163+0000", 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018497500000000001 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975314+0000", 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151328 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979256+0000", 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039412179999999998 2024-09-19T05:43:48.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001571296 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980877+0000", 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0387999999999999e-05 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980919+0000", 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1334000000000001e-05 2024-09-19T05:43:48.777 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.795318030000004, 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062599709999999996, 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.778 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974824+0000", 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2500000000000004e-07 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.779 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974829+0000", 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.66e-06 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974844+0000", 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5160000000000001e-05 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974852+0000", 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9570000000000002e-06 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.780 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975748+0000", 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089547100000000001 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975835+0000", 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7190999999999999e-05 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975997+0000", 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016230800000000001 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979261+0000", 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032635310000000001 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980973+0000", 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017117199999999999 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981027+0000", 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4732999999999999e-05 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981083+0000", 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5414999999999997e-05 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:30 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.795257499000002, 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040710980000000004, 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.783 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974883+0000", 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.784 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974884+0000", 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6400000000000001e-07 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974899+0000", 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974910+0000", 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0584000000000001e-05 2024-09-19T05:43:48.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974917+0000", 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9799999999999998e-06 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974994+0000", 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6612000000000004e-05 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975176+0000", 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018201400000000001 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975321+0000", 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014440899999999999 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977845+0000", 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002524409 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978841+0000", 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099560399999999998 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978896+0000", 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5075999999999999e-05 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978954+0000", 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8743999999999999e-05 2024-09-19T05:43:48.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:27 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.795189554000004, 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040869110000000004, 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974951+0000", 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974952+0000", 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974957+0000", 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7260000000000002e-06 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974966+0000", 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974972+0000", 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0970000000000002e-06 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975780+0000", 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00080748199999999999 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975894+0000", 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113867 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976037+0000", 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014333 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978068+0000", 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020306439999999999 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978965+0000", 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00089710699999999996 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979003+0000", 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8797999999999997e-05 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979038+0000", 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4668000000000001e-05 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(mds.0.4:29 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.795157836000001, 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062404959999999999, 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.794 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974983+0000", 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3900000000000005e-07 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.974999+0000", 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5817000000000001e-05 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975006+0000", 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5509999999999996e-06 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.975012+0000", 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8189999999999997e-06 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976240+0000", 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012283870000000001 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976367+0000", 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126465 2024-09-19T05:43:48.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976487+0000", 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120548 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980396+0000", 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003908702 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981127+0000", 2024-09-19T05:43:48.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00073081199999999995 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981182+0000", 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5921999999999998e-05 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981223+0000", 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0933999999999998e-05 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.799 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "pg_update_log_missing(2.1e epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:7bf7e7b5:::200.00000006:head by mds.0.4:37 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.794032381999997, 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028842130000000001, 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "reached pg", 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.800 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976108+0000", 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976110+0000", 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2869999999999999e-06 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976123+0000", 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3224000000000001e-05 2024-09-19T05:43:48.801 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976140+0000", 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.755e-05 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976157+0000", 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6464e-05 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.976275+0000", 2024-09-19T05:43:48.802 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117602 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978993+0000", 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0027180860000000002 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.792825504000007, 2024-09-19T05:43:48.803 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001691115, 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977315+0000", 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.804 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977316+0000", 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1209999999999999e-06 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977337+0000", 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0890000000000002e-05 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977356+0000", 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9032999999999999e-05 2024-09-19T05:43:48.805 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977365+0000", 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7649999999999999e-06 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977425+0000", 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0044999999999999e-05 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977449+0000", 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4006999999999999e-05 2024-09-19T05:43:48.806 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978959+0000", 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0015099379999999999 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979006+0000", 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7315999999999999e-05 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.807 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:30 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.792405396999996, 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118336, 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.808 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977735+0000", 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977736+0000", 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.096e-06 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977743+0000", 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0979999999999999e-06 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.809 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977756+0000", 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3891e-05 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977765+0000", 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7579999999999998e-06 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977830+0000", 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5184999999999993e-05 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.810 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977839+0000", 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4770000000000003e-06 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977854+0000", 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4831000000000001e-05 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:27 2.13 e22/19 ondisk, result = 0)", 2024-09-19T05:43:48.811 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.792167845999998, 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101439, 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977973+0000", 2024-09-19T05:43:48.812 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977974+0000", 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6899999999999996e-07 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977979+0000", 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1399999999999999e-06 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.813 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.977993+0000", 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4215e-05 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978001+0000", 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0260000000000007e-06 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978055+0000", 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3711000000000001e-05 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.814 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978062+0000", 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.978074+0000", 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2238999999999999e-05 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:48.815 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.790923797000005, 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0024064609999999999, 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979217+0000", 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979218+0000", 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0400000000000005e-07 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979227+0000", 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5880000000000004e-06 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979239+0000", 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:43:48.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979247+0000", 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8369999999999997e-06 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979307+0000", 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0438e-05 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979326+0000", 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8938999999999999e-05 2024-09-19T05:43:48.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981594+0000", 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0022679710000000001 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981623+0000", 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9329999999999999e-05 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.790623116999996, 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0021274990000000001, 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979518+0000", 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7800000000000001e-07 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979524+0000", 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8019999999999997e-06 2024-09-19T05:43:48.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979534+0000", 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8859999999999999e-06 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979542+0000", 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8189999999999992e-06 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979610+0000", 2024-09-19T05:43:48.822 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8433e-05 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979641+0000", 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1233999999999999e-05 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981625+0000", 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.001983964 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.823 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981645+0000", 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9783e-05 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.790562144999996, 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0020817100000000001, 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.824 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979579+0000", 2024-09-19T05:43:48.825 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.63e-07 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979598+0000", 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9072e-05 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979609+0000", 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0360000000000001e-05 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.826 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979615+0000", 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8700000000000003e-06 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979866+0000", 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000250454 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.979884+0000", 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7725000000000001e-05 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.827 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981647+0000", 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0017632629999999999 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981660+0000", 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3502999999999999e-05 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-19T05:43:48.828 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.790075725999998, 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0016104190000000001, 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.829 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980065+0000", 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980066+0000", 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8400000000000005e-07 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980070+0000", 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8010000000000003e-06 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980077+0000", 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.08e-06 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980082+0000", 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8450000000000002e-06 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980146+0000", 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3589000000000004e-05 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980166+0000", 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0514e-05 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981661+0000", 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014949539999999999 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981675+0000", 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:43:48.832 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.790025512, 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0018022209999999999, 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.833 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980115+0000", 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980116+0000", 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.834 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980126+0000", 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0429e-05 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980143+0000", 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6407999999999999e-05 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980150+0000", 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.835 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980385+0000", 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023545400000000001 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980403+0000", 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7652999999999999e-05 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981869+0000", 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0014661679999999999 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.836 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981917+0000", 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319000000000003e-05 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.789426559999995, 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000222196, 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.837 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980714+0000", 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.838 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980715+0000", 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980721+0000", 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2620000000000004e-06 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980735+0000", 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3756e-05 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.839 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980759+0000", 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980813+0000", 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4249999999999997e-05 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980821+0000", 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7540000000000006e-06 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.840 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980936+0000", 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115574 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.789358497999999, 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00030787300000000001, 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:48.841 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980782+0000", 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980783+0000", 2024-09-19T05:43:48.842 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980786+0000", 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5279999999999999e-06 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980795+0000", 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0790000000000007e-06 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.843 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980802+0000", 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4910000000000002e-06 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980962+0000", 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015999100000000001 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980968+0000", 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.874e-06 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.844 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981090+0000", 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012244099999999999 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop_reply(mds.0.4:29 2.15 e22/19 ondisk, result = 0)", 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.789328592000004, 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00041785999999999999, 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:43:48.845 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980812+0000", 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1600000000000002e-07 2024-09-19T05:43:48.846 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980816+0000", 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6660000000000001e-06 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980823+0000", 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6789999999999999e-06 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980827+0000", 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6480000000000002e-06 2024-09-19T05:43:48.847 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981115+0000", 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028752699999999999 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981121+0000", 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981230+0000", 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108546 2024-09-19T05:43:48.848 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.789305960999997, 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011264039999999999, 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.849 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980835+0000", 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.850 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980840+0000", 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1540000000000001e-06 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980848+0000", 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.851 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980853+0000", 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9030000000000003e-06 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980907+0000", 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4224999999999997e-05 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.980937+0000", 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9852000000000001e-05 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.852 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981931+0000", 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00099441099999999995 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981961+0000", 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0144000000000001e-05 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-19T05:43:48.853 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 70.788787717999995, 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0013984010000000001, 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:43:48.854 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981353+0000", 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981354+0000", 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0160000000000001e-06 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981361+0000", 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2919999999999998e-06 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.855 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981373+0000", 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1439e-05 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981385+0000", 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2041999999999999e-05 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981446+0000", 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1446999999999996e-05 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.856 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.981469+0000", 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2331e-05 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982708+0000", 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0012396359999999999 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:42:37.982751+0000", 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3198000000000002e-05 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.857 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.858 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:43:48.858 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:43:48.858 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:48.859 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:48.858+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:48.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:48.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:48.953+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f52480728a0 con 0x7f5248073660 2024-09-19T05:43:48.960 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:48.958+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:48.971 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:48.971 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:48.971 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:48.971 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:48.980 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:49.060 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:49.059+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:49.129 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:49.129 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:49.129 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:49.130 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:49.130 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:49.138 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:49.162 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:49.161+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:49.249 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:16 2.1d 2:b871830b:::605.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.272255766000001, 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029411070000000001, 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974707+0000", 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974719+0000", 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1449e-05 2024-09-19T05:43:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974732+0000", 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3788000000000001e-05 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974743+0000", 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0831e-05 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974751+0000", 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1659999999999994e-06 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974913+0000", 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016199899999999999 2024-09-19T05:43:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975022+0000", 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108857 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011757200000000001 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976884+0000", 2024-09-19T05:43:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017442969999999999 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977555+0000", 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067127799999999998 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977604+0000", 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8068999999999997e-05 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977648+0000", 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4801e-05 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:17 2.1d 2:b871830b:::605.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.272189248999993, 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034736569999999998, 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974774+0000", 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6800000000000001e-07 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974776+0000", 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974783+0000", 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2940000000000002e-06 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974786+0000", 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6959999999999998e-06 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975441+0000", 2024-09-19T05:43:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065484099999999997 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975514+0000", 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2608999999999995e-05 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975640+0000", 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125869 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977680+0000", 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020402739999999999 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978154+0000", 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047386900000000003 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978206+0000", 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2250000000000003e-05 2024-09-19T05:43:49.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978247+0000", 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1369000000000003e-05 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:20 2.1c 2:3ed09add:::607.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:49.262 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.272154986999993, 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034986209999999999, 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:49.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974808+0000", 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8999999999999998e-07 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974818+0000", 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6870000000000006e-06 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974823+0000", 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974826+0000", 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9639999999999999e-06 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974869+0000", 2024-09-19T05:43:49.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3380999999999997e-05 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975009+0000", 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013971099999999999 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975140+0000", 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131235 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976896+0000", 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001756482 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978210+0000", 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013140090000000001 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978275+0000", 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000004e-05 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978307+0000", 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899000000000001e-05 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:21 2.1c 2:3ed09add:::607.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.272121390999999, 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035825869999999999, 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:49.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974842+0000", 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974846+0000", 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4809999999999998e-06 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974856+0000", 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0701e-05 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974862+0000", 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0669999999999997e-06 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976006+0000", 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001143205 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976080+0000", 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4290999999999997e-05 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976213+0000", 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013278500000000001 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977693+0000", 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014800810000000001 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978354+0000", 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000660971 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978385+0000", 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1423000000000001e-05 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978424+0000", 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9131000000000002e-05 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.272090954999996, 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037252449999999999, 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974872+0000", 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974873+0000", 2024-09-19T05:43:49.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2200000000000004e-07 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974876+0000", 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.965e-06 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974882+0000", 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3729999999999998e-06 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974890+0000", 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039999999999995e-06 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.974947+0000", 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6505000000000002e-05 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975044+0000", 2024-09-19T05:43:49.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6933000000000002e-05 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975162+0000", 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117879 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977622+0000", 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024597519999999999 2024-09-19T05:43:49.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978546+0000", 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092433599999999999 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978574+0000", 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7923e-05 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978597+0000", 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3553e-05 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:30 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.271382574, 2024-09-19T05:43:49.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002124711, 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975580+0000", 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975581+0000", 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0470000000000001e-06 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975592+0000", 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.097e-05 2024-09-19T05:43:49.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975603+0000", 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0891e-05 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975611+0000", 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4569999999999999e-06 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975662+0000", 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1424000000000001e-05 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.975687+0000", 2024-09-19T05:43:49.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4913999999999999e-05 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977661+0000", 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019738020000000002 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977705+0000", 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4206000000000003e-05 2024-09-19T05:43:49.286 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:24 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.270600311999999, 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012912539999999999, 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.288 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:49.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976363+0000", 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976364+0000", 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2799999999999995e-07 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976370+0000", 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8369999999999999e-06 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976381+0000", 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0744000000000001e-05 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976388+0000", 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2320000000000004e-06 2024-09-19T05:43:49.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976435+0000", 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6726e-05 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976452+0000", 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6629999999999998e-05 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977623+0000", 2024-09-19T05:43:49.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001170894 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977654+0000", 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1362999999999998e-05 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:49.292 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.270410945999998, 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001404389, 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976552+0000", 2024-09-19T05:43:49.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976553+0000", 2024-09-19T05:43:49.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9800000000000003e-07 2024-09-19T05:43:49.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976558+0000", 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7740000000000001e-06 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976568+0000", 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976575+0000", 2024-09-19T05:43:49.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976621+0000", 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6136e-05 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976640+0000", 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8604e-05 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977917+0000", 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00127725 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977956+0000", 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9249000000000003e-05 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.297 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.270360666000002, 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099816800000000002, 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976602+0000", 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976603+0000", 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2500000000000001e-07 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976610+0000", 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0210000000000002e-06 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976625+0000", 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5128e-05 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976638+0000", 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2995e-05 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976709+0000", 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0659000000000002e-05 2024-09-19T05:43:49.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977601+0000", 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089194000000000003 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:25 2.7 e22/19 2:e0b41d06:::609.00000000:head v 22'2, mlcod=0'0)", 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.270136399999998, 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001150365, 2024-09-19T05:43:49.301 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976827+0000", 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-07 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976836+0000", 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3049999999999998e-06 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976845+0000", 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4020000000000002e-06 2024-09-19T05:43:49.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976850+0000", 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6890000000000001e-06 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976916+0000", 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5437000000000004e-05 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.976933+0000", 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6640999999999999e-05 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977941+0000", 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001008175 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977977+0000", 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6264999999999999e-05 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:16 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.269572197000002, 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000264987, 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:49.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977391+0000", 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977392+0000", 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3600000000000002e-07 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977396+0000", 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6190000000000002e-06 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977412+0000", 2024-09-19T05:43:49.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5919000000000001e-05 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977420+0000", 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0779999999999996e-06 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977543+0000", 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012284899999999999 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977550+0000", 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.977656+0000", 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010567499999999999 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:17 2.1d e22/19 ondisk, result = 0)", 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.268891491000005, 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183666, 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978072+0000", 2024-09-19T05:43:49.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978076+0000", 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.019e-06 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978089+0000", 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2828e-05 2024-09-19T05:43:49.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978095+0000", 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4389999999999997e-06 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978141+0000", 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5649999999999998e-05 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978149+0000", 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5820000000000004e-06 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978255+0000", 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010655299999999999 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:20 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.268827837000003, 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176971, 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978135+0000", 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978136+0000", 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2300000000000001e-07 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978139+0000", 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8100000000000002e-06 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978148+0000", 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978155+0000", 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0169999999999999e-06 2024-09-19T05:43:49.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978198+0000", 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3251999999999999e-05 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978205+0000", 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2250000000000003e-06 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978312+0000", 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010694800000000001 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:21 2.1c e22/19 ondisk, result = 0)", 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:49.320 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.268793518999999, 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000262632, 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978170+0000", 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978173+0000", 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8169999999999999e-06 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978180+0000", 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4839999999999998e-06 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978185+0000", 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978347+0000", 2024-09-19T05:43:49.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016165400000000001 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978351+0000", 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1389999999999997e-06 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978432+0000", 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1260999999999993e-05 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 71.268510565, 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000148286, 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:49.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978452+0000", 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978453+0000", 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-07 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978458+0000", 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.899e-06 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978472+0000", 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3733e-05 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978479+0000", 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8040000000000004e-06 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978539+0000", 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0038999999999999e-05 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978543+0000", 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2699999999999998e-06 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:37.978601+0000", 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7881999999999999e-05 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.328 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.252134229000006, 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001029049, 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994829+0000", 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994831+0000", 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2299999999999998e-06 2024-09-19T05:43:49.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994844+0000", 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.291e-05 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994875+0000", 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0898999999999997e-05 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994899+0000", 2024-09-19T05:43:49.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3972999999999999e-05 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.994994+0000", 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5068000000000005e-05 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995015+0000", 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1166e-05 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995800+0000", 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078543299999999999 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:43.995858+0000", 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000001e-05 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(2.1d 39/38 [PushReplyOp(2:b871830b:::605.00000000:head)])", 2024-09-19T05:43:49.333 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.688826536000001, 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019826500000000001, 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558136+0000", 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558137+0000", 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0100000000000001e-07 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558140+0000", 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4690000000000002e-06 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558149+0000", 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5329999999999994e-06 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558157+0000", 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5639999999999997e-06 2024-09-19T05:43:49.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558207+0000", 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9462000000000002e-05 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.558335+0000", 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012793599999999999 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPushReply(2.1c 39/38 [PushReplyOp(2:3ed09add:::607.00000000:head)])", 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.687046723999998, 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027980799999999999, 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559916+0000", 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559917+0000", 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8299999999999998e-07 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559920+0000", 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1360000000000001e-06 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559929+0000", 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6829999999999997e-06 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559935+0000", 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.559989+0000", 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3659999999999998e-05 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:19.560196+0000", 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207071 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(mds.0.4:45 2.6 e42/19 2:654134d2:::mds0_openfiles.0:head v 42'3, mlcod=22'1)", 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.247111479000001, 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084542299999999999, 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:49.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999852+0000", 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999853+0000", 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.857e-06 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999869+0000", 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5271e-05 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999890+0000", 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1373999999999999e-05 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999902+0000", 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1603999999999999e-05 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:24.999981+0000", 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8935000000000002e-05 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000009+0000", 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8702999999999999e-05 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000650+0000", 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064042099999999996 2024-09-19T05:43:49.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:25.000697+0000", 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7258000000000001e-05 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.346 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:49.348 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:49.348 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:49.346+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:49.449 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:49.448+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:49.459 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:49.459 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:49.459 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:49.468 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:49.550 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:49.548+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:49.579 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:49.580 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:49.580 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:49.580 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:49.580 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:49.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:49.637 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.635+0000 7f523dffb640 1 -- 172.21.15.2:0/1339414507 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v52) ==== 87+0+0 (secure 0 0 0) 0x7f52440590b0 con 0x7f5248073660 2024-09-19T05:43:49.638 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:43:49.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f522c06c2b0 msgr2=0x7f522c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f522c06c2b0 0x7f522c06e770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5238034000 tx=0x7f523802e000 comp rx=0 tx=0).stop 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5248073660 msgr2=0x7f52481f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5248073660 0x7f52481f4ba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f524400c490 tx=0x7f524400c950 comp rx=0 tx=0).stop 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 shutdown_connections 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f522c06c2b0 0x7f522c06e770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52481584c0 0x7f52480c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52480c86f0 0x7f52481f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 --2- 172.21.15.2:0/1339414507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5248073660 0x7f52481f4ba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 >> 172.21.15.2:0/1339414507 conn(0x7f52480cd470 msgr2=0x7f5248156500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 shutdown_connections 2024-09-19T05:43:49.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:49.638+0000 7f524f801640 1 -- 172.21.15.2:0/1339414507 wait complete. 2024-09-19T05:43:49.650 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:49.649+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:49.748 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:49.748 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.865722192000007, 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:49.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:49.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:49.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.756 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:54 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.756 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:49.756 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.226637384, 2024-09-19T05:43:49.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067459440000000002, 2024-09-19T05:43:49.757 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 54 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518129+0000", 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518130+0000", 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:43:49.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518136+0000", 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.502e-06 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518146+0000", 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518152+0000", 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5250000000000002e-06 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518503+0000", 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000350723 2024-09-19T05:43:49.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518551+0000", 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8143999999999998e-05 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518627+0000", 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5791999999999995e-05 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519754+0000", 2024-09-19T05:43:49.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001127007 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524795+0000", 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050413899999999998 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524828+0000", 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2292999999999997e-05 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.524875+0000", 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7781000000000002e-05 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.226594855000002, 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070119079999999999, 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 55 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518172+0000", 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518173+0000", 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6300000000000004e-07 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518256+0000", 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.297e-05 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518274+0000", 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8040999999999999e-05 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518279+0000", 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8069999999999999e-06 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518867+0000", 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00058892600000000003 2024-09-19T05:43:49.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518917+0000", 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9113000000000002e-05 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518995+0000", 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8603999999999995e-05 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519756+0000", 2024-09-19T05:43:49.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095299999999998 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525108+0000", 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053522029999999998 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525141+0000", 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2774999999999997e-05 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525184+0000", 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2753000000000002e-05 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.770 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.226477295999999, 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070489530000000002, 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 56 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518290+0000", 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7800000000000002e-07 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518293+0000", 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8990000000000001e-06 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518298+0000", 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4609999999999999e-06 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.518308+0000", 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1730000000000001e-06 2024-09-19T05:43:49.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519370+0000", 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001062131 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519428+0000", 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8174000000000002e-05 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.519506+0000", 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8282999999999996e-05 2024-09-19T05:43:49.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.520014+0000", 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050832899999999996 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525233+0000", 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052187140000000002 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525294+0000", 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0702000000000003e-05 2024-09-19T05:43:49.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.525338+0000", 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4608999999999999e-05 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:49.776 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.197219961000002, 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074872619999999997, 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547547+0000", 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547587+0000", 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9784999999999999e-05 2024-09-19T05:43:49.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547592+0000", 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1719999999999998e-06 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547595+0000", 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.242e-06 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549550+0000", 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019545980000000001 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549594+0000", 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4005000000000003e-05 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.549664+0000", 2024-09-19T05:43:49.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0269000000000001e-05 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550463+0000", 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079894600000000003 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.554977+0000", 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045143689999999998 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555009+0000", 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1955999999999997e-05 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555034+0000", 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4698e-05 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:49.783 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.197163023000002, 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075222300000000004, 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 88 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547604+0000", 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4799999999999998e-07 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547607+0000", 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1200000000000002e-06 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547612+0000", 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1250000000000003e-06 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.547616+0000", 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8970000000000001e-06 2024-09-19T05:43:49.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550051+0000", 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024350740000000002 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550093+0000", 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2375e-05 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550166+0000", 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2997000000000006e-05 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.550616+0000", 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044985000000000001 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555073+0000", 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004457478 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555103+0000", 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9346e-05 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.555126+0000", 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-19T05:43:49.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.043103290999994, 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068439319999999996, 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 115 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701664+0000", 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701722+0000", 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8013000000000002e-05 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701727+0000", 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701730+0000", 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.118e-06 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705609+0000", 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038791059999999998 2024-09-19T05:43:49.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705647+0000", 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7950000000000001e-05 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.705709+0000", 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2005e-05 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706359+0000", 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064942999999999999 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708456+0000", 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020977259999999998 2024-09-19T05:43:49.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708486+0000", 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0156000000000001e-05 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708507+0000", 2024-09-19T05:43:49.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1061999999999999e-05 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.043030959999996, 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068450919999999997, 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 116 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701736+0000", 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701740+0000", 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-06 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701743+0000", 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3890000000000001e-06 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701746+0000", 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706078+0000", 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004331896 2024-09-19T05:43:49.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706122+0000", 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4662e-05 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706185+0000", 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3039000000000004e-05 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706686+0000", 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050062399999999997 2024-09-19T05:43:49.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708534+0000", 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018486049999999999 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708561+0000", 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6089999999999999e-05 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708581+0000", 2024-09-19T05:43:49.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0387e-05 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 86.043017410999994, 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068967100000000003, 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701749+0000", 2024-09-19T05:43:49.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701750+0000", 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1e-07 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701752+0000", 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701755+0000", 2024-09-19T05:43:49.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3469999999999999e-06 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.701758+0000", 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706375+0000", 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046177049999999997 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706415+0000", 2024-09-19T05:43:49.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9248000000000001e-05 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706480+0000", 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5656999999999998e-05 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.706867+0000", 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038665200000000002 2024-09-19T05:43:49.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708603+0000", 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001735777 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708629+0000", 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6016e-05 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.708646+0000", 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7359000000000001e-05 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.637850510000007, 2024-09-19T05:43:49.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073636079999999998, 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 137 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106916+0000", 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106917+0000", 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0900000000000001e-07 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106980+0000", 2024-09-19T05:43:49.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3962999999999997e-05 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106986+0000", 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2120000000000002e-06 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106989+0000", 2024-09-19T05:43:49.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2899999999999998e-06 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111080+0000", 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040908819999999997 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111156+0000", 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5690000000000002e-05 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111283+0000", 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012769800000000001 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112457+0000", 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011733990000000001 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114182+0000", 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00172543 2024-09-19T05:43:49.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114235+0000", 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3287e-05 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114280+0000", 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.637771912999995, 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010021159, 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 138 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.816 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106995+0000", 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-07 2024-09-19T05:43:49.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.106997+0000", 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.277e-06 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107001+0000", 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1709999999999998e-06 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107003+0000", 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.627e-06 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111867+0000", 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048637200000000002 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111908+0000", 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1329999999999999e-05 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:49.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.111979+0000", 2024-09-19T05:43:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1136999999999993e-05 2024-09-19T05:43:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113073+0000", 2024-09-19T05:43:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001093849 2024-09-19T05:43:49.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:49.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116925+0000", 2024-09-19T05:43:49.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038513890000000002 2024-09-19T05:43:49.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:49.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.116976+0000", 2024-09-19T05:43:49.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1706000000000003e-05 2024-09-19T05:43:49.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117016+0000", 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9691000000000003e-05 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:49.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.637759974999994, 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010095936999999999, 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:49.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:49.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107007+0000", 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0200000000000001e-07 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107009+0000", 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7600000000000001e-06 2024-09-19T05:43:49.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107012+0000", 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0460000000000001e-06 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107014+0000", 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112495+0000", 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054809289999999998 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112579+0000", 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3565000000000004e-05 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:49.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.112710+0000", 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013096900000000001 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114138+0000", 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001427967 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117051+0000", 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002913461 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117081+0000", 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9575999999999999e-05 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117103+0000", 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1851000000000001e-05 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.637748895000001, 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010364564999999999, 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107018+0000", 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.03e-07 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107077+0000", 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9332e-05 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107083+0000", 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2329999999999997e-06 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107086+0000", 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2780000000000002e-06 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113097+0000", 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060114670000000004 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113179+0000", 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1165e-05 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113309+0000", 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013028200000000001 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114142+0000", 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083262099999999997 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117293+0000", 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003151432 2024-09-19T05:43:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117344+0000", 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-05 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.117383+0000", 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8392000000000001e-05 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.637674329999996, 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011082557999999999, 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 141 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107093+0000", 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8299999999999998e-07 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107095+0000", 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9690000000000001e-06 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107098+0000", 2024-09-19T05:43:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.332e-06 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.107101+0000", 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113801+0000", 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067008229999999998 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113853+0000", 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1088000000000003e-05 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.113922+0000", 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9765999999999996e-05 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.114371+0000", 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044916900000000002 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118090+0000", 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003718641 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118139+0000", 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9322e-05 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118175+0000", 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5627000000000003e-05 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:144 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.626247789999994, 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006919644, 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118519+0000", 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6199999999999999e-07 2024-09-19T05:43:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118567+0000", 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7898000000000001e-05 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118572+0000", 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9439999999999996e-06 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118575+0000", 2024-09-19T05:43:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0989999999999999e-06 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119547+0000", 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097127900000000002 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119622+0000", 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5331000000000001e-05 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.119747+0000", 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125055 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120744+0000", 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099654699999999997 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125236+0000", 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044923769999999997 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125400+0000", 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016360800000000001 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125439+0000", 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9144000000000003e-05 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:145 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.626185648000003, 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006948582, 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118581+0000", 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6800000000000002e-07 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118628+0000", 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6100000000000002e-05 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118633+0000", 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5639999999999996e-06 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.118637+0000", 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6449999999999998e-06 2024-09-19T05:43:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120201+0000", 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00156438 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120285+0000", 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3487999999999998e-05 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120405+0000", 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011992799999999999 2024-09-19T05:43:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.120949+0000", 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054465799999999995 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125477+0000", 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045277429999999999 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125507+0000", 2024-09-19T05:43:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9799999999999999e-05 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.125530+0000", 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3008000000000001e-05 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.148962225999995, 2024-09-19T05:43:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072257390000000001, 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595805+0000", 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:43:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595807+0000", 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595812+0000", 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7310000000000001e-06 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595815+0000", 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596850+0000", 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001035263 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596893+0000", 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.278e-05 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.596960+0000", 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6886999999999998e-05 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598057+0000", 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010974909999999999 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.602954+0000", 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048968950000000001 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603005+0000", 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0732000000000001e-05 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603030+0000", 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5245e-05 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.148940862000003, 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073256010000000002, 2024-09-19T05:43:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 157 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595826+0000", 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-07 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595828+0000", 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9439999999999999e-06 2024-09-19T05:43:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595832+0000", 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0980000000000004e-06 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595835+0000", 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597200+0000", 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013651729999999999 2024-09-19T05:43:50.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597241+0000", 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0515000000000001e-05 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597319+0000", 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8186e-05 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598139+0000", 2024-09-19T05:43:50.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082006699999999995 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603070+0000", 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049314579999999997 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603122+0000", 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1878e-05 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603152+0000", 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9234000000000002e-05 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.148923159999995, 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073920080000000003, 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 158 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595844+0000", 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6600000000000003e-07 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595881+0000", 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6709999999999999e-05 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595886+0000", 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.733e-06 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595889+0000", 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597584+0000", 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016944359999999999 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597628+0000", 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4387e-05 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.597695+0000", 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6548e-05 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598618+0000", 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092280499999999996 2024-09-19T05:43:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603187+0000", 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045694189999999999 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603215+0000", 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7789000000000001e-05 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603236+0000", 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0809999999999999e-05 2024-09-19T05:43:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 85.148869774999994, 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074183829999999997, 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 159 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595897+0000", 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6899999999999999e-07 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595899+0000", 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595903+0000", 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.467e-06 2024-09-19T05:43:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.595908+0000", 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1290000000000004e-06 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598283+0000", 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023746259999999999 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598330+0000", 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7277999999999997e-05 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598406+0000", 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6497999999999998e-05 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.598853+0000", 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044686500000000001 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603260+0000", 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044065240000000002 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603296+0000", 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-05 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:24.603315+0000", 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9528999999999999e-05 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:50.589 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:50.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:50.591 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:50.588+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:50.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:50.691 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:50.689+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:50.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.733+0000 7f4ac7023640 1 Processor -- start 2024-09-19T05:43:50.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.734+0000 7f4ac7023640 1 -- start start 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.734+0000 7f4ac7023640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4ac0154cf0 0x7f4ac00c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.734+0000 7f4ac7023640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac0070990 con 0x7f4ac015aa20 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.734+0000 7f4ac7023640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac00c8830 con 0x7f4ac00cd340 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.734+0000 7f4ac7023640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac00c6fb0 con 0x7f4ac0154cf0 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.735+0000 7f4ac4dbf640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4ac0154cf0 0x7f4ac00c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.735+0000 7f4abffff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4ac015aa20 0x7f4ac0070c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49190/0 (socket says 172.21.15.2:49190) 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.735+0000 7f4ac4dbf640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4ac0154cf0 0x7f4ac00c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59308/0 (socket says 172.21.15.2:59308) 2024-09-19T05:43:50.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.735+0000 7f4abffff640 1 -- 172.21.15.2:0/3929185082 learned_addr learned my addr 172.21.15.2:0/3929185082 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.735+0000 7f4ac4dbf640 1 -- 172.21.15.2:0/3929185082 >> v1:172.21.15.123:6789/0 conn(0x7f4ac00cd340 legacy=0x7f4ac01652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.735+0000 7f4ac4dbf640 1 -- 172.21.15.2:0/3929185082 >> v1:172.21.15.2:6789/0 conn(0x7f4ac015aa20 legacy=0x7f4ac0070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.735+0000 7f4ac4dbf640 1 -- 172.21.15.2:0/3929185082 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac00700f0 con 0x7f4ac0154cf0 2024-09-19T05:43:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.736+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3929185082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac0154cf0 0x7f4ac00c8ff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78787cfafaca555f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.736+0000 7f4abf7fe640 1 -- 172.21.15.2:0/3929185082 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4037070 con 0x7f4ac0154cf0 2024-09-19T05:43:50.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.736+0000 7f4abf7fe640 1 -- 172.21.15.2:0/3929185082 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab4032440 con 0x7f4ac0154cf0 2024-09-19T05:43:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.736+0000 7f4abf7fe640 1 -- 172.21.15.2:0/3929185082 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4032810 con 0x7f4ac0154cf0 2024-09-19T05:43:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.737+0000 7f4ac7023640 1 -- 172.21.15.2:0/3929185082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac0154cf0 msgr2=0x7f4ac00c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.737+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3929185082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac0154cf0 0x7f4ac00c8ff0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a230 comp rx=0 tx=0).stop 2024-09-19T05:43:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.737+0000 7f4ac7023640 1 -- 172.21.15.2:0/3929185082 shutdown_connections 2024-09-19T05:43:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.737+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3929185082 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac0154cf0 0x7f4ac00c8ff0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.737+0000 7f4ac7023640 1 -- 172.21.15.2:0/3929185082 >> 172.21.15.2:0/3929185082 conn(0x7f4ac0072ad0 msgr2=0x7f4ac0072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.737+0000 7f4ac7023640 1 -- 172.21.15.2:0/3929185082 shutdown_connections 2024-09-19T05:43:50.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.737+0000 7f4ac7023640 1 -- 172.21.15.2:0/3929185082 wait complete. 2024-09-19T05:43:50.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.739+0000 7f4ac7023640 1 Processor -- start 2024-09-19T05:43:50.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.739+0000 7f4ac7023640 1 -- start start 2024-09-19T05:43:50.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac7023640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac00cd340 0x7f4ac01e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac7023640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ac0154cf0 0x7f4ac01e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac7023640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ac015aa20 0x7f4ac01e5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac7023640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac00c8ad0 con 0x7f4ac00cd340 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac7023640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac00706b0 con 0x7f4ac015aa20 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac7023640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac015ae20 con 0x7f4ac0154cf0 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac4dbf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac00cd340 0x7f4ac01e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac4dbf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac00cd340 0x7f4ac01e2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59316/0 (socket says 172.21.15.2:59316) 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.740+0000 7f4ac4dbf640 1 -- 172.21.15.2:0/3777086370 learned_addr learned my addr 172.21.15.2:0/3777086370 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ac0154cf0 0x7f4ac01e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ac015aa20 0x7f4ac01e5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac4dbf640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ac0154cf0 msgr2=0x7f4ac01e4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ac0154cf0 0x7f4ac01e4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac4dbf640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ac015aa20 msgr2=0x7f4ac01e5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ac015aa20 0x7f4ac01e5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac4dbf640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac0149f90 con 0x7f4ac00cd340 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ac015aa20 0x7f4ac01e5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4ac0154cf0 0x7f4ac01e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac00cd340 0x7f4ac01e2e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40098c0 tx=0x7f4ab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4037040 con 0x7f4ac00cd340 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab4030770 con 0x7f4ac00cd340 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4030b40 con 0x7f4ac00cd340 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac014a270 con 0x7f4ac00cd340 2024-09-19T05:43:50.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.741+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac014a830 con 0x7f4ac00cd340 2024-09-19T05:43:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.742+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4ac00c7530 con 0x7f4ac00cd340 2024-09-19T05:43:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.743+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4ab4044020 con 0x7f4ac00cd340 2024-09-19T05:43:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.744+0000 7f4abd7fa640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4aa406c2b0 0x7f4aa406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.744+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f4ab40ac4e0 con 0x7f4ac00cd340 2024-09-19T05:43:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.744+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v52) ==== 54+0+16 (secure 0 0 0) 0x7f4ab4032e20 con 0x7f4ac00cd340 2024-09-19T05:43:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.744+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4aa406c2b0 0x7f4aa406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.744+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4ac014aa70 0x7f4ac0159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.744+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ac00c7ef0 con 0x7f4ac014aa70 2024-09-19T05:43:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.744+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4ac014aa70 0x7f4ac0159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.745+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4aa406c2b0 0x7f4aa406e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4ac01596f0 tx=0x7f4aac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.745+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4ac014aa70 0x7f4ac0159d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.746+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac00c7ef0 con 0x7f4ac014aa70 2024-09-19T05:43:50.748 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:50.748 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:50.757 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:50.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.760+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac006c900 con 0x7f4ac014aa70 2024-09-19T05:43:50.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.763+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac006c900 con 0x7f4ac014aa70 2024-09-19T05:43:50.765 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.763+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4ac014b890 0x7f4ac014bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.763+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4ac014c1b0 con 0x7f4ac014b890 2024-09-19T05:43:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.763+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4ac014b890 0x7f4ac014bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.764+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4ac014b890 0x7f4ac014bc70 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.764+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac014c1b0 con 0x7f4ac014b890 2024-09-19T05:43:50.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.779+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac01e3350 con 0x7f4ac014b890 2024-09-19T05:43:50.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.781+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac01e3350 con 0x7f4ac014b890 2024-09-19T05:43:50.783 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.781+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f4ac01470d0 0x7f4ac01474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.781+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4ac0147a10 con 0x7f4ac01470d0 2024-09-19T05:43:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.781+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f4ac01470d0 0x7f4ac01474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.782+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f4ac01470d0 0x7f4ac01474d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.782+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4aac001ce0 con 0x7f4ac01470d0 2024-09-19T05:43:50.791 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:50.790+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:50.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.796+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac0062880 con 0x7f4ac01470d0 2024-09-19T05:43:50.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.798+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac0062880 con 0x7f4ac01470d0 2024-09-19T05:43:50.800 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:50.801 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:50.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.798+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4ac0063360 0x7f4ac00cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.798+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4ac014d3a0 con 0x7f4ac0063360 2024-09-19T05:43:50.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.798+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4ac0063360 0x7f4ac00cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.799+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4ac0063360 0x7f4ac00cb2a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.799+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac014d3a0 con 0x7f4ac0063360 2024-09-19T05:43:50.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.813+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac014d3a0 con 0x7f4ac0063360 2024-09-19T05:43:50.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.815+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac014d3a0 con 0x7f4ac0063360 2024-09-19T05:43:50.817 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.815+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4ac00cbfa0 0x7f4ac002c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.815+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4ac0206860 con 0x7f4ac00cbfa0 2024-09-19T05:43:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.816+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4ac00cbfa0 0x7f4ac002c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.816+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4ac00cbfa0 0x7f4ac002c640 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.817+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac0206860 con 0x7f4ac00cbfa0 2024-09-19T05:43:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.831+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac002cb80 con 0x7f4ac00cbfa0 2024-09-19T05:43:50.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.833+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac002cb80 con 0x7f4ac00cbfa0 2024-09-19T05:43:50.835 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:50.836 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:50.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.833+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac002d2a0 0x7f4ac002d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.833+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f4ac0209a70 con 0x7f4ac002d2a0 2024-09-19T05:43:50.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.834+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac002d2a0 0x7f4ac002d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.834+0000 7f4abffff640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac002d2a0 0x7f4ac002d720 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.835+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac0209a70 con 0x7f4ac002d2a0 2024-09-19T05:43:50.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.849+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac02071e0 con 0x7f4ac002d2a0 2024-09-19T05:43:50.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.851+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac02071e0 con 0x7f4ac002d2a0 2024-09-19T05:43:50.853 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:50.853 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:50.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.851+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4ac0207900 0x7f4ac0209cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.851+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4ac020cca0 con 0x7f4ac0207900 2024-09-19T05:43:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.851+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4ac0207900 0x7f4ac0209cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.852+0000 7f4ac55c0640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4ac0207900 0x7f4ac0209cf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.852+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac020cca0 con 0x7f4ac0207900 2024-09-19T05:43:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.866+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac020cc70 con 0x7f4ac0207900 2024-09-19T05:43:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.868+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac020cc70 con 0x7f4ac0207900 2024-09-19T05:43:50.871 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.869+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f4ac020a7e0 0x7f4ac020ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.869+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4ac0210070 con 0x7f4ac020a7e0 2024-09-19T05:43:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.869+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f4ac020a7e0 0x7f4ac020ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.869+0000 7f4ac4dbf640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f4ac020a7e0 0x7f4ac020ac60 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:50.872 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:50.872 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.870+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac0210070 con 0x7f4ac020a7e0 2024-09-19T05:43:50.881 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.884+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4ac020b2b0 con 0x7f4ac020a7e0 2024-09-19T05:43:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.886+0000 7f4abd7fa640 1 -- 172.21.15.2:0/3777086370 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4ac020b2b0 con 0x7f4ac020a7e0 2024-09-19T05:43:50.889 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4ac014aa70 msgr2=0x7f4ac0159d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f4ac014aa70 0x7f4ac0159d10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4ac014b890 msgr2=0x7f4ac014bc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4ac014b890 0x7f4ac014bc70 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f4ac01470d0 msgr2=0x7f4ac01474d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f4ac01470d0 0x7f4ac01474d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4ac0063360 msgr2=0x7f4ac00cb2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4ac0063360 0x7f4ac00cb2a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4ac00cbfa0 msgr2=0x7f4ac002c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f4ac00cbfa0 0x7f4ac002c640 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac002d2a0 msgr2=0x7f4ac002d720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4ac002d2a0 0x7f4ac002d720 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4ac0207900 msgr2=0x7f4ac0209cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4ac0207900 0x7f4ac0209cf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f4ac020a7e0 msgr2=0x7f4ac020ac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f4ac020a7e0 0x7f4ac020ac60 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac55c0640 1 -- 172.21.15.2:0/3777086370 reap_dead start 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac55c0640 1 -- 172.21.15.2:0/3777086370 reap_dead start 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac55c0640 1 -- 172.21.15.2:0/3777086370 reap_dead start 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac55c0640 1 -- 172.21.15.2:0/3777086370 reap_dead start 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac55c0640 1 -- 172.21.15.2:0/3777086370 reap_dead start 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4aa406c2b0 msgr2=0x7f4aa406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4aa406c2b0 0x7f4aa406e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4ac01596f0 tx=0x7f4aac009040 comp rx=0 tx=0).stop 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac00cd340 msgr2=0x7f4ac01e2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac00cd340 0x7f4ac01e2e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40098c0 tx=0x7f4ab40046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 shutdown_connections 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f4aa406c2b0 0x7f4aa406e770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 --2- 172.21.15.2:0/3777086370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ac00cd340 0x7f4ac01e2e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 >> 172.21.15.2:0/3777086370 conn(0x7f4ac0072ad0 msgr2=0x7f4ac015afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 shutdown_connections 2024-09-19T05:43:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:50.887+0000 7f4ac7023640 1 -- 172.21.15.2:0/3777086370 wait complete. 2024-09-19T05:43:50.894 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:50.892+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:50.928 INFO:tasks.workunit.client.0.smithi002.stderr:Updating files: 95% (12316/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T05:43:50.995 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:50.993+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:50.999 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:50.999 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.008 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:51.096 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:51.094+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:51.157 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:51.157 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.166 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:51.196 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:51.195+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:51.298 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:51.296+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:51.314 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:51.314 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.323 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:51.398 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:51.396+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:51.430 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:51.430 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:51.499 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:51.497+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:51.562 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:51.563 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:51.617 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:51.614+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:51.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:43:51.682 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:51.682 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:51.701 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:51.699+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.757+0000 7f65c7e42640 1 Processor -- start 2024-09-19T05:43:51.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.758+0000 7f65c7e42640 1 -- start start 2024-09-19T05:43:51.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.758+0000 7f65c7e42640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f65c0070910 0x7f65c0073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.758+0000 7f65c7e42640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f65c0070cf0 0x7f65c0151170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.758+0000 7f65c7e42640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c00c6430 con 0x7f65c0073300 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.758+0000 7f65c7e42640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f65c015b530 con 0x7f65c0070910 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.758+0000 7f65c7e42640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f65c0073d90 con 0x7f65c0070cf0 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c5bde640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f65c0070910 0x7f65c0073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c5bde640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f65c0070910 0x7f65c0073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38946/0 (socket says 172.21.15.2:38946) 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c5bde640 1 -- 172.21.15.2:0/3377211995 learned_addr learned my addr 172.21.15.2:0/3377211995 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c63df640 1 --2- 172.21.15.2:0/3377211995 >> v2:172.21.15.123:3300/0 conn(0x7f65c0070cf0 0x7f65c0151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c5bde640 1 -- 172.21.15.2:0/3377211995 >> v2:172.21.15.123:3300/0 conn(0x7f65c0070cf0 msgr2=0x7f65c0151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c5bde640 1 --2- 172.21.15.2:0/3377211995 >> v2:172.21.15.123:3300/0 conn(0x7f65c0070cf0 0x7f65c0151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:51.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c5bde640 1 -- 172.21.15.2:0/3377211995 >> v1:172.21.15.123:6789/0 conn(0x7f65c0073300 legacy=0x7f65c015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.759+0000 7f65c5bde640 1 -- 172.21.15.2:0/3377211995 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c0157830 con 0x7f65c0070910 2024-09-19T05:43:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.760+0000 7f65c5bde640 1 --2- 172.21.15.2:0/3377211995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c0073700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65c00c6770 tx=0x7f65bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26864cb7dc14be5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.760+0000 7f65c4bdc640 1 -- 172.21.15.2:0/3377211995 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc037070 con 0x7f65c0070910 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.760+0000 7f65c4bdc640 1 -- 172.21.15.2:0/3377211995 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65bc032440 con 0x7f65c0070910 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.760+0000 7f65c4bdc640 1 -- 172.21.15.2:0/3377211995 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc032810 con 0x7f65c0070910 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.760+0000 7f65c7e42640 1 -- 172.21.15.2:0/3377211995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 msgr2=0x7f65c0073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.760+0000 7f65c7e42640 1 --2- 172.21.15.2:0/3377211995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c0073700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65c00c6770 tx=0x7f65bc02a270 comp rx=0 tx=0).stop 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.761+0000 7f65c7e42640 1 -- 172.21.15.2:0/3377211995 shutdown_connections 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.761+0000 7f65c7e42640 1 --2- 172.21.15.2:0/3377211995 >> v2:172.21.15.123:3300/0 conn(0x7f65c0070cf0 0x7f65c0151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.761+0000 7f65c7e42640 1 --2- 172.21.15.2:0/3377211995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c0073700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.761+0000 7f65c7e42640 1 -- 172.21.15.2:0/3377211995 >> 172.21.15.2:0/3377211995 conn(0x7f65c00cd4d0 msgr2=0x7f65c00cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.761+0000 7f65c7e42640 1 -- 172.21.15.2:0/3377211995 shutdown_connections 2024-09-19T05:43:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.761+0000 7f65c7e42640 1 -- 172.21.15.2:0/3377211995 wait complete. 2024-09-19T05:43:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.762+0000 7f65c7e42640 1 Processor -- start 2024-09-19T05:43:51.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.762+0000 7f65c7e42640 1 -- start start 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c7e42640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c014d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c7e42640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65c0070cf0 0x7f65c00c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c7e42640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65c0073300 0x7f65c0147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c7e42640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0157dd0 con 0x7f65c0070910 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c7e42640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0157af0 con 0x7f65c0073300 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c7e42640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f65c01585d0 con 0x7f65c0070cf0 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c014d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c014d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38960/0 (socket says 172.21.15.2:38960) 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 -- 172.21.15.2:0/1826266759 learned_addr learned my addr 172.21.15.2:0/1826266759 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c63df640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65c0073300 0x7f65c0147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c53dd640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65c0070cf0 0x7f65c00c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 -- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65c0070cf0 msgr2=0x7f65c00c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65c0070cf0 0x7f65c00c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 -- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65c0073300 msgr2=0x7f65c0147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65c0073300 0x7f65c0147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 -- 172.21.15.2:0/1826266759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c00ccd50 con 0x7f65c0070910 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65c5bde640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c014d500 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000c00 tx=0x7f65bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.763+0000 7f65b67fc640 1 -- 172.21.15.2:0/1826266759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc037040 con 0x7f65c0070910 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.764+0000 7f65b67fc640 1 -- 172.21.15.2:0/1826266759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65bc042070 con 0x7f65c0070910 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.764+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c014e0a0 con 0x7f65c0070910 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.764+0000 7f65b67fc640 1 -- 172.21.15.2:0/1826266759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc030880 con 0x7f65c0070910 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.764+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c00c9620 con 0x7f65c0070910 2024-09-19T05:43:51.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.765+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c0150170 con 0x7f65c0070910 2024-09-19T05:43:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.765+0000 7f65b67fc640 1 -- 172.21.15.2:0/1826266759 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f65bc030a20 con 0x7f65c0070910 2024-09-19T05:43:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.766+0000 7f65b67fc640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f65a406c2e0 0x7f65a406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:51.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.766+0000 7f65b67fc640 1 -- 172.21.15.2:0/1826266759 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f65bc0abbe0 con 0x7f65c0070910 2024-09-19T05:43:51.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.766+0000 7f65c53dd640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f65a406c2e0 0x7f65a406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:51.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.766+0000 7f65c53dd640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f65a406c2e0 0x7f65a406e7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65c014f1a0 tx=0x7f65b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:51.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.774+0000 7f65b67fc640 1 -- 172.21.15.2:0/1826266759 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f65c0150170 con 0x7f65c0070910 2024-09-19T05:43:51.802 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:51.799+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:51.834 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:51.834 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.843 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:51.926 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:51.925+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:51.932 INFO:tasks.workunit.client.0.smithi002.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T05:43:51.932 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-19T05:43:51.932 INFO:tasks.workunit.client.0.smithi002.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T05:43:51.932 INFO:tasks.workunit.client.0.smithi002.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T05:43:51.932 INFO:tasks.workunit.client.0.smithi002.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T05:43:51.932 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-19T05:43:51.932 INFO:tasks.workunit.client.0.smithi002.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr: git switch -c 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr:Or undo this operation with: 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr: git switch - 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr: 2024-09-19T05:43:51.933 INFO:tasks.workunit.client.0.smithi002.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T05:43:51.938 DEBUG:teuthology.orchestra.run.smithi002:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T05:43:51.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:51.960+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f65c00c4240 con 0x7f65c0070910 2024-09-19T05:43:51.983 INFO:tasks.workunit.client.0.smithi002.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T05:43:51.985 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:43:51.985 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T05:43:51.986 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:51.986 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:51.996 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:52.004 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:52.002+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:52.105 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:52.102+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:52.139 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:52.139 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:52.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:52.206 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:52.204+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:52.290 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:52.290 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:52.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:52.343 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:52.340+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:52.402 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:52.402 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:52.407 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:52.405+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:52.411 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:52.509 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:52.507+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:52.553 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:52.553 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:52.562 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:52.591 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T05:43:52.609 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:52.608+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:52.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.651+0000 7f65b67fc640 1 -- 172.21.15.2:0/1826266759 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v53) ==== 97+0+0 (secure 0 0 0) 0x7f65bc079c70 con 0x7f65c0070910 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f65a406c2e0 msgr2=0x7f65a406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f65a406c2e0 0x7f65a406e7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f65c014f1a0 tx=0x7f65b0009040 comp rx=0 tx=0).stop 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 msgr2=0x7f65c014d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c014d500 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f65bc000c00 tx=0x7f65bc0046c0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 shutdown_connections 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f65a406c2e0 0x7f65a406e7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65c0073300 0x7f65c0147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f65c0070cf0 0x7f65c00c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 --2- 172.21.15.2:0/1826266759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65c0070910 0x7f65c014d500 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 >> 172.21.15.2:0/1826266759 conn(0x7f65c00cd4d0 msgr2=0x7f65c00c67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:52.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 shutdown_connections 2024-09-19T05:43:52.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.655+0000 7f65c7e42640 1 -- 172.21.15.2:0/1826266759 wait complete. 2024-09-19T05:43:52.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:52.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:52.659 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-19T05:43:52.659 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-19T05:43:52.666 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T05:43:52.668 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:52.668 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:52.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:52.708 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T05:43:52.710 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:43:52.710 INFO:tasks.workunit.client.0.smithi002.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T05:43:52.719 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:52.717+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:52.750 INFO:tasks.workunit.client.0.smithi002.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T05:43:52.754 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:43:52.754 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T05:43:52.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.765+0000 7f987759e640 1 Processor -- start 2024-09-19T05:43:52.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.765+0000 7f987759e640 1 -- start start 2024-09-19T05:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987759e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f98780cb910 0x7f98780cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987759e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98780cc6f0 con 0x7f98780cb530 2024-09-19T05:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987759e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98780cc9d0 con 0x7f987815b7c0 2024-09-19T05:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987759e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f987815b330 con 0x7f98780cb910 2024-09-19T05:43:52.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987659c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f98780cb530 0x7f98780b2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44952/0 (socket says 172.21.15.2:44952) 2024-09-19T05:43:52.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987659c640 1 -- 172.21.15.2:0/727720180 learned_addr learned my addr 172.21.15.2:0/727720180 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f9876d9d640 1 --2- 172.21.15.2:0/727720180 >> v2:172.21.15.2:3300/0 conn(0x7f98780cb910 0x7f98780cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987559a640 1 -- 172.21.15.2:0/727720180 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681722183 0 0) 0x7f98780cc6f0 con 0x7f98780cb530 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987559a640 1 -- 172.21.15.2:0/727720180 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9860003520 con 0x7f98780cb530 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f987559a640 1 -- 172.21.15.2:0/727720180 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358847066 0 0) 0x7f98780cc9d0 con 0x7f987815b7c0 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f9876d9d640 1 -- 172.21.15.2:0/727720180 >> v1:172.21.15.123:6789/0 conn(0x7f987815b7c0 legacy=0x7f987815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f9876d9d640 1 -- 172.21.15.2:0/727720180 >> v1:172.21.15.2:6789/0 conn(0x7f98780cb530 legacy=0x7f98780b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.766+0000 7f9876d9d640 1 -- 172.21.15.2:0/727720180 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987815ace0 con 0x7f98780cb910 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.767+0000 7f9876d9d640 1 --2- 172.21.15.2:0/727720180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 0x7f98780cc040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9870009d90 tx=0x7f987002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470681f8dc650ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.767+0000 7f987559a640 1 -- 172.21.15.2:0/727720180 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870037070 con 0x7f98780cb910 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.767+0000 7f987559a640 1 -- 172.21.15.2:0/727720180 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9870007740 con 0x7f98780cb910 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.767+0000 7f987559a640 1 -- 172.21.15.2:0/727720180 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870007b10 con 0x7f98780cb910 2024-09-19T05:43:52.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.768+0000 7f987759e640 1 -- 172.21.15.2:0/727720180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 msgr2=0x7f98780cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.768+0000 7f987759e640 1 --2- 172.21.15.2:0/727720180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 0x7f98780cc040 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9870009d90 tx=0x7f987002a780 comp rx=0 tx=0).stop 2024-09-19T05:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.768+0000 7f987759e640 1 -- 172.21.15.2:0/727720180 shutdown_connections 2024-09-19T05:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.768+0000 7f987759e640 1 --2- 172.21.15.2:0/727720180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 0x7f98780cc040 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.768+0000 7f987759e640 1 -- 172.21.15.2:0/727720180 >> 172.21.15.2:0/727720180 conn(0x7f9878067b30 msgr2=0x7f9878067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.768+0000 7f987759e640 1 -- 172.21.15.2:0/727720180 shutdown_connections 2024-09-19T05:43:52.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.768+0000 7f987759e640 1 -- 172.21.15.2:0/727720180 wait complete. 2024-09-19T05:43:52.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.769+0000 7f987759e640 1 Processor -- start 2024-09-19T05:43:52.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.769+0000 7f987759e640 1 -- start start 2024-09-19T05:43:52.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f987759e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98780cb530 0x7f987815e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f987759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 0x7f9878155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f987759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987815b7c0 0x7f98781576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f987759e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9878154910 con 0x7f98780cb910 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f987759e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98780b3d40 con 0x7f987815b7c0 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f987759e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9878154a90 con 0x7f98780cb530 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9875d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 0x7f9878155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f987659c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98780cb530 0x7f987815e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9876d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987815b7c0 0x7f98781576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9875d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 0x7f9878155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38968/0 (socket says 172.21.15.2:38968) 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9875d9b640 1 -- 172.21.15.2:0/1069990319 learned_addr learned my addr 172.21.15.2:0/1069990319 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9875d9b640 1 -- 172.21.15.2:0/1069990319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98780cb530 msgr2=0x7f987815e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9875d9b640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98780cb530 0x7f987815e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9875d9b640 1 -- 172.21.15.2:0/1069990319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987815b7c0 msgr2=0x7f98781576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.770+0000 7f9875d9b640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987815b7c0 0x7f98781576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.771+0000 7f9875d9b640 1 -- 172.21.15.2:0/1069990319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9878156c10 con 0x7f98780cb910 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.771+0000 7f9875d9b640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98780cb910 0x7f9878155f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c3c0 tx=0x7f986c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.771+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c017070 con 0x7f98780cb910 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.771+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c0035f0 con 0x7f98780cb910 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.771+0000 7f987759e640 1 -- 172.21.15.2:0/1069990319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98781f4570 con 0x7f98780cb910 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.771+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c0039e0 con 0x7f98780cb910 2024-09-19T05:43:52.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.771+0000 7f987759e640 1 -- 172.21.15.2:0/1069990319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9878157190 con 0x7f98780cb910 2024-09-19T05:43:52.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.772+0000 7f987759e640 1 -- 172.21.15.2:0/1069990319 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98781f48b0 con 0x7f98780cb910 2024-09-19T05:43:52.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.773+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f986c01a070 con 0x7f98780cb910 2024-09-19T05:43:52.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.773+0000 7f9866ffd640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f985806c2b0 0x7f985806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.773+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7f986c08ae30 con 0x7f98780cb910 2024-09-19T05:43:52.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.773+0000 7f987659c640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f985806c2b0 0x7f985806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.773+0000 7f9866ffd640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f98580744b0 0x7f98580768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.773+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9858076f80 con 0x7f98580744b0 2024-09-19T05:43:52.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.774+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f986c08b2c0 con 0x7f98780cb910 2024-09-19T05:43:52.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.774+0000 7f9876d9d640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f98580744b0 0x7f98580768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.774+0000 7f987659c640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f985806c2b0 0x7f985806e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9868005880 tx=0x7f9868009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.774+0000 7f9876d9d640 1 --2- 172.21.15.2:0/1069990319 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f98580744b0 0x7f98580768d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.775+0000 7f9866ffd640 1 -- 172.21.15.2:0/1069990319 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9858076f80 con 0x7f98580744b0 2024-09-19T05:43:52.783 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-19T05:43:52.785 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-19T05:43:52.785 DEBUG:teuthology.orchestra.run.smithi002:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-19T05:43:52.802 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:52.802 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:52.802 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:52.802 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:52.802 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:52.802 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:52.803 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:52.803 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:52.803 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:52.803 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:52.803 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:52.803 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:52.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:52.805 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:52.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-19T05:43:52.822 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:52.822 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:52.824 INFO:tasks.workunit.client.0.smithi002.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-19T05:43:52.825 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:52.822+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:52.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:52.896 INFO:tasks.workunit.client.0.smithi002.stdout: Iozone: Performance Test of File I/O 2024-09-19T05:43:52.896 INFO:tasks.workunit.client.0.smithi002.stdout: Version $Revision: 3.494 $ 2024-09-19T05:43:52.896 INFO:tasks.workunit.client.0.smithi002.stdout: Compiled for 64 bit mode. 2024-09-19T05:43:52.896 INFO:tasks.workunit.client.0.smithi002.stdout: Build: linux-AMD64 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: Run began: Thu Sep 19 05:43:52 2024 2024-09-19T05:43:52.897 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Include close in write timing 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Include fsync in write timing 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: File size set to 1048576 kB 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Record Size 16 kB 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Output is in kBytes/sec 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Processor cache line size set to 32 bytes. 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: File stride size set to 17 * record size. 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Throughput test with 1 process 2024-09-19T05:43:52.898 INFO:tasks.workunit.client.0.smithi002.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-19T05:43:52.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:52.911 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:52.910+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:52.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.930+0000 7f6d4ea8d640 1 Processor -- start 2024-09-19T05:43:52.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.930+0000 7f6d4ea8d640 1 -- start start 2024-09-19T05:43:52.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4ea8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d4806c040 0x7f6d4815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4ea8d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6d480c8ee0 0x7f6d4815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4ea8d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d480739e0 con 0x7f6d48154d60 2024-09-19T05:43:52.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4ea8d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f6d4815ed70 con 0x7f6d480c8ee0 2024-09-19T05:43:52.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4ea8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4815e160 con 0x7f6d4806c040 2024-09-19T05:43:52.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4d02a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6d480c8ee0 0x7f6d4815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d47fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d4806c040 0x7f6d4815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4c829640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6d48154d60 0x7f6d480c8060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44966/0 (socket says 172.21.15.2:44966) 2024-09-19T05:43:52.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4c829640 1 -- 172.21.15.2:0/727743981 learned_addr learned my addr 172.21.15.2:0/727743981 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:52.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4d02a640 1 -- 172.21.15.2:0/727743981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d4806c040 msgr2=0x7f6d4815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:52.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4d02a640 1 --2- 172.21.15.2:0/727743981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d4806c040 0x7f6d4815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d4d02a640 1 -- 172.21.15.2:0/727743981 >> v1:172.21.15.2:6789/0 conn(0x7f6d48154d60 legacy=0x7f6d480c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.931+0000 7f6d477fe640 1 -- 172.21.15.2:0/727743981 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972179460 0 0) 0x7f6d480739e0 con 0x7f6d48154d60 2024-09-19T05:43:52.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.932+0000 7f6d4d02a640 1 -- 172.21.15.2:0/727743981 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4815e440 con 0x7f6d480c8ee0 2024-09-19T05:43:52.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.933+0000 7f6d4d02a640 1 --2- 172.21.15.2:0/727743981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d480c8ee0 0x7f6d4815b0b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009d90 tx=0x7f6d4002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5d5e5dae47d262 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.933+0000 7f6d477fe640 1 -- 172.21.15.2:0/727743981 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40007740 con 0x7f6d480c8ee0 2024-09-19T05:43:52.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.933+0000 7f6d477fe640 1 -- 172.21.15.2:0/727743981 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d400078e0 con 0x7f6d480c8ee0 2024-09-19T05:43:52.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.933+0000 7f6d477fe640 1 -- 172.21.15.2:0/727743981 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40007cb0 con 0x7f6d480c8ee0 2024-09-19T05:43:52.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.934+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/727743981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d480c8ee0 msgr2=0x7f6d4815b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.934+0000 7f6d4ea8d640 1 --2- 172.21.15.2:0/727743981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d480c8ee0 0x7f6d4815b0b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009d90 tx=0x7f6d4002a660 comp rx=0 tx=0).stop 2024-09-19T05:43:52.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.934+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/727743981 shutdown_connections 2024-09-19T05:43:52.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.934+0000 7f6d4ea8d640 1 --2- 172.21.15.2:0/727743981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d480c8ee0 0x7f6d4815b0b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.934+0000 7f6d4ea8d640 1 --2- 172.21.15.2:0/727743981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d4806c040 0x7f6d4815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.934+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/727743981 >> 172.21.15.2:0/727743981 conn(0x7f6d4806f3d0 msgr2=0x7f6d480707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:52.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.935+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/727743981 shutdown_connections 2024-09-19T05:43:52.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.935+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/727743981 wait complete. 2024-09-19T05:43:52.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.936+0000 7f6d4ea8d640 1 Processor -- start 2024-09-19T05:43:52.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.936+0000 7f6d4ea8d640 1 -- start start 2024-09-19T05:43:52.938 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:52.939 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4ea8d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4806c040 0x7f6d48155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4ea8d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d480c8ee0 0x7f6d48156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4ea8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d48154d60 0x7f6d48153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4ea8d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6d480c8b80 con 0x7f6d4806c040 2024-09-19T05:43:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4ea8d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4815ea00 con 0x7f6d480c8ee0 2024-09-19T05:43:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4ea8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6d480c8d00 con 0x7f6d48154d60 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4c829640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4806c040 0x7f6d48155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4d02a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d48154d60 0x7f6d48153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d47fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d480c8ee0 0x7f6d48156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4c829640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4806c040 0x7f6d48155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38984/0 (socket says 172.21.15.2:38984) 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4c829640 1 -- 172.21.15.2:0/1038084625 learned_addr learned my addr 172.21.15.2:0/1038084625 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.937+0000 7f6d4d02a640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d48154d60 0x7f6d48153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:33158/0 (socket says 172.21.15.2:33158) 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4c829640 1 -- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d48154d60 msgr2=0x7f6d48153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4c829640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6d48154d60 0x7f6d48153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4c829640 1 -- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d480c8ee0 msgr2=0x7f6d48156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4c829640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d480c8ee0 0x7f6d48156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4c829640 1 -- 172.21.15.2:0/1038084625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d481f4360 con 0x7f6d4806c040 2024-09-19T05:43:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4c829640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4806c040 0x7f6d48155f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00c430 tx=0x7f6d3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0074e0 con 0x7f6d4806c040 2024-09-19T05:43:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d3c0047e0 con 0x7f6d4806c040 2024-09-19T05:43:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/1038084625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d481554a0 con 0x7f6d4806c040 2024-09-19T05:43:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004bd0 con 0x7f6d4806c040 2024-09-19T05:43:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.938+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/1038084625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d480c9ed0 con 0x7f6d4806c040 2024-09-19T05:43:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.939+0000 7f6d4ea8d640 1 -- 172.21.15.2:0/1038084625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d481f4640 con 0x7f6d4806c040 2024-09-19T05:43:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.940+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6d3c011070 con 0x7f6d4806c040 2024-09-19T05:43:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.940+0000 7f6d457fa640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d2c06c2b0 0x7f6d2c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7f6d3c08af00 con 0x7f6d4806c040 2024-09-19T05:43:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d47fff640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d2c06c2b0 0x7f6d2c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d457fa640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6d2c0744b0 0x7f6d2c0768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d2c076f80 con 0x7f6d2c0744b0 2024-09-19T05:43:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f6d3c08d600 con 0x7f6d4806c040 2024-09-19T05:43:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d4d02a640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6d2c0744b0 0x7f6d2c0768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d4d02a640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6d2c0744b0 0x7f6d2c0768d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.941+0000 7f6d47fff640 1 --2- 172.21.15.2:0/1038084625 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d2c06c2b0 0x7f6d2c06e770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d480c3d50 tx=0x7f6d34016040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:52.942+0000 7f6d457fa640 1 -- 172.21.15.2:0/1038084625 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d2c076f80 con 0x7f6d2c0744b0 2024-09-19T05:43:52.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:52.998 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:52.998 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:52.998 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:52.998 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:52.998 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:52.998 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:52.998 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:52.999 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:52.999 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:52.999 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:52.999 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:52.999 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:53.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:53.002 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:53.002 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-19T05:43:53.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.032+0000 7fa87a798640 1 Processor -- start 2024-09-19T05:43:53.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.032+0000 7fa87a798640 1 -- start start 2024-09-19T05:43:53.035 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:53.033+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa87a798640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8740cd4e0 0x7fa8740b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa87a798640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa87415aa90 0x7fa8740b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa87a798640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa8740cd8c0 0x7fa87415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa87a798640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fa87415eec0 con 0x7fa87415aa90 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa87a798640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa87415ebe0 con 0x7fa8740cd8c0 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa87a798640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa8740b2e00 con 0x7fa8740cd4e0 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa8737fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa87415aa90 0x7fa8740b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa8737fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa87415aa90 0x7fa8740b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38986/0 (socket says 172.21.15.2:38986) 2024-09-19T05:43:53.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa8737fe640 1 -- 172.21.15.2:0/3424951683 learned_addr learned my addr 172.21.15.2:0/3424951683 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa873fff640 1 --2- 172.21.15.2:0/3424951683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8740cd4e0 0x7fa8740b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.033+0000 7fa878d35640 1 --2- 172.21.15.2:0/3424951683 >> v2:172.21.15.123:3300/0 conn(0x7fa8740cd8c0 0x7fa87415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.034+0000 7fa8737fe640 1 -- 172.21.15.2:0/3424951683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8740cd4e0 msgr2=0x7fa8740b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:53.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.034+0000 7fa8737fe640 1 --2- 172.21.15.2:0/3424951683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8740cd4e0 0x7fa8740b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.034+0000 7fa8737fe640 1 -- 172.21.15.2:0/3424951683 >> v2:172.21.15.123:3300/0 conn(0x7fa8740cd8c0 msgr2=0x7fa87415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.034+0000 7fa8737fe640 1 --2- 172.21.15.2:0/3424951683 >> v2:172.21.15.123:3300/0 conn(0x7fa8740cd8c0 0x7fa87415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.034+0000 7fa8737fe640 1 -- 172.21.15.2:0/3424951683 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8740b30e0 con 0x7fa87415aa90 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.035+0000 7fa8737fe640 1 --2- 172.21.15.2:0/3424951683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa87415aa90 0x7fa8740b4100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa87415edf0 tx=0x7fa85c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a97b3a9d0e3eef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.035+0000 7fa872ffd640 1 -- 172.21.15.2:0/3424951683 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c037070 con 0x7fa87415aa90 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.035+0000 7fa872ffd640 1 -- 172.21.15.2:0/3424951683 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa85c032440 con 0x7fa87415aa90 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.035+0000 7fa872ffd640 1 -- 172.21.15.2:0/3424951683 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c032810 con 0x7fa87415aa90 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 -- 172.21.15.2:0/3424951683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa87415aa90 msgr2=0x7fa8740b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 --2- 172.21.15.2:0/3424951683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa87415aa90 0x7fa8740b4100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa87415edf0 tx=0x7fa85c02a1c0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 -- 172.21.15.2:0/3424951683 shutdown_connections 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 --2- 172.21.15.2:0/3424951683 >> v2:172.21.15.123:3300/0 conn(0x7fa8740cd8c0 0x7fa87415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 --2- 172.21.15.2:0/3424951683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa87415aa90 0x7fa8740b4100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 --2- 172.21.15.2:0/3424951683 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8740cd4e0 0x7fa8740b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 -- 172.21.15.2:0/3424951683 >> 172.21.15.2:0/3424951683 conn(0x7fa874067b30 msgr2=0x7fa874067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:53.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 -- 172.21.15.2:0/3424951683 shutdown_connections 2024-09-19T05:43:53.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.036+0000 7fa87a798640 1 -- 172.21.15.2:0/3424951683 wait complete. 2024-09-19T05:43:53.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.037+0000 7fa87a798640 1 Processor -- start 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.037+0000 7fa87a798640 1 -- start start 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa87a798640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cd4e0 0x7fa8741e6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa87a798640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8740cd8c0 0x7fa8741e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa87a798640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa87415aa90 0x7fa8741f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa87a798640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa87415e470 con 0x7fa8740cd8c0 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa87a798640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740b36a0 con 0x7fa8740cd4e0 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa87a798640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa87415e5f0 con 0x7fa87415aa90 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa8737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8740cd8c0 0x7fa8741e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa8737fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8740cd8c0 0x7fa8741e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:38990/0 (socket says 172.21.15.2:38990) 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa873fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cd4e0 0x7fa8741e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa8737fe640 1 -- 172.21.15.2:0/1291832702 learned_addr learned my addr 172.21.15.2:0/1291832702 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.038+0000 7fa878d35640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa87415aa90 0x7fa8741f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa873fff640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa87415aa90 msgr2=0x7fa8741f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa87415aa90 0x7fa8741f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa873fff640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8740cd8c0 msgr2=0x7fa8741e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8740cd8c0 0x7fa8741e6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa873fff640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8741e4c50 con 0x7fa8740cd4e0 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cd4e0 0x7fa8741e6130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa86400e8d0 tx=0x7fa86400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864017070 con 0x7fa8740cd4e0 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86400c940 con 0x7fa8740cd4e0 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8741e3f10 con 0x7fa8740cd4e0 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86400ccf0 con 0x7fa8740cd4e0 2024-09-19T05:43:53.043 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.039+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741e51a0 con 0x7fa8740cd4e0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.040+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa8741e54e0 con 0x7fa8740cd4e0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.041+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa86401a070 con 0x7fa8740cd4e0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.041+0000 7fa870ff9640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa85406c2e0 0x7fa85406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.041+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7fa86408b5d0 con 0x7fa8740cd4e0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.041+0000 7fa8737fe640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa85406c2e0 0x7fa85406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.042+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7fa8741e54e0 con 0x7fa8740cd4e0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.042+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa8741f1670 0x7fa8740c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.042+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8740c86a0 con 0x7fa8741f1670 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.042+0000 7fa878d35640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa8741f1670 0x7fa8740c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.042+0000 7fa8737fe640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa85406c2e0 0x7fa85406e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa85c035040 tx=0x7fa85c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.042+0000 7fa878d35640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa8741f1670 0x7fa8740c99e0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.043+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8740c86a0 con 0x7fa8741f1670 2024-09-19T05:43:53.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.074+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa8740c83c0 con 0x7fa8741f1670 2024-09-19T05:43:53.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.076+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa8740c83c0 con 0x7fa8741f1670 2024-09-19T05:43:53.078 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.076+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa874156ad0 0x7fa874156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.076+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa87406d3d0 con 0x7fa874156ad0 2024-09-19T05:43:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.076+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa874156ad0 0x7fa874156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.077+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa874156ad0 0x7fa874156eb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.077+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa87406d3d0 con 0x7fa874156ad0 2024-09-19T05:43:53.093 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:53.094 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:53.103 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:53.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.104+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa8741573f0 con 0x7fa874156ad0 2024-09-19T05:43:53.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.106+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa8741573f0 con 0x7fa874156ad0 2024-09-19T05:43:53.109 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:53.109 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:53.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.107+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa8741f4040 0x7fa874062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.107+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa874157600 con 0x7fa8741f4040 2024-09-19T05:43:53.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.107+0000 7fa8737fe640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa8741f4040 0x7fa874062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.107+0000 7fa8737fe640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa8741f4040 0x7fa874062630 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.108+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa874157600 con 0x7fa8741f4040 2024-09-19T05:43:53.113 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:53.111+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:53.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.123+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa874062b70 con 0x7fa8741f4040 2024-09-19T05:43:53.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.124+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa874062b70 con 0x7fa8741f4040 2024-09-19T05:43:53.127 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:53.127 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.125+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa87415bb00 0x7fa87415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.125+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa87415c480 con 0x7fa87415bb00 2024-09-19T05:43:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.125+0000 7fa878d35640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa87415bb00 0x7fa87415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.125+0000 7fa878d35640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa87415bb00 0x7fa87415bf40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.126+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa87415c480 con 0x7fa87415bb00 2024-09-19T05:43:53.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.142+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa87415c480 con 0x7fa87415bb00 2024-09-19T05:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.144+0000 7efef0465640 1 Processor -- start 2024-09-19T05:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.144+0000 7efef0465640 1 -- start start 2024-09-19T05:43:53.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.144+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa87415c480 con 0x7fa87415bb00 2024-09-19T05:43:53.147 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:53.147 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:53.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa87402cb50 0x7fa87402cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa87415c910 con 0x7fa87402cb50 2024-09-19T05:43:53.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa87402cb50 0x7fa87402cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7efef0465640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efee8150fd0 0x7efee80c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7efef0465640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efee80b2d70 0x7efee80c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7efef0465640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee80c3730 con 0x7efee80736e0 2024-09-19T05:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7efef0465640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7efee80c92c0 con 0x7efee80b2d70 2024-09-19T05:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.145+0000 7efef0465640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efee80c39d0 con 0x7efee8150fd0 2024-09-19T05:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeee201640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efee8150fd0 0x7efee80c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeeda00640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efee80b2d70 0x7efee80c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeee201640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efee8150fd0 0x7efee80c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:38288/0 (socket says 172.21.15.2:38288) 2024-09-19T05:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeee201640 1 -- 172.21.15.2:0/3686217159 learned_addr learned my addr 172.21.15.2:0/3686217159 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:53.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa87402cb50 0x7fa87402cf90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeed1ff640 1 -- 172.21.15.2:0/3686217159 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 563013762 0 0) 0x7efee80c3730 con 0x7efee80736e0 2024-09-19T05:43:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeed1ff640 1 -- 172.21.15.2:0/3686217159 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efed4003520 con 0x7efee80736e0 2024-09-19T05:43:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeee201640 1 -- 172.21.15.2:0/3686217159 >> v2:172.21.15.2:3300/0 conn(0x7efee80b2d70 msgr2=0x7efee80c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeee201640 1 --2- 172.21.15.2:0/3686217159 >> v2:172.21.15.2:3300/0 conn(0x7efee80b2d70 0x7efee80c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeee201640 1 -- 172.21.15.2:0/3686217159 >> v1:172.21.15.2:6789/0 conn(0x7efee80736e0 legacy=0x7efee815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7efeee201640 1 -- 172.21.15.2:0/3686217159 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee80c3cb0 con 0x7efee8150fd0 2024-09-19T05:43:53.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.146+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa87415c910 con 0x7fa87402cb50 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.147+0000 7efeee201640 1 --2- 172.21.15.2:0/3686217159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee8150fd0 0x7efee80c3000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efee806f720 tx=0x7efee402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d69397298e9d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efeed1ff640 1 -- 172.21.15.2:0/3686217159 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4037070 con 0x7efee8150fd0 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efeed1ff640 1 -- 172.21.15.2:0/3686217159 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee402ae50 con 0x7efee8150fd0 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efeed1ff640 1 -- 172.21.15.2:0/3686217159 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4032860 con 0x7efee8150fd0 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efef0465640 1 -- 172.21.15.2:0/3686217159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee8150fd0 msgr2=0x7efee80c3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efef0465640 1 --2- 172.21.15.2:0/3686217159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee8150fd0 0x7efee80c3000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efee806f720 tx=0x7efee402a300 comp rx=0 tx=0).stop 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efef0465640 1 -- 172.21.15.2:0/3686217159 shutdown_connections 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efef0465640 1 --2- 172.21.15.2:0/3686217159 >> v2:172.21.15.2:3300/0 conn(0x7efee80b2d70 0x7efee80c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efef0465640 1 --2- 172.21.15.2:0/3686217159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee8150fd0 0x7efee80c3000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.148+0000 7efef0465640 1 -- 172.21.15.2:0/3686217159 >> 172.21.15.2:0/3686217159 conn(0x7efee80cd1d0 msgr2=0x7efee80c96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:53.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.149+0000 7efef0465640 1 -- 172.21.15.2:0/3686217159 shutdown_connections 2024-09-19T05:43:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.149+0000 7efef0465640 1 -- 172.21.15.2:0/3686217159 wait complete. 2024-09-19T05:43:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.151+0000 7efef0465640 1 Processor -- start 2024-09-19T05:43:53.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.151+0000 7efef0465640 1 -- start start 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efef0465640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee80736e0 0x7efee80cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efef0465640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efee80b2d70 0x7efee81ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efef0465640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efee8150fd0 0x7efee81ee370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efef0465640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efee81576e0 con 0x7efee8150fd0 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efef0465640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efee8157400 con 0x7efee80736e0 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efef0465640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efee8073280 con 0x7efee80b2d70 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efeeda00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efee80b2d70 0x7efee81ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efeee201640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee80736e0 0x7efee80cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efeeea02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efee8150fd0 0x7efee81ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efeeda00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efee80b2d70 0x7efee81ede30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:33182/0 (socket says 172.21.15.2:33182) 2024-09-19T05:43:53.155 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efeee201640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee80736e0 0x7efee80cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:38292/0 (socket says 172.21.15.2:38292) 2024-09-19T05:43:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.152+0000 7efeeda00640 1 -- 172.21.15.2:0/3807435247 learned_addr learned my addr 172.21.15.2:0/3807435247 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.153+0000 7efeeda00640 1 -- 172.21.15.2:0/3807435247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee80736e0 msgr2=0x7efee80cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.153+0000 7efeeda00640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee80736e0 0x7efee80cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.156 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.153+0000 7efeeda00640 1 -- 172.21.15.2:0/3807435247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efee8150fd0 msgr2=0x7efee81ee370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.153+0000 7efeeda00640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efee8150fd0 0x7efee81ee370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.153+0000 7efeeda00640 1 -- 172.21.15.2:0/3807435247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee815a780 con 0x7efee80b2d70 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.153+0000 7efeee201640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee80736e0 0x7efee80cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.153+0000 7efeeda00640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efee80b2d70 0x7efee81ede30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efed8010df0 tx=0x7efed800e730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.154+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed801e070 con 0x7efee80b2d70 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.154+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed8011660 con 0x7efee80b2d70 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.154+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8011a50 con 0x7efee80b2d70 2024-09-19T05:43:53.157 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.154+0000 7efef0465640 1 -- 172.21.15.2:0/3807435247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee8159a90 con 0x7efee80b2d70 2024-09-19T05:43:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.154+0000 7efef0465640 1 -- 172.21.15.2:0/3807435247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81ed630 con 0x7efee80b2d70 2024-09-19T05:43:53.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.155+0000 7efef0465640 1 -- 172.21.15.2:0/3807435247 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efee8159560 con 0x7efee80b2d70 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.156+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efed8021070 con 0x7efee80b2d70 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.156+0000 7efedeffd640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efecc06c2b0 0x7efecc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.156+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7efed808dff0 con 0x7efee80b2d70 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.157+0000 7efeee201640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efecc06c2b0 0x7efecc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.157+0000 7efedeffd640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efecc0744b0 0x7efecc0768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.157+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efed800ce50 con 0x7efecc0744b0 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.157+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7efed8095050 con 0x7efee80b2d70 2024-09-19T05:43:53.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.157+0000 7efeeea02640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efecc0744b0 0x7efecc0768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.157+0000 7efeeea02640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efecc0744b0 0x7efecc0768d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.158+0000 7efeee201640 1 --2- 172.21.15.2:0/3807435247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efecc06c2b0 0x7efecc06e770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efee4034000 tx=0x7efee402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.160 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.158+0000 7efedeffd640 1 -- 172.21.15.2:0/3807435247 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efed800ce50 con 0x7efecc0744b0 2024-09-19T05:43:53.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.162+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa87402d4d0 con 0x7fa87402cb50 2024-09-19T05:43:53.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.164+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa87402d4d0 con 0x7fa87402cb50 2024-09-19T05:43:53.167 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:53.167 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:53.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.165+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa874204af0 0x7fa874206ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.165+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa87415cac0 con 0x7fa874204af0 2024-09-19T05:43:53.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.165+0000 7fa8737fe640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa874204af0 0x7fa874206ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.166+0000 7fa8737fe640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa874204af0 0x7fa874206ee0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.166+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa85c032cf0 con 0x7fa874204af0 2024-09-19T05:43:53.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.180+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa87402da10 con 0x7fa874204af0 2024-09-19T05:43:53.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.182+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa87402da10 con 0x7fa874204af0 2024-09-19T05:43:53.184 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.183+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa874207930 0x7fa874209d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.183+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa87420cba0 con 0x7fa874207930 2024-09-19T05:43:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.183+0000 7fa878d35640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa874207930 0x7fa874209d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.183+0000 7fa878d35640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa874207930 0x7fa874209d20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.184+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa87420cba0 con 0x7fa874207930 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:53.187 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:53.190 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:53.191 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:53.191 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-19T05:43:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.200+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa87420a530 con 0x7fa874207930 2024-09-19T05:43:53.204 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.201+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa87420a530 con 0x7fa874207930 2024-09-19T05:43:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.202+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa87420ac50 0x7fa87420b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.202+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa87420ff70 con 0x7fa87420ac50 2024-09-19T05:43:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.202+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa87420ac50 0x7fa87420b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.205 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.203+0000 7fa873fff640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa87420ac50 0x7fa87420b090 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.206 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.203+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa87420ff70 con 0x7fa87420ac50 2024-09-19T05:43:53.214 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:53.212+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:53.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.218+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fa87420ee70 con 0x7fa87420ac50 2024-09-19T05:43:53.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa870ff9640 1 -- 172.21.15.2:0/1291832702 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa87420ee70 con 0x7fa87420ac50 2024-09-19T05:43:53.222 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa8741f1670 msgr2=0x7fa8740c99e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fa8741f1670 0x7fa8740c99e0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa874156ad0 msgr2=0x7fa874156eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa874156ad0 0x7fa874156eb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa8741f4040 msgr2=0x7fa874062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fa8741f4040 0x7fa874062630 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa87415bb00 msgr2=0x7fa87415bf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.224 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa87415bb00 0x7fa87415bf40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa87402cb50 msgr2=0x7fa87402cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fa87402cb50 0x7fa87402cf90 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa874204af0 msgr2=0x7fa874206ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.220+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fa874204af0 0x7fa874206ee0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa874207930 msgr2=0x7fa874209d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fa874207930 0x7fa874209d20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa87420ac50 msgr2=0x7fa87420b090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fa87420ac50 0x7fa87420b090 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa85406c2e0 msgr2=0x7fa85406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa878d35640 1 -- 172.21.15.2:0/1291832702 reap_dead start 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa85406c2e0 0x7fa85406e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa85c035040 tx=0x7fa85c02e000 comp rx=0 tx=0).stop 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa878d35640 1 -- 172.21.15.2:0/1291832702 reap_dead start 2024-09-19T05:43:53.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa878d35640 1 -- 172.21.15.2:0/1291832702 reap_dead start 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa878d35640 1 -- 172.21.15.2:0/1291832702 reap_dead start 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa878d35640 1 -- 172.21.15.2:0/1291832702 reap_dead start 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cd4e0 msgr2=0x7fa8741e6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cd4e0 0x7fa8741e6130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa86400e8d0 tx=0x7fa86400ed90 comp rx=0 tx=0).stop 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 shutdown_connections 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa85406c2e0 0x7fa85406e7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 --2- 172.21.15.2:0/1291832702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8740cd4e0 0x7fa8741e6130 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 >> 172.21.15.2:0/1291832702 conn(0x7fa874067b30 msgr2=0x7fa8741585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 shutdown_connections 2024-09-19T05:43:53.226 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.221+0000 7fa87a798640 1 -- 172.21.15.2:0/1291832702 wait complete. 2024-09-19T05:43:53.227 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:53.227 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:53.227 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:53.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:53.314 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:53.313+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:53.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.384+0000 7f667759e640 1 Processor -- start 2024-09-19T05:43:53.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.386+0000 7f667759e640 1 -- start start 2024-09-19T05:43:53.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.389+0000 7f667759e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d08d0 0x7f66780c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.393+0000 7f667659c640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f66780d04f0 0x7f667815b460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:41784/0 (socket says 172.21.15.2:41784) 2024-09-19T05:43:53.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.399+0000 7f667759e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f66780c9290 0x7f66780c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.399+0000 7f667759e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66780c7940 con 0x7f66780d04f0 2024-09-19T05:43:53.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.399+0000 7f667759e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f66780c7710 con 0x7f66780c9290 2024-09-19T05:43:53.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.399+0000 7f667759e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f66780c88c0 con 0x7f66780d08d0 2024-09-19T05:43:53.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.400+0000 7f6675d9b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d08d0 0x7f66780c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.400+0000 7f667659c640 1 -- 172.21.15.2:0/668630508 learned_addr learned my addr 172.21.15.2:0/668630508 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:53.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.404+0000 7f6676d9d640 1 --2- 172.21.15.2:0/668630508 >> v2:172.21.15.123:3300/0 conn(0x7f66780c9290 0x7f66780c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.405+0000 7f667559a640 1 -- 172.21.15.2:0/668630508 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140680036 0 0) 0x7f66780c7940 con 0x7f66780d04f0 2024-09-19T05:43:53.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.405+0000 7f667559a640 1 -- 172.21.15.2:0/668630508 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6660003520 con 0x7f66780d04f0 2024-09-19T05:43:53.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.406+0000 7f6676d9d640 1 -- 172.21.15.2:0/668630508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d08d0 msgr2=0x7f66780c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:53.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.406+0000 7f6676d9d640 1 --2- 172.21.15.2:0/668630508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d08d0 0x7f66780c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.406+0000 7f6676d9d640 1 -- 172.21.15.2:0/668630508 >> v1:172.21.15.123:6789/0 conn(0x7f66780d04f0 legacy=0x7f667815b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.407+0000 7f6676d9d640 1 -- 172.21.15.2:0/668630508 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66780c7c00 con 0x7f66780c9290 2024-09-19T05:43:53.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.412+0000 7f6676d9d640 1 --2- 172.21.15.2:0/668630508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66780c9290 0x7f66780c6490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f666c0097a0 tx=0x7f666c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e5e0c4147b1619 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.414+0000 7f667559a640 1 -- 172.21.15.2:0/668630508 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c037070 con 0x7f66780c9290 2024-09-19T05:43:53.416 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:53.414+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:53.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.414+0000 7f667559a640 1 -- 172.21.15.2:0/668630508 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666c032440 con 0x7f66780c9290 2024-09-19T05:43:53.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.419+0000 7f667559a640 1 -- 172.21.15.2:0/668630508 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c032850 con 0x7f66780c9290 2024-09-19T05:43:53.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.423+0000 7f667759e640 1 -- 172.21.15.2:0/668630508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66780c9290 msgr2=0x7f66780c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.423+0000 7f667759e640 1 --2- 172.21.15.2:0/668630508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66780c9290 0x7f66780c6490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f666c0097a0 tx=0x7f666c02a230 comp rx=0 tx=0).stop 2024-09-19T05:43:53.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.423+0000 7f667759e640 1 -- 172.21.15.2:0/668630508 shutdown_connections 2024-09-19T05:43:53.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.423+0000 7f667759e640 1 --2- 172.21.15.2:0/668630508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66780c9290 0x7f66780c6490 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.423+0000 7f667759e640 1 --2- 172.21.15.2:0/668630508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d08d0 0x7f66780c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.423+0000 7f667759e640 1 -- 172.21.15.2:0/668630508 >> 172.21.15.2:0/668630508 conn(0x7f667806cb20 msgr2=0x7f667806e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:53.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.430+0000 7f667759e640 1 -- 172.21.15.2:0/668630508 shutdown_connections 2024-09-19T05:43:53.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.430+0000 7f667759e640 1 -- 172.21.15.2:0/668630508 wait complete. 2024-09-19T05:43:53.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.433+0000 7f667759e640 1 Processor -- start 2024-09-19T05:43:53.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.438+0000 7f667759e640 1 -- start start 2024-09-19T05:43:53.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.440+0000 7f667759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66780c9290 0x7f6678200970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.440+0000 7f667759e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d04f0 0x7f667806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.440+0000 7f667759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66780d08d0 0x7f66780ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.440+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66780d08d0 0x7f66780ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.441+0000 7f6676d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66780d08d0 0x7f66780ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39024/0 (socket says 172.21.15.2:39024) 2024-09-19T05:43:53.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.441+0000 7f6676d9d640 1 -- 172.21.15.2:0/1966146045 learned_addr learned my addr 172.21.15.2:0/1966146045 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:53.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.441+0000 7f667759e640 1 -- 172.21.15.2:0/1966146045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f66780c8550 con 0x7f66780d08d0 2024-09-19T05:43:53.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.441+0000 7f667759e640 1 -- 172.21.15.2:0/1966146045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66780c8270 con 0x7f66780c9290 2024-09-19T05:43:53.444 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.441+0000 7f667759e640 1 -- 172.21.15.2:0/1966146045 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f667815bea0 con 0x7f66780d04f0 2024-09-19T05:43:53.446 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.443+0000 7f6676d9d640 1 -- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d04f0 msgr2=0x7f667806f960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:43:53.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.443+0000 7f6676d9d640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66780d04f0 0x7f667806f960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.443+0000 7f6676d9d640 1 -- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66780c9290 msgr2=0x7f6678200970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:43:53.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.443+0000 7f6676d9d640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66780c9290 0x7f6678200970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.443+0000 7f6676d9d640 1 -- 172.21.15.2:0/1966146045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66781fc9f0 con 0x7f66780d08d0 2024-09-19T05:43:53.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.443+0000 7f6676d9d640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66780d08d0 0x7f66780ccfc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f666c02a820 tx=0x7f666c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.445+0000 7f6666ffd640 1 -- 172.21.15.2:0/1966146045 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c040020 con 0x7f66780d08d0 2024-09-19T05:43:53.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.446+0000 7f6666ffd640 1 -- 172.21.15.2:0/1966146045 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666c038040 con 0x7f66780d08d0 2024-09-19T05:43:53.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.446+0000 7f667759e640 1 -- 172.21.15.2:0/1966146045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667806f0d0 con 0x7f66780d08d0 2024-09-19T05:43:53.454 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.452+0000 7f667759e640 1 -- 172.21.15.2:0/1966146045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781fccb0 con 0x7f66780d08d0 2024-09-19T05:43:53.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.456+0000 7f6666ffd640 1 -- 172.21.15.2:0/1966146045 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c037070 con 0x7f66780d08d0 2024-09-19T05:43:53.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.457+0000 7f6666ffd640 1 -- 172.21.15.2:0/1966146045 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f666c043020 con 0x7f66780d08d0 2024-09-19T05:43:53.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.460+0000 7f6666ffd640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f665806c3b0 0x7f665806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.460+0000 7f6666ffd640 1 -- 172.21.15.2:0/1966146045 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7f666c0ab1d0 con 0x7f66780d08d0 2024-09-19T05:43:53.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.461+0000 7f6664ff9640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f665c001650 0x7f665c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.461+0000 7f6664ff9640 1 -- 172.21.15.2:0/1966146045 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f665c006bb0 con 0x7f665c001650 2024-09-19T05:43:53.464 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.461+0000 7f667659c640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f665806c3b0 0x7f665806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.467+0000 7f6675d9b640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f665c001650 0x7f665c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.470+0000 7f667659c640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f665806c3b0 0x7f665806e870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6668000f60 tx=0x7f66680010a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.471+0000 7f6675d9b640 1 --2- 172.21.15.2:0/1966146045 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f665c001650 0x7f665c003b10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.473+0000 7f6666ffd640 1 -- 172.21.15.2:0/1966146045 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f665c006bb0 con 0x7f665c001650 2024-09-19T05:43:53.516 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:53.515+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:53.612 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:53.617 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:53.616+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:53.619 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:53.620 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:53.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-19T05:43:53.641 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:53.642 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:53.676 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:53.718 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:53.716+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:53.819 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:53.818+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:53.920 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:53.918+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:53.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.928+0000 7f627384c640 1 Processor -- start 2024-09-19T05:43:53.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.928+0000 7f627384c640 1 -- start start 2024-09-19T05:43:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f627384c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c06c040 0x7f626c15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f627384c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f626c0c8ee0 0x7f626c15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f627384c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626c0739e0 con 0x7f626c154d60 2024-09-19T05:43:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f627384c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f626c15ed70 con 0x7f626c0c8ee0 2024-09-19T05:43:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f627384c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f626c15e160 con 0x7f626c06c040 2024-09-19T05:43:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f62715e8640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f626c154d60 0x7f626c0c8060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44984/0 (socket says 172.21.15.2:44984) 2024-09-19T05:43:53.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f62715e8640 1 -- 172.21.15.2:0/3382335361 learned_addr learned my addr 172.21.15.2:0/3382335361 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:53.932 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:43:53.932 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 initial writers = 1014353.81 kB/sec 2024-09-19T05:43:53.932 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 initial writers = 1014146.64 kB/sec 2024-09-19T05:43:53.932 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 1014353.81 kB/sec 2024-09-19T05:43:53.932 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 1014353.81 kB/sec 2024-09-19T05:43:53.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f6271de9640 1 --2- 172.21.15.2:0/3382335361 >> v2:172.21.15.2:3300/0 conn(0x7f626c0c8ee0 0x7f626c15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f6270de7640 1 --2- 172.21.15.2:0/3382335361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c06c040 0x7f626c15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.933 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 1014353.81 kB/sec 2024-09-19T05:43:53.933 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:43:53.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f6271de9640 1 -- 172.21.15.2:0/3382335361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c06c040 msgr2=0x7f626c15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f6271de9640 1 --2- 172.21.15.2:0/3382335361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c06c040 0x7f626c15ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f6271de9640 1 -- 172.21.15.2:0/3382335361 >> v1:172.21.15.2:6789/0 conn(0x7f626c154d60 legacy=0x7f626c0c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f6271de9640 1 -- 172.21.15.2:0/3382335361 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c15e440 con 0x7f626c0c8ee0 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.929+0000 7f6263fff640 1 -- 172.21.15.2:0/3382335361 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852397610 0 0) 0x7f626c0739e0 con 0x7f626c154d60 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.930+0000 7f6271de9640 1 --2- 172.21.15.2:0/3382335361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c0c8ee0 0x7f626c15b0b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6264009d90 tx=0x7f626402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ed90e07fd18618 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.931+0000 7f6263fff640 1 -- 172.21.15.2:0/3382335361 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264007740 con 0x7f626c0c8ee0 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.931+0000 7f6263fff640 1 -- 172.21.15.2:0/3382335361 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62640078e0 con 0x7f626c0c8ee0 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.931+0000 7f6263fff640 1 -- 172.21.15.2:0/3382335361 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264007cb0 con 0x7f626c0c8ee0 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 -- 172.21.15.2:0/3382335361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c0c8ee0 msgr2=0x7f626c15b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 --2- 172.21.15.2:0/3382335361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c0c8ee0 0x7f626c15b0b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6264009d90 tx=0x7f626402a660 comp rx=0 tx=0).stop 2024-09-19T05:43:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 -- 172.21.15.2:0/3382335361 shutdown_connections 2024-09-19T05:43:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 --2- 172.21.15.2:0/3382335361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c0c8ee0 0x7f626c15b0b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 --2- 172.21.15.2:0/3382335361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c06c040 0x7f626c15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 -- 172.21.15.2:0/3382335361 >> 172.21.15.2:0/3382335361 conn(0x7f626c06f3d0 msgr2=0x7f626c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 -- 172.21.15.2:0/3382335361 shutdown_connections 2024-09-19T05:43:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.932+0000 7f627384c640 1 -- 172.21.15.2:0/3382335361 wait complete. 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.933+0000 7f627384c640 1 Processor -- start 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.933+0000 7f627384c640 1 -- start start 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f627384c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f626c06c040 0x7f626c1f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f627384c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c0c8ee0 0x7f626c159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f627384c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c154d60 0x7f626c155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f627384c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f626c0c8b80 con 0x7f626c154d60 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f627384c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f626c15ea00 con 0x7f626c06c040 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f627384c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f626c0c8d00 con 0x7f626c0c8ee0 2024-09-19T05:43:53.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c154d60 0x7f626c155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c154d60 0x7f626c155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39050/0 (socket says 172.21.15.2:39050) 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 -- 172.21.15.2:0/1474090965 learned_addr learned my addr 172.21.15.2:0/1474090965 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f62715e8640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f626c06c040 0x7f626c1f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6270de7640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c0c8ee0 0x7f626c159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 -- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c0c8ee0 msgr2=0x7f626c159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f626c0c8ee0 0x7f626c159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 -- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f626c06c040 msgr2=0x7f626c1f0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f626c06c040 0x7f626c1f0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.934+0000 7f6271de9640 1 -- 172.21.15.2:0/1474090965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c1f1320 con 0x7f626c154d60 2024-09-19T05:43:53.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.935+0000 7f6271de9640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f626c154d60 0x7f626c155b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6264009eb0 tx=0x7f6264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.935+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264040070 con 0x7f626c154d60 2024-09-19T05:43:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.935+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6264031b40 con 0x7f626c154d60 2024-09-19T05:43:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.935+0000 7f627384c640 1 -- 172.21.15.2:0/1474090965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c156d90 con 0x7f626c154d60 2024-09-19T05:43:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.935+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6264045460 con 0x7f626c154d60 2024-09-19T05:43:53.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.936+0000 7f627384c640 1 -- 172.21.15.2:0/1474090965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c157350 con 0x7f626c154d60 2024-09-19T05:43:53.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.936+0000 7f627384c640 1 -- 172.21.15.2:0/1474090965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f626c155700 con 0x7f626c154d60 2024-09-19T05:43:53.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.937+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6264039040 con 0x7f626c154d60 2024-09-19T05:43:53.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.937+0000 7f6261ffb640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f625006c300 0x7f625006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.937+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7f62640ac210 con 0x7f626c154d60 2024-09-19T05:43:53.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.937+0000 7f6261ffb640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6250074520 0x7f6250076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:53.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.937+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6250076ff0 con 0x7f6250074520 2024-09-19T05:43:53.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.937+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f62640b1060 con 0x7f626c154d60 2024-09-19T05:43:53.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.937+0000 7f62715e8640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f625006c300 0x7f625006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.938+0000 7f6270de7640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6250074520 0x7f6250076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:53.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.938+0000 7f62715e8640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f625006c300 0x7f625006e7c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6268000f60 tx=0x7f6268007e40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.938+0000 7f6270de7640 1 --2- 172.21.15.2:0/1474090965 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6250074520 0x7f6250076940 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:53.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:53.939+0000 7f6261ffb640 1 -- 172.21.15.2:0/1474090965 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6250076ff0 con 0x7f6250074520 2024-09-19T05:43:53.965 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:53.965 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:53.965 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:53.966 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:53.967 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:12 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.67959239100000002, 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.301330818, 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:53.968 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 12 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:43:53.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285078+0000", 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8979999999999998e-06 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288881+0000", 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038028900000000002 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:53.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288908+0000", 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6831000000000001e-05 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288922+0000", 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4647999999999999e-05 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289034+0000", 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011129 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289158+0000", 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012398199999999999 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289283+0000", 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012596100000000001 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302442+0000", 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01315887 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586307+0000", 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28386498300000002 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586357+0000", 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9406000000000001e-05 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586405+0000", 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8059000000000002e-05 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.973 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:16 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66546865099999997, 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28733763699999998, 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:53.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 16 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299338+0000", 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013993499999999999 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305742+0000", 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064047180000000002 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305795+0000", 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2988000000000002e-05 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305808+0000", 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.307e-05 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305883+0000", 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4265999999999997e-05 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306078+0000", 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195281 2024-09-19T05:43:53.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306298+0000", 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000219662 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.319662+0000", 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0133641 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586451+0000", 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26678922700000002 2024-09-19T05:43:53.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586494+0000", 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3102999999999997e-05 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586535+0000", 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1287e-05 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:36 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.64813026399999996, 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54259264799999996, 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316539+0000", 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1310000000000002e-06 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326313+0000", 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0097740320000000002 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326341+0000", 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8178e-05 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326354+0000", 2024-09-19T05:43:53.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2767e-05 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326454+0000", 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9741999999999995e-05 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326598+0000", 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000144454 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:53.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326740+0000", 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014189899999999999 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.381404+0000", 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054663549999999998 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859031+0000", 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47762663900000002 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859074+0000", 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3578999999999997e-05 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859129+0000", 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4676999999999998e-05 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:40 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:43:53.985 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.63824091699999996, 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29103886899999998, 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 40 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326428+0000", 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9160000000000001e-06 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:53.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343603+0000", 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017174545999999999 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343619+0000", 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6260999999999999e-05 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343630+0000", 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0774999999999999e-05 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343729+0000", 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8694000000000001e-05 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343846+0000", 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117588 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343969+0000", 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012271600000000001 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.376370+0000", 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032401093999999998 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617348+0000", 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24097808100000001 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617405+0000", 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6577999999999999e-05 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617464+0000", 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9620000000000002e-05 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:76 3.3f 3:ff5d5af4:::10000000002.00000049:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.392283+0000", 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.57238299100000001, 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.178010692, 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 76 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.392283+0000", 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.392283+0000", 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.392287+0000", 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8380000000000004e-06 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.396676+0000", 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043888399999999998 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:53.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.396692+0000", 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6359999999999999e-05 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.396702+0000", 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.611e-06 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.396799+0000", 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6948999999999996e-05 2024-09-19T05:43:53.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.397606+0000", 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00080693900000000001 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.397895+0000", 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028923300000000001 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:53.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415614+0000", 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017719271000000002 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570172+0000", 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.154557678 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570238+0000", 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5984999999999999e-05 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570294+0000", 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5988e-05 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:81 3.34 3:2c8ffcca:::10000000002.0000004e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430277+0000", 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.53438976400000004, 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.191051688, 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:53.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 81 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430277+0000", 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430277+0000", 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430280+0000", 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2909999999999999e-06 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436911+0000", 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066311540000000002 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:53.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436939+0000", 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7449000000000001e-05 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436958+0000", 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9961000000000001e-05 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.441769+0000", 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048106640000000001 2024-09-19T05:43:54.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.442074+0000", 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030465400000000001 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.442286+0000", 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021271199999999999 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.498387+0000", 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056100160000000003 2024-09-19T05:43:54.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621195+0000", 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12280806800000001 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621253+0000", 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8362000000000001e-05 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621328+0000", 2024-09-19T05:43:54.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5213e-05 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:89 3.15 3:ab7657d5:::10000000002.00000056:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.445212+0000", 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51945437699999997, 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.124686667, 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 89 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445212+0000", 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445212+0000", 2024-09-19T05:43:54.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445215+0000", 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.453440+0000", 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0082254119999999997 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.454292+0000", 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00085143399999999998 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.454309+0000", 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7821000000000001e-05 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.474539+0000", 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020229726 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.474742+0000", 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203141 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.474901+0000", 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158479 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.523627+0000", 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048725875000000002 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569764+0000", 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046137308000000002 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569840+0000", 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5659000000000002e-05 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569899+0000", 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9089000000000003e-05 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:100 3.34 3:2d119e8b:::10000000002.00000061:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.454397+0000", 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51026963400000003, 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16707623299999999, 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 100 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.454397+0000", 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.454397+0000", 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.454399+0000", 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.605e-06 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.464586+0000", 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010186172 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467024+0000", 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024386579999999998 2024-09-19T05:43:54.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467046+0000", 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.173e-05 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.482052+0000", 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0150061 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.484464+0000", 2024-09-19T05:43:54.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024117840000000001 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.484691+0000", 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022731300000000001 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.507526+0000", 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022835243000000002 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621380+0000", 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113853344 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621433+0000", 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3678999999999999e-05 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621473+0000", 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9604999999999998e-05 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:101 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.497555215, 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.392178684, 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 101 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467114+0000", 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.269e-06 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.477820+0000", 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010705696000000001 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.500766+0000", 2024-09-19T05:43:54.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022946191000000001 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508150+0000", 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073836930000000002 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508253+0000", 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103222 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508541+0000", 2024-09-19T05:43:54.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028815600000000002 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508725+0000", 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183766 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.543910+0000", 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035184897 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859196+0000", 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31528614500000002 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859243+0000", 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6727000000000002e-05 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859290+0000", 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6922000000000002e-05 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:117 3.36 3:6d44e697:::10000000002.00000072:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.604765+0000", 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.35990102800000001, 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.140929367, 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 117 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.021 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604765+0000", 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604765+0000", 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604770+0000", 2024-09-19T05:43:54.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.296e-06 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.608639+0000", 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038696310000000001 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.608680+0000", 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0760999999999998e-05 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.608696+0000", 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5565000000000001e-05 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.608794+0000", 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8889999999999997e-05 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.608974+0000", 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017979899999999999 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.609168+0000", 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019370699999999999 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.618714+0000", 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0095458379999999992 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.745539+0000", 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.126825153 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.745602+0000", 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3318999999999995e-05 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.745695+0000", 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2408e-05 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:92 3.1c e53/38 3:3aa14ed1:::10000000002.00000059:head v 53'3, mlcod=0'0)", 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.614885+0000", 2024-09-19T05:43:54.026 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34978126300000001, 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.166456355, 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614885+0000", 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614885+0000", 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614886+0000", 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2100000000000002e-07 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.619895+0000", 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050090220000000001 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.619911+0000", 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6599000000000002e-05 2024-09-19T05:43:54.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.619925+0000", 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3501e-05 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621012+0000", 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001087597 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621113+0000", 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100709 2024-09-19T05:43:54.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.781269+0000", 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.160156246 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.781341+0000", 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2260000000000003e-05 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.030 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:139 3.1c e53/38 3:384ad1d9:::10000000002.00000088:head v 53'4, mlcod=0'0)", 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.624566+0000", 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.340100817, 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16268069099999999, 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.624566+0000", 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.624566+0000", 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.624567+0000", 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2100000000000001e-06 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.647114+0000", 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022547627000000001 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.647139+0000", 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5055e-05 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.647155+0000", 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5993000000000001e-05 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.647802+0000", 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064620599999999997 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.647828+0000", 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6047e-05 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.787083+0000", 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.139255401 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.787246+0000", 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016315200000000001 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.034 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:121 3.15 3:abd4a84e:::10000000002.00000076:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.624594+0000", 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34007224800000002, 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.132214675, 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 121 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.624594+0000", 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.624594+0000", 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.624597+0000", 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2799999999999999e-06 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.628710+0000", 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041121639999999998 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.628920+0000", 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021025800000000001 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.628933+0000", 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3227999999999999e-05 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.629134+0000", 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200936 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630166+0000", 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010321149999999999 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630428+0000", 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026151100000000002 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.686870+0000", 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056442880000000001 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.756621+0000", 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069750915999999996 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.756758+0000", 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013627600000000001 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.756809+0000", 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1110999999999999e-05 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:128 3.c 3:316b7ecd:::10000000002.0000007d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.628963+0000", 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.33570317900000002, 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17669859600000001, 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 128 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.628963+0000", 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.628963+0000", 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.628966+0000", 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8820000000000001e-06 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.632660+0000", 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036938280000000001 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.632679+0000", 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8652e-05 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.632693+0000", 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.431e-05 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.633047+0000", 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035366999999999999 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.633808+0000", 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076160400000000004 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.634028+0000", 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021956999999999999 2024-09-19T05:43:54.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.781355+0000", 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14732689400000001 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805554+0000", 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024199847999999999 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805609+0000", 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4551e-05 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805662+0000", 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2787000000000001e-05 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:144 3.34 3:2feeb5a8:::10000000002.0000008d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.649787+0000", 2024-09-19T05:43:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.31487930400000003, 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15600565199999999, 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 144 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.649787+0000", 2024-09-19T05:43:54.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.649787+0000", 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.649790+0000", 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9349999999999999e-06 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.654041+0000", 2024-09-19T05:43:54.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004251276 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.654066+0000", 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4443e-05 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.654099+0000", 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3265000000000001e-05 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.655099+0000", 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010001859999999999 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.656129+0000", 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010301450000000001 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.666951+0000", 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010821595 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.787258+0000", 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12030732299999999 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805711+0000", 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018452941 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805755+0000", 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3588999999999999e-05 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805793+0000", 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7954000000000003e-05 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:153 3.34 3:2cf0bbf2:::10000000002.00000096:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.658834+0000", 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.30583258099999999, 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14707441600000001, 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 153 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.658834+0000", 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.658834+0000", 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.658837+0000", 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.976e-06 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.662774+0000", 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039376849999999998 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.662800+0000", 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5048000000000001e-05 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.662819+0000", 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9463e-05 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.761237+0000", 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.098418311999999994 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.761469+0000", 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000232196 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.761685+0000", 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021517899999999999 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.787272+0000", 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025587111999999999 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805829+0000", 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018557395000000001 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805874+0000", 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4811000000000002e-05 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805908+0000", 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4239000000000002e-05 2024-09-19T05:43:54.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:148 3.1c e53/38 3:3afba7d1:::10000000002.00000091:head v 53'5, mlcod=0'0)", 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.663316+0000", 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.30135033500000002, 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.159617179, 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.058 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.663316+0000", 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.663316+0000", 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.663318+0000", 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.474e-06 2024-09-19T05:43:54.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.695982+0000", 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032664408999999998 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.696000+0000", 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.823e-05 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.696014+0000", 2024-09-19T05:43:54.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3584000000000001e-05 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.760705+0000", 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.064690987000000005 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.760726+0000", 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1018000000000001e-05 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.822892+0000", 2024-09-19T05:43:54.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.062165848000000003 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.822933+0000", 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1628999999999997e-05 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:165 3.2c 3:3723e56f:::10000000002.000000a2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.666079+0000", 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.29858763900000002, 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16744595000000001, 2024-09-19T05:43:54.062 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 165 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.666079+0000", 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.666079+0000", 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.666081+0000", 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.52e-06 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687582+0000", 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021500819000000001 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687599+0000", 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7397e-05 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687611+0000", 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1258e-05 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.805946+0000", 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11833521399999999 2024-09-19T05:43:54.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.806096+0000", 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015049799999999999 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.806281+0000", 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000184835 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820633+0000", 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014351505000000001 2024-09-19T05:43:54.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.833360+0000", 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012727135000000001 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.833443+0000", 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3255000000000002e-05 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.833525+0000", 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1514000000000006e-05 2024-09-19T05:43:54.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:171 3.3b 3:ddecd1ef:::10000000002.000000a8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.687642+0000", 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.27702436400000002, 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.182415208, 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 171 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687642+0000", 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687642+0000", 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687644+0000", 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3970000000000001e-06 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.691939+0000", 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042946900000000003 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.698227+0000", 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006288086 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.699907+0000", 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016802410000000001 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.705341+0000", 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054335210000000002 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.705672+0000", 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033090499999999999 2024-09-19T05:43:54.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.705948+0000", 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027583600000000001 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.745465+0000", 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039517406999999997 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.869948+0000", 2024-09-19T05:43:54.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.124482989 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.870002+0000", 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4274000000000002e-05 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.870057+0000", 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4861999999999998e-05 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:209 3.3b 3:dfa9a0fc:::10000000002.000000ce:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.751195+0000", 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21347095599999999, 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13942587100000001, 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 209 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.751195+0000", 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.751195+0000", 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.751201+0000", 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5809999999999996e-06 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.757085+0000", 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058844359999999998 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.758851+0000", 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001765595 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.761729+0000", 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028779270000000002 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.769797+0000", 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080685240000000005 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.769969+0000", 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017170600000000001 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.770125+0000", 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015584299999999999 2024-09-19T05:43:54.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.794061+0000", 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023935807 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.890496+0000", 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.096435207999999994 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.890563+0000", 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6749999999999996e-05 2024-09-19T05:43:54.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.890621+0000", 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8493999999999999e-05 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.079 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:54.080 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:54.081 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:54.084 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-19T05:43:54.085 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:54.086 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:54.079+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:54.181 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:54.180+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.226+0000 7fd280c65640 1 Processor -- start 2024-09-19T05:43:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.226+0000 7fd280c65640 1 -- start start 2024-09-19T05:43:54.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd280c65640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd27c0caf40 0x7fd27c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd280c65640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c0ccbc0 con 0x7fd27c154c20 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd280c65640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c0cc600 con 0x7fd27c155000 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd280c65640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd27c0cc8e0 con 0x7fd27c0caf40 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27a59c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd27c154c20 0x7fd27c073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44994/0 (socket says 172.21.15.2:44994) 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27a59c640 1 -- 172.21.15.2:0/1993615273 learned_addr learned my addr 172.21.15.2:0/1993615273 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27ad9d640 1 --2- 172.21.15.2:0/1993615273 >> v2:172.21.15.123:3300/0 conn(0x7fd27c0caf40 0x7fd27c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27959a640 1 -- 172.21.15.2:0/1993615273 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256455543 0 0) 0x7fd27c0ccbc0 con 0x7fd27c154c20 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27959a640 1 -- 172.21.15.2:0/1993615273 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd264003520 con 0x7fd27c154c20 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27959a640 1 -- 172.21.15.2:0/1993615273 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3795509468 0 0) 0x7fd27c0cc600 con 0x7fd27c155000 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27959a640 1 -- 172.21.15.2:0/1993615273 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd27c0ccbc0 con 0x7fd27c155000 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27ad9d640 1 -- 172.21.15.2:0/1993615273 >> v1:172.21.15.123:6789/0 conn(0x7fd27c155000 legacy=0x7fd27c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27ad9d640 1 -- 172.21.15.2:0/1993615273 >> v1:172.21.15.2:6789/0 conn(0x7fd27c154c20 legacy=0x7fd27c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.227+0000 7fd27ad9d640 1 -- 172.21.15.2:0/1993615273 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c0cbc60 con 0x7fd27c0caf40 2024-09-19T05:43:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.228+0000 7fd27ad9d640 1 --2- 172.21.15.2:0/1993615273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 0x7fd27c0cd9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd274009d90 tx=0x7fd27402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7300f5e73c1e49a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.228+0000 7fd27959a640 1 -- 172.21.15.2:0/1993615273 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274007740 con 0x7fd27c0caf40 2024-09-19T05:43:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.228+0000 7fd27959a640 1 -- 172.21.15.2:0/1993615273 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2740078e0 con 0x7fd27c0caf40 2024-09-19T05:43:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.228+0000 7fd27959a640 1 -- 172.21.15.2:0/1993615273 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274007cb0 con 0x7fd27c0caf40 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.229+0000 7fd280c65640 1 -- 172.21.15.2:0/1993615273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 msgr2=0x7fd27c0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.229+0000 7fd280c65640 1 --2- 172.21.15.2:0/1993615273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 0x7fd27c0cd9e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd274009d90 tx=0x7fd27402a6f0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.229+0000 7fd280c65640 1 -- 172.21.15.2:0/1993615273 shutdown_connections 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.229+0000 7fd280c65640 1 --2- 172.21.15.2:0/1993615273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 0x7fd27c0cd9e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.229+0000 7fd280c65640 1 -- 172.21.15.2:0/1993615273 >> 172.21.15.2:0/1993615273 conn(0x7fd27c06d0e0 msgr2=0x7fd27c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.229+0000 7fd280c65640 1 -- 172.21.15.2:0/1993615273 shutdown_connections 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.229+0000 7fd280c65640 1 -- 172.21.15.2:0/1993615273 wait complete. 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.230+0000 7fd280c65640 1 Processor -- start 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.230+0000 7fd280c65640 1 -- start start 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd280c65640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 0x7fd27c146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd280c65640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd27c154c20 0x7fd27c0b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd280c65640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c155000 0x7fd27c1e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd280c65640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c073a50 con 0x7fd27c155000 2024-09-19T05:43:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd280c65640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c0cb430 con 0x7fd27c0caf40 2024-09-19T05:43:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd280c65640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd27c073bd0 con 0x7fd27c154c20 2024-09-19T05:43:54.235 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:54.235 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:54.235 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:54.235 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd27ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c155000 0x7fd27c1e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd27ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c155000 0x7fd27c1e7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39058/0 (socket says 172.21.15.2:39058) 2024-09-19T05:43:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd27ad9d640 1 -- 172.21.15.2:0/904709445 learned_addr learned my addr 172.21.15.2:0/904709445 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd279d9b640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd27c154c20 0x7fd27c0b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.238 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.231+0000 7fd27a59c640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 0x7fd27c146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd27ad9d640 1 -- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd27c154c20 msgr2=0x7fd27c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd27ad9d640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd27c154c20 0x7fd27c0b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd27ad9d640 1 -- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 msgr2=0x7fd27c146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd27ad9d640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd27c0caf40 0x7fd27c146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd27ad9d640 1 -- 172.21.15.2:0/904709445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c147950 con 0x7fd27c155000 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd27ad9d640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd27c155000 0x7fd27c1e7ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd274009eb0 tx=0x7fd274004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274040070 con 0x7fd27c155000 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274031bc0 con 0x7fd27c155000 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd280c65640 1 -- 172.21.15.2:0/904709445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c147c30 con 0x7fd27c155000 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274009450 con 0x7fd27c155000 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.232+0000 7fd280c65640 1 -- 172.21.15.2:0/904709445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c146440 con 0x7fd27c155000 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.233+0000 7fd280c65640 1 -- 172.21.15.2:0/904709445 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd27c147f10 con 0x7fd27c155000 2024-09-19T05:43:54.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.234+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd274039040 con 0x7fd27c155000 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.235+0000 7fd26affd640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd25c06c2b0 0x7fd25c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.235+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7fd2740ad310 con 0x7fd27c155000 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.235+0000 7fd26affd640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd25c0744b0 0x7fd25c0768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.235+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2740020b0 con 0x7fd25c0744b0 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.235+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd2740ad7a0 con 0x7fd27c155000 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.235+0000 7fd27a59c640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd25c06c2b0 0x7fd25c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.235+0000 7fd279d9b640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd25c0744b0 0x7fd25c0768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.236+0000 7fd27a59c640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd25c06c2b0 0x7fd25c06e770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd26c000f60 tx=0x7fd26c007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.236+0000 7fd279d9b640 1 --2- 172.21.15.2:0/904709445 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd25c0744b0 0x7fd25c0768d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.236+0000 7fd26affd640 1 -- 172.21.15.2:0/904709445 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2740020b0 con 0x7fd25c0744b0 2024-09-19T05:43:54.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:54.264 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:54.265 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:54.266 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:54.267 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:54.267 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-19T05:43:54.282 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:54.281+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:54.352 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:54.352 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:54.352 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:54.353 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:54.353 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:54.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:54.402 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:54.400+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.409+0000 7f715e303640 1 Processor -- start 2024-09-19T05:43:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.409+0000 7f715e303640 1 -- start start 2024-09-19T05:43:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.410+0000 7f715e303640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c6ec0 0x7f7158158840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.410+0000 7f715e303640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715815cd90 con 0x7f71580c7f40 2024-09-19T05:43:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.410+0000 7f715e303640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71580c86d0 con 0x7f71580c72a0 2024-09-19T05:43:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.410+0000 7f715e303640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f71580c8410 con 0x7f71580c6ec0 2024-09-19T05:43:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.410+0000 7f714f7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f71580c7f40 0x7f7158152410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44998/0 (socket says 172.21.15.2:44998) 2024-09-19T05:43:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.410+0000 7f714f7fe640 1 -- 172.21.15.2:0/330253593 learned_addr learned my addr 172.21.15.2:0/330253593 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.410+0000 7f7157fff640 1 --2- 172.21.15.2:0/330253593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c6ec0 0x7f7158158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.411+0000 7f71577fe640 1 -- 172.21.15.2:0/330253593 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963078530 0 0) 0x7f715815cd90 con 0x7f71580c7f40 2024-09-19T05:43:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.411+0000 7f71577fe640 1 -- 172.21.15.2:0/330253593 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7144003520 con 0x7f71580c7f40 2024-09-19T05:43:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.411+0000 7f71577fe640 1 -- 172.21.15.2:0/330253593 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1718248359 0 0) 0x7f71580c86d0 con 0x7f71580c72a0 2024-09-19T05:43:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.411+0000 7f7157fff640 1 -- 172.21.15.2:0/330253593 >> v1:172.21.15.123:6789/0 conn(0x7f71580c72a0 legacy=0x7f71581631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.411+0000 7f7157fff640 1 -- 172.21.15.2:0/330253593 >> v1:172.21.15.2:6789/0 conn(0x7f71580c7f40 legacy=0x7f7158152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.411+0000 7f7157fff640 1 -- 172.21.15.2:0/330253593 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715815bfd0 con 0x7f71580c6ec0 2024-09-19T05:43:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f7157fff640 1 --2- 172.21.15.2:0/330253593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c6ec0 0x7f7158158840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71480097d0 tx=0x7f714802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26eca5add93057e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f71577fe640 1 -- 172.21.15.2:0/330253593 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148037070 con 0x7f71580c6ec0 2024-09-19T05:43:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f71577fe640 1 -- 172.21.15.2:0/330253593 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7148032440 con 0x7f71580c6ec0 2024-09-19T05:43:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f71577fe640 1 -- 172.21.15.2:0/330253593 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71480327f0 con 0x7f71580c6ec0 2024-09-19T05:43:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f715e303640 1 -- 172.21.15.2:0/330253593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c6ec0 msgr2=0x7f7158158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f715e303640 1 --2- 172.21.15.2:0/330253593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c6ec0 0x7f7158158840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71480097d0 tx=0x7f714802a290 comp rx=0 tx=0).stop 2024-09-19T05:43:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f715e303640 1 -- 172.21.15.2:0/330253593 shutdown_connections 2024-09-19T05:43:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f715e303640 1 --2- 172.21.15.2:0/330253593 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c6ec0 0x7f7158158840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.412+0000 7f715e303640 1 -- 172.21.15.2:0/330253593 >> 172.21.15.2:0/330253593 conn(0x7f715806cd20 msgr2=0x7f715806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.413+0000 7f715e303640 1 -- 172.21.15.2:0/330253593 shutdown_connections 2024-09-19T05:43:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.413+0000 7f715e303640 1 -- 172.21.15.2:0/330253593 wait complete. 2024-09-19T05:43:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.414+0000 7f715e303640 1 Processor -- start 2024-09-19T05:43:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.414+0000 7f715e303640 1 -- start start 2024-09-19T05:43:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f715e303640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71580c6ec0 0x7f7158158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f715e303640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71580c72a0 0x7f7158159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f715e303640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c7f40 0x7f71581577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f715e303640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f715815c870 con 0x7f71580c72a0 2024-09-19T05:43:54.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f715e303640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f715815c590 con 0x7f71580c6ec0 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f715e303640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7158071500 con 0x7f71580c7f40 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71580c72a0 0x7f7158159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71580c72a0 0x7f7158159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39060/0 (socket says 172.21.15.2:39060) 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 -- 172.21.15.2:0/3794186482 learned_addr learned my addr 172.21.15.2:0/3794186482 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f7157fff640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71580c6ec0 0x7f7158158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f715c8a0640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c7f40 0x7f71581577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 -- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c7f40 msgr2=0x7f71581577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71580c7f40 0x7f71581577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 -- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71580c6ec0 msgr2=0x7f7158158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71580c6ec0 0x7f7158158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 -- 172.21.15.2:0/3794186482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7158159750 con 0x7f71580c72a0 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f714f7fe640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71580c72a0 0x7f7158159050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f714000c390 tx=0x7f714000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140017070 con 0x7f71580c72a0 2024-09-19T05:43:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.415+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7140005ce0 con 0x7f71580c72a0 2024-09-19T05:43:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.416+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71400072f0 con 0x7f71580c72a0 2024-09-19T05:43:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.416+0000 7f715e303640 1 -- 172.21.15.2:0/3794186482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71581ee970 con 0x7f71580c72a0 2024-09-19T05:43:54.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.416+0000 7f715e303640 1 -- 172.21.15.2:0/3794186482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7158156900 con 0x7f71580c72a0 2024-09-19T05:43:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.417+0000 7f715e303640 1 -- 172.21.15.2:0/3794186482 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f715806c960 con 0x7f71580c72a0 2024-09-19T05:43:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.417+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f714001a070 con 0x7f71580c72a0 2024-09-19T05:43:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f71557fa640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f713806c2b0 0x7f713806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7f714008a7f0 con 0x7f71580c72a0 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f71557fa640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7138074450 0x7f7138076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f7157fff640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f713806c2b0 0x7f713806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7140015e70 con 0x7f7138074450 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f714008f2c0 con 0x7f71580c72a0 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f715c8a0640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7138074450 0x7f7138076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f7157fff640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f713806c2b0 0x7f713806e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7148034000 tx=0x7f714802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.418+0000 7f715c8a0640 1 --2- 172.21.15.2:0/3794186482 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7138074450 0x7f7138076870 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.419+0000 7f71557fa640 1 -- 172.21.15.2:0/3794186482 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7140015e70 con 0x7f7138074450 2024-09-19T05:43:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:54.446 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:54.447 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:54.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:54.449 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:54.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-19T05:43:54.493 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:54.489+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:54.513 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:54.513 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:54.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:54.513 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 91.631524569999996, 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.514 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:54.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:54.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:54.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:7 3.2a 3:557f550e:::10000000002.00000004:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:43:54.520 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.2280762649999999, 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27494626900000002, 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 7 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:43:54.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282495+0000", 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4609999999999998e-06 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286920+0000", 2024-09-19T05:43:54.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044248229999999996 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286972+0000", 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1659000000000001e-05 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286986+0000", 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287117+0000", 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131328 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287358+0000", 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024141700000000001 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287543+0000", 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018436700000000001 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299025+0000", 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011482327000000001 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557305+0000", 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25828001099999998 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557360+0000", 2024-09-19T05:43:54.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5074000000000002e-05 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557439+0000", 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9018999999999997e-05 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:8 3.1c 3:3ae0713d:::10000000002.00000005:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.296068+0000", 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.2145015180000001, 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26154461800000001, 2024-09-19T05:43:54.526 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.296068+0000", 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.296068+0000", 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.296072+0000", 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3229999999999999e-06 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301751+0000", 2024-09-19T05:43:54.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056786409999999999 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301774+0000", 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3475000000000001e-05 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301792+0000", 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301884+0000", 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2305999999999993e-05 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302046+0000", 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016148399999999999 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302196+0000", 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000149746 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.334012+0000", 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031816143999999998 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557498+0000", 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22348642599999999 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557572+0000", 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3670000000000004e-05 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557612+0000", 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0459e-05 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:43 3.1a 3:5917aad8:::10000000002.00000028:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.337239+0000", 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.173329734, 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.064239559000000002, 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 43 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337239+0000", 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337239+0000", 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337242+0000", 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.559e-06 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.350884+0000", 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013642101 2024-09-19T05:43:54.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.350918+0000", 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3729000000000002e-05 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.350933+0000", 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5072000000000001e-05 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.351549+0000", 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061607800000000005 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.351723+0000", 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017419399999999999 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.351889+0000", 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016625 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.358475+0000", 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065857149999999998 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.401355+0000", 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042879408000000001 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.401422+0000", 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7373999999999996e-05 2024-09-19T05:43:54.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.401479+0000", 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7079e-05 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:45 3.37 3:ec154225:::10000000002.0000002a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.356538+0000", 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.1540315139999999, 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093610510999999993, 2024-09-19T05:43:54.538 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 45 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.356538+0000", 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.356538+0000", 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.356541+0000", 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1259999999999998e-06 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369575+0000", 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013033805000000001 2024-09-19T05:43:54.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369594+0000", 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9247000000000001e-05 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369606+0000", 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2062000000000001e-05 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369683+0000", 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7136000000000001e-05 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369835+0000", 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015142700000000001 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369987+0000", 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000152495 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.399199+0000", 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029212444000000001 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.450027+0000", 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050827259 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.450088+0000", 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1193999999999997e-05 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.450148+0000", 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0316000000000003e-05 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:47 3.37 3:ec3fc134:::10000000002.0000002c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.369644+0000", 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.1409252139999999, 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15125028900000001, 2024-09-19T05:43:54.544 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 47 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369644+0000", 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369644+0000", 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369647+0000", 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8789999999999999e-06 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.371739+0000", 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020923449999999998 2024-09-19T05:43:54.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.371764+0000", 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4794999999999999e-05 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.371778+0000", 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3712e-05 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.377906+0000", 2024-09-19T05:43:54.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006128451 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.378113+0000", 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000206744 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.378281+0000", 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000168572 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.404124+0000", 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025842774999999998 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.520779+0000", 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.116654297 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.520841+0000", 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2343999999999998e-05 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.520894+0000", 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3375000000000003e-05 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:59 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.402045+0000", 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.1085242580000001, 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20849380200000001, 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.402045+0000", 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.402045+0000", 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.402048+0000", 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8550000000000002e-06 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406834+0000", 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047862850000000004 2024-09-19T05:43:54.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406855+0000", 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0975000000000002e-05 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406868+0000", 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2842e-05 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406972+0000", 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104435 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.407208+0000", 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023539700000000001 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.407400+0000", 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019240899999999999 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.414059+0000", 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066590160000000002 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.610422+0000", 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19636292 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.610484+0000", 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1494000000000004e-05 2024-09-19T05:43:54.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.610539+0000", 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5173999999999997e-05 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:91 3.2a 3:5747858d:::10000000002.00000058:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.411935+0000", 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0986346300000001, 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21859863900000001, 2024-09-19T05:43:54.556 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 91 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.411935+0000", 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.411935+0000", 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.411937+0000", 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415027+0000", 2024-09-19T05:43:54.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030898309999999999 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415290+0000", 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026321599999999998 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415302+0000", 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1488000000000001e-05 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415368+0000", 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6047e-05 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415532+0000", 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016381100000000001 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415790+0000", 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000258452 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489636+0000", 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073845375000000005 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630388+0000", 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.140752496 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630468+0000", 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9944999999999994e-05 2024-09-19T05:43:54.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630533+0000", 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5194e-05 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:92 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0949282469999999, 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37051118500000002, 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415644+0000", 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.422983+0000", 2024-09-19T05:43:54.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073390779999999997 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423020+0000", 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7660000000000002e-05 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423036+0000", 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5359999999999999e-05 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423261+0000", 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225217 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423458+0000", 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196555 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423737+0000", 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027922100000000002 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.435447+0000", 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011709855999999999 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.785961+0000", 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35051446400000003 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786086+0000", 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012537599999999999 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786152+0000", 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691000000000005e-05 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:97 3.1a 3:5b190c62:::10000000002.0000005e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.427800+0000", 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0827694649999999, 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.062014485000000001, 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 97 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.427800+0000", 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.427800+0000", 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.427803+0000", 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4130000000000002e-06 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429781+0000", 2024-09-19T05:43:54.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019773500000000001 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429801+0000", 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429817+0000", 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5299e-05 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.431876+0000", 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00205879 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.432072+0000", 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196851 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.432273+0000", 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020102600000000001 2024-09-19T05:43:54.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.454877+0000", 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022603068 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489656+0000", 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034779657999999998 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489736+0000", 2024-09-19T05:43:54.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9587999999999998e-05 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489814+0000", 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8474999999999997e-05 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:104 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0807019250000001, 2024-09-19T05:43:54.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28556600799999998, 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429870+0000", 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-06 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446619+0000", 2024-09-19T05:43:54.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016749447000000001 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446650+0000", 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0218000000000001e-05 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446666+0000", 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6920999999999999e-05 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448857+0000", 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021902699999999998 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449011+0000", 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015441700000000001 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449168+0000", 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156527 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489828+0000", 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040659908000000002 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715285+0000", 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22545759500000001 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715345+0000", 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0269999999999997e-05 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715433+0000", 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7814999999999999e-05 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:12 3.2c e53/38 3:354fa1eb:::10000000002.00000009:head v 53'1, mlcod=0'0)", 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0799194729999999, 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.124694371, 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1199999999999999e-07 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541667+0000", 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.111017229 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541684+0000", 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7183000000000001e-05 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541697+0000", 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2177999999999999e-05 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541807+0000", 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110747 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541835+0000", 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7338000000000002e-05 2024-09-19T05:43:54.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.554544+0000", 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012709573 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555344+0000", 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079991100000000005 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:108 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.063850908, 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26890963299999998, 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.585 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446721+0000", 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8250000000000001e-06 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451943+0000", 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052222140000000002 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451961+0000", 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8117e-05 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451970+0000", 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7420000000000003e-06 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473034+0000", 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021064282 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473242+0000", 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020716699999999999 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473446+0000", 2024-09-19T05:43:54.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204396 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489832+0000", 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016385994000000001 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715506+0000", 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22567427700000001 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715577+0000", 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0548000000000003e-05 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715628+0000", 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070999999999999e-05 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:139 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:43:54.591 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0470395020000001, 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.328123891, 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:43:54.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463532+0000", 2024-09-19T05:43:54.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0339999999999999e-06 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468331+0000", 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047995099999999999 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468365+0000", 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3667000000000001e-05 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468382+0000", 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6651e-05 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468915+0000", 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053293299999999997 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469199+0000", 2024-09-19T05:43:54.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028406900000000002 2024-09-19T05:43:54.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469482+0000", 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028390000000000002 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.485026+0000", 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015543509 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791538+0000", 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30651197400000002 2024-09-19T05:43:54.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791602+0000", 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3780000000000003e-05 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791654+0000", 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1863999999999997e-05 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:148 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0285894680000001, 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34695000399999998, 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 148 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.599 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481982+0000", 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6309999999999999e-06 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489765+0000", 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077821310000000003 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489778+0000", 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3604999999999999e-05 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489787+0000", 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489875+0000", 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8039000000000002e-05 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490101+0000", 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022636299999999999 2024-09-19T05:43:54.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490426+0000", 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032453799999999997 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.506325+0000", 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015899149000000001 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828797+0000", 2024-09-19T05:43:54.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32247239900000002 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828865+0000", 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7521999999999995e-05 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828930+0000", 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4795999999999994e-05 2024-09-19T05:43:54.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:156 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0207342610000001, 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26935146300000001, 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:43:54.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489837+0000", 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2689999999999998e-06 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495152+0000", 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053150209999999996 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495356+0000", 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203537 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495373+0000", 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:43:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496071+0000", 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069810200000000003 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496246+0000", 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017547299999999999 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496469+0000", 2024-09-19T05:43:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022283200000000001 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.540840+0000", 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044371318999999999 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759082+0000", 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21824185500000001 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759137+0000", 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645000000000003e-05 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759186+0000", 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9499000000000002e-05 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:185 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0080544170000001, 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32688410400000001, 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 185 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502518+0000", 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.509973+0000", 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074553919999999999 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510031+0000", 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8356999999999997e-05 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510046+0000", 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932e-05 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510135+0000", 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8548000000000007e-05 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510886+0000", 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075159700000000001 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511079+0000", 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019261500000000001 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532794+0000", 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021714989000000001 2024-09-19T05:43:54.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829312+0000", 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29651747699999997 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829360+0000", 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8396000000000002e-05 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829399+0000", 2024-09-19T05:43:54.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8961999999999997e-05 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:198 3.37 3:ec162ecd:::10000000002.000000c3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.518231+0000", 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.99233801499999996, 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063948191000000001, 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 198 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518231+0000", 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518231+0000", 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518235+0000", 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.693e-06 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531247+0000", 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013012271000000001 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531283+0000", 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5550999999999999e-05 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531299+0000", 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6178999999999999e-05 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531471+0000", 2024-09-19T05:43:54.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017213499999999999 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532503+0000", 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010324290000000001 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532706+0000", 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020218699999999999 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:54.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.554522+0000", 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021816397000000001 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.582075+0000", 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027552706999999999 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.582127+0000", 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.202e-05 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.582179+0000", 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2621999999999999e-05 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:257 3.21 e53/38 3:86aaefcc:::10000000002.000000fe:head v 53'5, mlcod=53'3)", 2024-09-19T05:43:54.624 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.522406+0000", 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.98816354299999998, 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.088968622999999997, 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522406+0000", 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522406+0000", 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522408+0000", 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9470000000000002e-06 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570461+0000", 2024-09-19T05:43:54.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048053235999999999 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570477+0000", 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.658e-05 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570489+0000", 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1989e-05 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570569+0000", 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9709999999999994e-05 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570636+0000", 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6829999999999995e-05 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.611243+0000", 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040607074999999999 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.611374+0000", 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013125600000000001 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:119 3.f e53/38 3:f3777bdc:::10000000002.00000074:head v 53'3, mlcod=0'0)", 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.641756+0000", 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86881288199999995, 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.089892116999999994, 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:54.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.641756+0000", 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.641756+0000", 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:54.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.641758+0000", 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5340000000000001e-06 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715078+0000", 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073320084999999993 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715103+0000", 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4929000000000001e-05 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:54.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715118+0000", 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4707e-05 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715319+0000", 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200965 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:54.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715347+0000", 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8847999999999999e-05 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.731604+0000", 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016256900000000001 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:54.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.731648+0000", 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4149e-05 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:54.634 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:54.636 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:54.641 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:54.634+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:54.642 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:43:54.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.596+0000 7fa70ff2c640 1 Processor -- start 2024-09-19T05:43:54.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.596+0000 7fa70ff2c640 1 -- start start 2024-09-19T05:43:54.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70ff2c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cba20 0x7fa7081549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.642 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70ff2c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa70815ab40 con 0x7fa7080cb640 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70ff2c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa70815ade0 con 0x7fa70815ede0 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70ff2c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa7080c5ef0 con 0x7fa7080cba20 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70dcc8640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa7080cb640 0x7fa7080c54c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45014/0 (socket says 172.21.15.2:45014) 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70dcc8640 1 -- 172.21.15.2:0/1220578144 learned_addr learned my addr 172.21.15.2:0/1220578144 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70d4c7640 1 --2- 172.21.15.2:0/1220578144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cba20 0x7fa7081549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70ccc6640 1 -- 172.21.15.2:0/1220578144 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4224178885 0 0) 0x7fa70815ab40 con 0x7fa7080cb640 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.597+0000 7fa70ccc6640 1 -- 172.21.15.2:0/1220578144 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6f0003520 con 0x7fa7080cb640 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.598+0000 7fa70ccc6640 1 -- 172.21.15.2:0/1220578144 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262075807 0 0) 0x7fa70815ade0 con 0x7fa70815ede0 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.598+0000 7fa70ccc6640 1 -- 172.21.15.2:0/1220578144 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa70815ab40 con 0x7fa70815ede0 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.598+0000 7fa70d4c7640 1 -- 172.21.15.2:0/1220578144 >> v1:172.21.15.123:6789/0 conn(0x7fa70815ede0 legacy=0x7fa7080c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.598+0000 7fa70d4c7640 1 -- 172.21.15.2:0/1220578144 >> v1:172.21.15.2:6789/0 conn(0x7fa7080cb640 legacy=0x7fa7080c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.598+0000 7fa70d4c7640 1 -- 172.21.15.2:0/1220578144 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70815b0c0 con 0x7fa7080cba20 2024-09-19T05:43:54.643 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.598+0000 7fa70d4c7640 1 --2- 172.21.15.2:0/1220578144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cba20 0x7fa7081549e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0097d0 tx=0x7fa6fc02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6a54645feea102 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ccc6640 1 -- 172.21.15.2:0/1220578144 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc037070 con 0x7fa7080cba20 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ccc6640 1 -- 172.21.15.2:0/1220578144 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6fc032440 con 0x7fa7080cba20 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ccc6640 1 -- 172.21.15.2:0/1220578144 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0327f0 con 0x7fa7080cba20 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ff2c640 1 -- 172.21.15.2:0/1220578144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cba20 msgr2=0x7fa7081549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ff2c640 1 --2- 172.21.15.2:0/1220578144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cba20 0x7fa7081549e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0097d0 tx=0x7fa6fc02a290 comp rx=0 tx=0).stop 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ff2c640 1 -- 172.21.15.2:0/1220578144 shutdown_connections 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ff2c640 1 --2- 172.21.15.2:0/1220578144 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cba20 0x7fa7081549e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ff2c640 1 -- 172.21.15.2:0/1220578144 >> 172.21.15.2:0/1220578144 conn(0x7fa708067950 msgr2=0x7fa708067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ff2c640 1 -- 172.21.15.2:0/1220578144 shutdown_connections 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.599+0000 7fa70ff2c640 1 -- 172.21.15.2:0/1220578144 wait complete. 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.600+0000 7fa70ff2c640 1 Processor -- start 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.600+0000 7fa70ff2c640 1 -- start start 2024-09-19T05:43:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70ff2c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cb640 0x7fa7080c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70ff2c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7080cba20 0x7fa7080cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70ff2c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa70815ede0 0x7fa7080c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70ff2c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa70815b960 con 0x7fa7080cba20 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70ff2c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa70815b680 con 0x7fa70815ede0 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70ff2c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa7080c5bd0 con 0x7fa7080cb640 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70d4c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7080cba20 0x7fa7080cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70d4c7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7080cba20 0x7fa7080cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39074/0 (socket says 172.21.15.2:39074) 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70d4c7640 1 -- 172.21.15.2:0/392691343 learned_addr learned my addr 172.21.15.2:0/392691343 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70dcc8640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cb640 0x7fa7080c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.601+0000 7fa70e4c9640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa70815ede0 0x7fa7080c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70d4c7640 1 -- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cb640 msgr2=0x7fa7080c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70d4c7640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7080cb640 0x7fa7080c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70d4c7640 1 -- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa70815ede0 msgr2=0x7fa7080c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70d4c7640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa70815ede0 0x7fa7080c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70d4c7640 1 -- 172.21.15.2:0/392691343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70814ffb0 con 0x7fa7080cba20 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70d4c7640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7080cba20 0x7fa7080cfcc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc035040 tx=0x7fa6fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc041020 con 0x7fa7080cba20 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6fc03d720 con 0x7fa7080cba20 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70ff2c640 1 -- 172.21.15.2:0/392691343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7080c8890 con 0x7fa7080cba20 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc037040 con 0x7fa7080cba20 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.602+0000 7fa70ff2c640 1 -- 172.21.15.2:0/392691343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa708150250 con 0x7fa7080cba20 2024-09-19T05:43:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.603+0000 7fa70ff2c640 1 -- 172.21.15.2:0/392691343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7080c54c0 con 0x7fa7080cba20 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.603+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa6fc044020 con 0x7fa7080cba20 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.604+0000 7fa7067fc640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa6f406c2b0 0x7fa6f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.604+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7fa6fc0aba00 con 0x7fa7080cba20 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.604+0000 7fa70dcc8640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa6f406c2b0 0x7fa6f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.604+0000 7fa7067fc640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa6f40743c0 0x7fa6f40767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.604+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa6fc001d30 con 0x7fa6f40743c0 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.604+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa6fc0ade60 con 0x7fa7080cba20 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.604+0000 7fa70e4c9640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa6f40743c0 0x7fa6f40767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.605+0000 7fa70dcc8640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa6f406c2b0 0x7fa6f406e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80012e0 tx=0x7fa6f8001ad0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.605+0000 7fa70e4c9640 1 --2- 172.21.15.2:0/392691343 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fa6f40743c0 0x7fa6f40767e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.605+0000 7fa7067fc640 1 -- 172.21.15.2:0/392691343 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa6fc001d30 con 0x7fa6f40743c0 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches: 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:0 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:1 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:abort 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:assert 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:bench [] [] [] [] 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs debug_inject_read_zeros 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs files list 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:bluefs stats 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:bluestore allocator dump block 2024-09-19T05:43:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command 2024-09-19T05:43:54.649 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-19T05:43:54.671 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:43:54.737 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:54.735+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:54.787 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:54.787 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:54.787 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:54.787 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:54.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.787+0000 7f398a168640 1 Processor -- start 2024-09-19T05:43:54.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.787+0000 7f398a168640 1 -- start start 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.788+0000 7f398a168640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3984071110 0x7f398406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.788+0000 7f398a168640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39840706c0 con 0x7f39840b0ee0 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.788+0000 7f398a168640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398415b7d0 con 0x7f398406ea80 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.788+0000 7f398a168640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f398406e6d0 con 0x7f3984071110 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.788+0000 7f39837fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f39840b0ee0 0x7f39840ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45024/0 (socket says 172.21.15.2:45024) 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.788+0000 7f39837fe640 1 -- 172.21.15.2:0/2902955489 learned_addr learned my addr 172.21.15.2:0/2902955489 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.788+0000 7f3983fff640 1 --2- 172.21.15.2:0/2902955489 >> v2:172.21.15.2:3300/0 conn(0x7f3984071110 0x7f398406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.789+0000 7f3982ffd640 1 -- 172.21.15.2:0/2902955489 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138122205 0 0) 0x7f39840706c0 con 0x7f39840b0ee0 2024-09-19T05:43:54.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.789+0000 7f3982ffd640 1 -- 172.21.15.2:0/2902955489 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3970003520 con 0x7f39840b0ee0 2024-09-19T05:43:54.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.789+0000 7f3982ffd640 1 -- 172.21.15.2:0/2902955489 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242541373 0 0) 0x7f398415b7d0 con 0x7f398406ea80 2024-09-19T05:43:54.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.789+0000 7f3982ffd640 1 -- 172.21.15.2:0/2902955489 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39840706c0 con 0x7f398406ea80 2024-09-19T05:43:54.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.789+0000 7f3983fff640 1 -- 172.21.15.2:0/2902955489 >> v1:172.21.15.123:6789/0 conn(0x7f398406ea80 legacy=0x7f3984163440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.789+0000 7f3983fff640 1 -- 172.21.15.2:0/2902955489 >> v1:172.21.15.2:6789/0 conn(0x7f39840b0ee0 legacy=0x7f39840ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.789+0000 7f3983fff640 1 -- 172.21.15.2:0/2902955489 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3984070960 con 0x7f3984071110 2024-09-19T05:43:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f3983fff640 1 --2- 172.21.15.2:0/2902955489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3984071110 0x7f398406cd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f397c009d90 tx=0x7f397c02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e4b84cde6e8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f3982ffd640 1 -- 172.21.15.2:0/2902955489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c037070 con 0x7f3984071110 2024-09-19T05:43:54.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f3982ffd640 1 -- 172.21.15.2:0/2902955489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c007740 con 0x7f3984071110 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f3982ffd640 1 -- 172.21.15.2:0/2902955489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c007b10 con 0x7f3984071110 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f398a168640 1 -- 172.21.15.2:0/2902955489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3984071110 msgr2=0x7f398406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f398a168640 1 --2- 172.21.15.2:0/2902955489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3984071110 0x7f398406cd20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f397c009d90 tx=0x7f397c02a780 comp rx=0 tx=0).stop 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f398a168640 1 -- 172.21.15.2:0/2902955489 shutdown_connections 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f398a168640 1 --2- 172.21.15.2:0/2902955489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3984071110 0x7f398406cd20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f398a168640 1 -- 172.21.15.2:0/2902955489 >> 172.21.15.2:0/2902955489 conn(0x7f398406c240 msgr2=0x7f3984067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.790+0000 7f398a168640 1 -- 172.21.15.2:0/2902955489 shutdown_connections 2024-09-19T05:43:54.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.791+0000 7f398a168640 1 -- 172.21.15.2:0/2902955489 wait complete. 2024-09-19T05:43:54.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.792+0000 7f398a168640 1 Processor -- start 2024-09-19T05:43:54.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.793+0000 7f398a168640 1 -- start start 2024-09-19T05:43:54.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f398a168640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398406ea80 0x7f398414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f398a168640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984071110 0x7f3984155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f398a168640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39840b0ee0 0x7f3984157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f398a168640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f398415bab0 con 0x7f39840b0ee0 2024-09-19T05:43:54.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f398a168640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39840703b0 con 0x7f3984071110 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f398a168640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f39840b12c0 con 0x7f398406ea80 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39840b0ee0 0x7f3984157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39840b0ee0 0x7f3984157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39094/0 (socket says 172.21.15.2:39094) 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 -- 172.21.15.2:0/2254972310 learned_addr learned my addr 172.21.15.2:0/2254972310 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f39837fe640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398406ea80 0x7f398414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f397affd640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984071110 0x7f3984155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 -- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398406ea80 msgr2=0x7f398414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398406ea80 0x7f398414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 -- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984071110 msgr2=0x7f3984155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984071110 0x7f3984155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 -- 172.21.15.2:0/2254972310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39840c8530 con 0x7f39840b0ee0 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f39837fe640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398406ea80 0x7f398414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:54.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.794+0000 7f3983fff640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39840b0ee0 0x7f3984157ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f397c009eb0 tx=0x7f397c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.795+0000 7f3980ff9640 1 -- 172.21.15.2:0/2254972310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c042020 con 0x7f39840b0ee0 2024-09-19T05:43:54.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.795+0000 7f3980ff9640 1 -- 172.21.15.2:0/2254972310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f397c032870 con 0x7f39840b0ee0 2024-09-19T05:43:54.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.795+0000 7f3980ff9640 1 -- 172.21.15.2:0/2254972310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c037040 con 0x7f39840b0ee0 2024-09-19T05:43:54.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.795+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39841f0dd0 con 0x7f39840b0ee0 2024-09-19T05:43:54.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.795+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39841f10b0 con 0x7f39840b0ee0 2024-09-19T05:43:54.800 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:54.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.796+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39840c98c0 con 0x7f39840b0ee0 2024-09-19T05:43:54.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.796+0000 7f3980ff9640 1 -- 172.21.15.2:0/2254972310 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f397c045020 con 0x7f39840b0ee0 2024-09-19T05:43:54.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.797+0000 7f3980ff9640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f396406c2b0 0x7f396406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:54.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.797+0000 7f3980ff9640 1 -- 172.21.15.2:0/2254972310 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7f397c0ac320 con 0x7f39840b0ee0 2024-09-19T05:43:54.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.797+0000 7f39837fe640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f396406c2b0 0x7f396406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:54.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.797+0000 7f39837fe640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f396406c2b0 0x7f396406e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3974000f60 tx=0x7f3974007d70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:54.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.801+0000 7f3980ff9640 1 -- 172.21.15.2:0/2254972310 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f397c04c030 con 0x7f39840b0ee0 2024-09-19T05:43:54.845 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:54.843+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:54.911 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:54.911 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:54.911 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:54.911 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:54.911 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:54.920 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:54.938 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:54.937+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:54.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:54.992+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f3984070ef0 con 0x7f39840b0ee0 2024-09-19T05:43:55.040 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:55.038+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:55.070 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:67 3.16 3:6a386151:::10000000002.00000040:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.343583+0000", 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.7242365369999999, 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25356932300000001, 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 67 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343583+0000", 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343583+0000", 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343586+0000", 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.055e-06 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348022+0000", 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044362830000000001 2024-09-19T05:43:55.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348101+0000", 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8364999999999999e-05 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348119+0000", 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419000000000001e-05 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348552+0000", 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00043272500000000002 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348832+0000", 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028048999999999999 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.349057+0000", 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022431799999999999 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.370068+0000", 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021011193000000001 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.597045+0000", 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22697705000000001 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.597098+0000", 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2871999999999998e-05 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.597152+0000", 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4552999999999998e-05 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.077 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:77 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.719663223, 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40221933999999998, 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 77 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348160+0000", 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0459999999999999e-06 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362035+0000", 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013874391999999999 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362063+0000", 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7804e-05 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362076+0000", 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3200000000000001e-05 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362162+0000", 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6469000000000002e-05 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362317+0000", 2024-09-19T05:43:55.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015507100000000001 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362454+0000", 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013636799999999999 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.372327+0000", 2024-09-19T05:43:55.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0098729219999999993 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750183+0000", 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37785643600000002 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750233+0000", 2024-09-19T05:43:55.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9960999999999998e-05 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750376+0000", 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142671 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:87 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:43:55.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.6851982969999999, 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39257614200000002, 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382623+0000", 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2529999999999999e-06 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390766+0000", 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081425790000000005 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390808+0000", 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2311999999999997e-05 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390827+0000", 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-19T05:43:55.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390955+0000", 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012825900000000001 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391148+0000", 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000192673 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391432+0000", 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028362599999999998 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.455303+0000", 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063870983000000006 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775090+0000", 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31978768600000002 2024-09-19T05:43:55.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775146+0000", 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5393000000000003e-05 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775197+0000", 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1699000000000001e-05 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:94 3.5 3:a282a134:::10000000002.0000005b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.426351+0000", 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.6414684660000001, 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951865500000001, 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 94 2024-09-19T05:43:55.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.426351+0000", 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.426351+0000", 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.426355+0000", 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7639999999999999e-06 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429819+0000", 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034642869999999999 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436292+0000", 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064730129999999997 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436316+0000", 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3966000000000001e-05 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436395+0000", 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9042999999999995e-05 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436567+0000", 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000171519 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436734+0000", 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016694 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.465303+0000", 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028569302000000001 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555768+0000", 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090465569999999995 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555820+0000", 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1591000000000001e-05 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555870+0000", 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9660000000000002e-05 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:98 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.6299889859999999, 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42340282099999998, 2024-09-19T05:43:55.097 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 98 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437834+0000", 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.442377+0000", 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045437719999999997 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448982+0000", 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066041210000000001 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449009+0000", 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7898999999999998e-05 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449078+0000", 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8301999999999998e-05 2024-09-19T05:43:55.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449208+0000", 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013036300000000001 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449349+0000", 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000140791 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515702+0000", 2024-09-19T05:43:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066353102999999997 2024-09-19T05:43:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861115+0000", 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34541303600000001 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861183+0000", 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8238999999999995e-05 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861233+0000", 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0081999999999999e-05 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:140 3.20 3:06fc074c:::10000000002.00000089:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.466108+0000", 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.6017118560000001, 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.148574505, 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.466108+0000", 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.466108+0000", 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.466111+0000", 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4970000000000001e-06 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472886+0000", 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067752580000000001 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472908+0000", 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1637e-05 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472921+0000", 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3111e-05 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473382+0000", 2024-09-19T05:43:55.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046065800000000002 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473609+0000", 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022706000000000001 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.474321+0000", 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071182000000000005 2024-09-19T05:43:55.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.500062+0000", 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025741151 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614497+0000", 2024-09-19T05:43:55.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.114435146 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614595+0000", 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8363000000000007e-05 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614682+0000", 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6804000000000004e-05 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:143 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.594857083, 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40938193499999997, 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 143 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472965+0000", 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7879999999999998e-06 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486591+0000", 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013625417000000001 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486611+0000", 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0285e-05 2024-09-19T05:43:55.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486624+0000", 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2763e-05 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501834+0000", 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01521081 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504117+0000", 2024-09-19T05:43:55.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022829479999999999 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504306+0000", 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018812599999999999 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515709+0000", 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403529000000001 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882232+0000", 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36652316899999998 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882300+0000", 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7539000000000005e-05 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882344+0000", 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4561000000000002e-05 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.118 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:145 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.5811693600000001, 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39312144199999999, 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.119 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486653+0000", 2024-09-19T05:43:55.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0089999999999999e-06 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491674+0000", 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050208450000000003 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491698+0000", 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4457999999999999e-05 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491721+0000", 2024-09-19T05:43:55.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2294999999999999e-05 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510737+0000", 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019016000000000002 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510880+0000", 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014287999999999999 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511053+0000", 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017286299999999999 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555682+0000", 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044629755 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879645+0000", 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.323962272 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879715+0000", 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0400000000000004e-05 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879772+0000", 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6665e-05 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:150 3.e 3:7197baa1:::10000000002.00000093:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.491742+0000", 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.576077178, 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11303964800000001, 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 150 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.126 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491742+0000", 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491742+0000", 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491746+0000", 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.19e-06 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514720+0000", 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022974511 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514758+0000", 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7523e-05 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514776+0000", 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8130000000000001e-05 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515792+0000", 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001016215 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.516183+0000", 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039145100000000002 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.516512+0000", 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032865299999999999 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555335+0000", 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038823158000000003 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604640+0000", 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049304459000000002 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604699+0000", 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9694999999999997e-05 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604782+0000", 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2663000000000005e-05 2024-09-19T05:43:55.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:167 3.3a 3:5e20dd20:::10000000002.000000a4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.514813+0000", 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.5530067789999999, 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29944009100000002, 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514813+0000", 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514813+0000", 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514817+0000", 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7890000000000001e-06 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518589+0000", 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037720029999999999 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518610+0000", 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.143e-05 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518622+0000", 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2247e-05 2024-09-19T05:43:55.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522643+0000", 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040203679999999999 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522829+0000", 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186455 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.523002+0000", 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017278899999999999 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555658+0000", 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032656213000000003 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.814128+0000", 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25847003600000001 2024-09-19T05:43:55.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.814192+0000", 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3682000000000005e-05 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.814253+0000", 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1079000000000003e-05 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:170 3.3c 3:3ecd43ae:::10000000002.000000a7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.518652+0000", 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.54916751, 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28339853500000001, 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 170 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518652+0000", 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518652+0000", 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518655+0000", 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9840000000000001e-06 2024-09-19T05:43:55.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521398+0000", 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002743152 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521420+0000", 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1549999999999999e-05 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521432+0000", 2024-09-19T05:43:55.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2377e-05 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521509+0000", 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6852000000000002e-05 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521667+0000", 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015837499999999999 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521822+0000", 2024-09-19T05:43:55.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015457599999999999 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.553336+0000", 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031513855 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.801926+0000", 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.248589854 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.801985+0000", 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599000000000003e-05 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.802051+0000", 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5361e-05 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:175 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.144 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.540603977, 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35525569800000001, 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 175 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:43:55.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527221+0000", 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2299999999999999e-06 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531495+0000", 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004274211 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556121+0000", 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024626329999999998 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556132+0000", 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0740000000000001e-05 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556187+0000", 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5096000000000002e-05 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556372+0000", 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185059 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556560+0000", 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000187803 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.562115+0000", 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055550449999999998 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882386+0000", 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32027064199999999 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882435+0000", 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8990999999999999e-05 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882471+0000", 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6551000000000003e-05 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:176 3.26 3:66aef6e9:::10000000002.000000ad:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.561465+0000", 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.5063549489999999, 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32111455300000002, 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 176 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.561465+0000", 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.561465+0000", 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.561468+0000", 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2279999999999998e-06 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569661+0000", 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081929380000000003 2024-09-19T05:43:55.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569698+0000", 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7387e-05 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569715+0000", 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6761000000000001e-05 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569805+0000", 2024-09-19T05:43:55.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0279999999999996e-05 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569939+0000", 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013416799999999999 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570107+0000", 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016782400000000001 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.589296+0000", 2024-09-19T05:43:55.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019189319999999999 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882511+0000", 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29321486099999999 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882549+0000", 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7620000000000002e-05 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882579+0000", 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0165999999999999e-05 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:201 3.26 3:64ff7f1d:::10000000002.000000c6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.588102+0000", 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.479717833, 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29461220900000001, 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 201 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.588102+0000", 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.588102+0000", 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.588105+0000", 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3340000000000002e-06 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.599947+0000", 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011842020999999999 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.599974+0000", 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7016e-05 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.599987+0000", 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2884999999999999e-05 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600062+0000", 2024-09-19T05:43:55.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4686999999999998e-05 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600259+0000", 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019709600000000001 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600439+0000", 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180652 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.658837+0000", 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058397772000000001 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882617+0000", 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22377992199999999 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882677+0000", 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9871999999999999e-05 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882714+0000", 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6952000000000001e-05 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:61 3.11 e53/20 3:8969f477:::10000000002.0000003a:head v 53'1, mlcod=0'0)", 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.4694038790000001, 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26692470600000001, 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9400000000000004e-07 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.860816+0000", 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26239986500000001 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.860910+0000", 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4100999999999999e-05 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.860926+0000", 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5460000000000001e-05 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861060+0000", 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013446000000000001 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861089+0000", 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8538e-05 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.865245+0000", 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041562079999999998 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.865340+0000", 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5279999999999996e-05 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:202 3.3c 3:3e5783f1:::10000000002.000000c7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.600017+0000", 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.46780227, 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22482739400000001, 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 202 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600017+0000", 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600017+0000", 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600020+0000", 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9790000000000002e-06 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.605076+0000", 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050553270000000001 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.605091+0000", 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5926e-05 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.605101+0000", 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0019e-05 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.606538+0000", 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014364740000000001 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.606704+0000", 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016607300000000001 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.606870+0000", 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016643300000000001 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.626640+0000", 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019769554000000002 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.824727+0000", 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19808663300000001 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.824792+0000", 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5210999999999996e-05 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.824845+0000", 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2765e-05 2024-09-19T05:43:55.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:204 3.26 3:6430d4f3:::10000000002.000000c9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.621455+0000", 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.446364915, 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26136332899999998, 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 204 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621455+0000", 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621455+0000", 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621458+0000", 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6490000000000001e-06 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638150+0000", 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016691620000000001 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638191+0000", 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1056999999999997e-05 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638218+0000", 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7064e-05 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638552+0000", 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033372099999999999 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.639010+0000", 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000458598 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.639458+0000", 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044719699999999998 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.726753+0000", 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087295090000000006 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882750+0000", 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15599700399999999 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882787+0000", 2024-09-19T05:43:55.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7755e-05 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882818+0000", 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0574000000000002e-05 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:207 3.26 3:65689812:::10000000002.000000cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.638282+0000", 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.42953763, 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24464840600000001, 2024-09-19T05:43:55.179 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 207 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638282+0000", 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638282+0000", 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638286+0000", 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.561e-06 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651627+0000", 2024-09-19T05:43:55.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013340250999999999 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651645+0000", 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7807000000000001e-05 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651655+0000", 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0705e-05 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.658901+0000", 2024-09-19T05:43:55.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072462009999999999 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.659059+0000", 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015735599999999999 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.659257+0000", 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019826699999999999 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.729942+0000", 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.070684565000000005 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882855+0000", 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.152913733 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882898+0000", 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2595000000000001e-05 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882930+0000", 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2364999999999999e-05 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:211 3.26 3:65bb0d93:::10000000002.000000d0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.651679+0000", 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.4161409570000001, 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.231370309, 2024-09-19T05:43:55.185 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 211 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651679+0000", 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651679+0000", 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651682+0000", 2024-09-19T05:43:55.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.661340+0000", 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0096578649999999999 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.661360+0000", 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0250000000000001e-05 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.661371+0000", 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0754000000000001e-05 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.713738+0000", 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052367128999999998 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715955+0000", 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022169910000000002 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.716154+0000", 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019889300000000001 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.738197+0000", 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022043618000000001 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882967+0000", 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14477016200000001 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.883018+0000", 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1097999999999998e-05 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.883049+0000", 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0443999999999999e-05 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:249 3.16 3:689177d4:::10000000002.000000f6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.685256+0000", 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.3825637239999999, 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20163096899999999, 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 249 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.685256+0000", 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.685256+0000", 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.685259+0000", 2024-09-19T05:43:55.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.146e-06 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687156+0000", 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00189657 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687174+0000", 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8909000000000001e-05 2024-09-19T05:43:55.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687188+0000", 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3309000000000001e-05 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.730187+0000", 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042999355000000003 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.730356+0000", 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016865000000000001 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.730558+0000", 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020220500000000001 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.770736+0000", 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040177998999999999 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886787+0000", 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11605080399999999 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886844+0000", 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7083999999999998e-05 2024-09-19T05:43:55.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886887+0000", 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2938000000000001e-05 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:55.198 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:55.200 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:55.200 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:55.197+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:55.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:55.301 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:55.299+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:55.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.346+0000 7fd96d0d2640 1 Processor -- start 2024-09-19T05:43:55.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.346+0000 7fd96d0d2640 1 -- start start 2024-09-19T05:43:55.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96d0d2640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd968154d70 0x7fd96815b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96d0d2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9680c8d10 0x7fd968070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96d0d2640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd968070200 con 0x7fd9680c3330 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96d0d2640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd96815b090 con 0x7fd968154d70 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96d0d2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd9680c37e0 con 0x7fd9680c8d10 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd966d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd968154d70 0x7fd96815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96759e640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd9680c3330 0x7fd9680ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45026/0 (socket says 172.21.15.2:45026) 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd966d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd968154d70 0x7fd96815b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39100/0 (socket says 172.21.15.2:39100) 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96759e640 1 -- 172.21.15.2:0/3761564140 learned_addr learned my addr 172.21.15.2:0/3761564140 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd96659c640 1 --2- 172.21.15.2:0/3761564140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9680c8d10 0x7fd968070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd966d9d640 1 -- 172.21.15.2:0/3761564140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9680c8d10 msgr2=0x7fd968070780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:55.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd966d9d640 1 --2- 172.21.15.2:0/3761564140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9680c8d10 0x7fd968070780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd966d9d640 1 -- 172.21.15.2:0/3761564140 >> v1:172.21.15.2:6789/0 conn(0x7fd9680c3330 legacy=0x7fd9680ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.347+0000 7fd966d9d640 1 -- 172.21.15.2:0/3761564140 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9680704a0 con 0x7fd968154d70 2024-09-19T05:43:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.348+0000 7fd966d9d640 1 --2- 172.21.15.2:0/3761564140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd968154d70 0x7fd96815b520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd9580097a0 tx=0x7fd95802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b835ba70bdf281cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.348+0000 7fd965d9b640 1 -- 172.21.15.2:0/3761564140 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958037070 con 0x7fd968154d70 2024-09-19T05:43:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.348+0000 7fd965d9b640 1 -- 172.21.15.2:0/3761564140 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd958032440 con 0x7fd968154d70 2024-09-19T05:43:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.348+0000 7fd965d9b640 1 -- 172.21.15.2:0/3761564140 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958032810 con 0x7fd968154d70 2024-09-19T05:43:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 -- 172.21.15.2:0/3761564140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd968154d70 msgr2=0x7fd96815b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/3761564140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd968154d70 0x7fd96815b520 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd9580097a0 tx=0x7fd95802a180 comp rx=0 tx=0).stop 2024-09-19T05:43:55.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 -- 172.21.15.2:0/3761564140 shutdown_connections 2024-09-19T05:43:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/3761564140 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd9680c8d10 0x7fd968070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/3761564140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd968154d70 0x7fd96815b520 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 -- 172.21.15.2:0/3761564140 >> 172.21.15.2:0/3761564140 conn(0x7fd96806c240 msgr2=0x7fd968067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 -- 172.21.15.2:0/3761564140 shutdown_connections 2024-09-19T05:43:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.349+0000 7fd96d0d2640 1 -- 172.21.15.2:0/3761564140 wait complete. 2024-09-19T05:43:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.351+0000 7fd96d0d2640 1 Processor -- start 2024-09-19T05:43:55.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.351+0000 7fd96d0d2640 1 -- start start 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.352+0000 7fd96d0d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9680c3330 0x7fd9681f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.352+0000 7fd96d0d2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9680c8d10 0x7fd9680c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.352+0000 7fd96d0d2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd968154d70 0x7fd9681f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.352+0000 7fd96d0d2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd96806e710 con 0x7fd9680c8d10 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.352+0000 7fd96d0d2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd96815ad20 con 0x7fd9680c3330 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.352+0000 7fd96d0d2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd9680c5330 con 0x7fd968154d70 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.352+0000 7fd96659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9680c8d10 0x7fd9680c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9680c8d10 0x7fd9680c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39110/0 (socket says 172.21.15.2:39110) 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 -- 172.21.15.2:0/1348411947 learned_addr learned my addr 172.21.15.2:0/1348411947 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:55.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9680c3330 0x7fd9681f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96759e640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd968154d70 0x7fd9681f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd968154d70 msgr2=0x7fd9681f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd968154d70 0x7fd9681f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9680c3330 msgr2=0x7fd9681f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9680c3330 0x7fd9681f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9681f12e0 con 0x7fd9680c8d10 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9680c3330 0x7fd9681f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96659c640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9680c8d10 0x7fd9680c7200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00e8e0 tx=0x7fd95c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c017070 con 0x7fd9680c8d10 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c00c940 con 0x7fd9680c8d10 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9680c7dd0 con 0x7fd9680c8d10 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00cd10 con 0x7fd9680c8d10 2024-09-19T05:43:55.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.353+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9680c7b00 con 0x7fd9680c8d10 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.354+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd96806eba0 con 0x7fd9680c8d10 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.355+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd95c01a070 con 0x7fd9680c8d10 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.355+0000 7fd94f7fe640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd95406c2b0 0x7fd95406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.355+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7fd95c08b970 con 0x7fd9680c8d10 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.356+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7fd9681f12e0 con 0x7fd9680c8d10 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.356+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd95406c2b0 0x7fd95406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.356+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd968153ce0 0x7fd9680c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.356+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9680c8090 con 0x7fd968153ce0 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.356+0000 7fd96759e640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd968153ce0 0x7fd9680c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.356+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd95406c2b0 0x7fd95406e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd958034000 tx=0x7fd95802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.360 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:55.360 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:55.360 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:55.360 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:55.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.356+0000 7fd96759e640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd968153ce0 0x7fd9680c6720 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.360 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.357+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9680c8090 con 0x7fd968153ce0 2024-09-19T05:43:55.368 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:55.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.372+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd96806d510 con 0x7fd968153ce0 2024-09-19T05:43:55.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.373+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd96806d510 con 0x7fd968153ce0 2024-09-19T05:43:55.375 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:55.376 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.373+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd9681540c0 0x7fd9680b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.373+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd9680b2db0 con 0x7fd9681540c0 2024-09-19T05:43:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.374+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd9681540c0 0x7fd9680b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.374+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd9681540c0 0x7fd9680b2870 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.375+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9680b2db0 con 0x7fd9681540c0 2024-09-19T05:43:55.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.389+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9680aed50 con 0x7fd9681540c0 2024-09-19T05:43:55.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.391+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9680aed50 con 0x7fd9681540c0 2024-09-19T05:43:55.393 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:55.394 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:55.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.391+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd9681f3de0 0x7fd9681f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.391+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd9681f4720 con 0x7fd9681f3de0 2024-09-19T05:43:55.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.391+0000 7fd96659c640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd9681f3de0 0x7fd9681f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.392+0000 7fd96659c640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd9681f3de0 0x7fd9681f41e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.395 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.392+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd95c091020 con 0x7fd9681f3de0 2024-09-19T05:43:55.401 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:55.400+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:55.408 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.406+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd968062900 con 0x7fd9681f3de0 2024-09-19T05:43:55.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.408+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd968062900 con 0x7fd9681f3de0 2024-09-19T05:43:55.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:55.411 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:55.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.408+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd968150a80 0x7fd968150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.408+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd9680633e0 con 0x7fd968150a80 2024-09-19T05:43:55.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.408+0000 7fd96759e640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd968150a80 0x7fd968150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.409+0000 7fd96759e640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd968150a80 0x7fd968150ea0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.409+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9680633e0 con 0x7fd968150a80 2024-09-19T05:43:55.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.423+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9681513e0 con 0x7fd968150a80 2024-09-19T05:43:55.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.424+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9681513e0 con 0x7fd968150a80 2024-09-19T05:43:55.427 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:55.427 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:55.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.425+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd96802cb50 0x7fd96802cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.425+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd9681519b0 con 0x7fd96802cb50 2024-09-19T05:43:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.425+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd96802cb50 0x7fd96802cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.426+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd96802cb50 0x7fd96802cf90 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.426+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9681519b0 con 0x7fd96802cb50 2024-09-19T05:43:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.443+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9681519b0 con 0x7fd96802cb50 2024-09-19T05:43:55.447 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.445+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9681519b0 con 0x7fd96802cb50 2024-09-19T05:43:55.447 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:55.448 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:55.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.445+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd968204c50 0x7fd968207040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.445+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fd96802da90 con 0x7fd968204c50 2024-09-19T05:43:55.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.446+0000 7fd96659c640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd968204c50 0x7fd968207040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.448 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.446+0000 7fd96659c640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd968204c50 0x7fd968207040 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.449 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.447+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd96802da90 con 0x7fd968204c50 2024-09-19T05:43:55.463 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.461+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd968207580 con 0x7fd968204c50 2024-09-19T05:43:55.465 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.463+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd968207580 con 0x7fd968204c50 2024-09-19T05:43:55.465 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:55.466 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:55.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.463+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd968207a90 0x7fd968209e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.463+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fd96820ce60 con 0x7fd968207a90 2024-09-19T05:43:55.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.463+0000 7fd96759e640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd968207a90 0x7fd968209e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.464+0000 7fd96759e640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd968207a90 0x7fd968209e80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.467 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.464+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd96820ce60 con 0x7fd968207a90 2024-09-19T05:43:55.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.478+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd96820bba0 con 0x7fd968207a90 2024-09-19T05:43:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.480+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd96820bba0 con 0x7fd968207a90 2024-09-19T05:43:55.482 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:55.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.480+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd96820c2c0 0x7fd96820c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:55.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.480+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fd968210230 con 0x7fd96820c2c0 2024-09-19T05:43:55.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.480+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd96820c2c0 0x7fd96820c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:55.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.481+0000 7fd966d9d640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd96820c2c0 0x7fd96820c6e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:55.483 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:55.483 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:55.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.481+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd968210230 con 0x7fd96820c2c0 2024-09-19T05:43:55.492 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:55.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.495+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd96820f0b0 con 0x7fd96820c2c0 2024-09-19T05:43:55.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.497+0000 7fd94f7fe640 1 -- 172.21.15.2:0/1348411947 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd96820f0b0 con 0x7fd96820c2c0 2024-09-19T05:43:55.500 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:55.501 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:55.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd968153ce0 msgr2=0x7fd9680c6720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd968153ce0 0x7fd9680c6720 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd9681540c0 msgr2=0x7fd9680b2870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd9681540c0 0x7fd9680b2870 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd9681f3de0 msgr2=0x7fd9681f41e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd9681f3de0 0x7fd9681f41e0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd968150a80 msgr2=0x7fd968150ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd968150a80 0x7fd968150ea0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd96802cb50 msgr2=0x7fd96802cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd96802cb50 0x7fd96802cf90 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd968204c50 msgr2=0x7fd968207040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd968204c50 0x7fd968207040 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96759e640 1 -- 172.21.15.2:0/1348411947 reap_dead start 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96759e640 1 -- 172.21.15.2:0/1348411947 reap_dead start 2024-09-19T05:43:55.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd968207a90 msgr2=0x7fd968209e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd968207a90 0x7fd968209e80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd96820c2c0 msgr2=0x7fd96820c6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96759e640 1 -- 172.21.15.2:0/1348411947 reap_dead start 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd96820c2c0 0x7fd96820c6e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd95406c2b0 msgr2=0x7fd95406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd95406c2b0 0x7fd95406e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd958034000 tx=0x7fd95802e000 comp rx=0 tx=0).stop 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9680c8d10 msgr2=0x7fd9680c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9680c8d10 0x7fd9680c7200 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00e8e0 tx=0x7fd95c00eda0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 shutdown_connections 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd96820c2c0 0x7fd96820c6e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd95406c2b0 0x7fd95406e770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 --2- 172.21.15.2:0/1348411947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9680c8d10 0x7fd9680c7200 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 >> 172.21.15.2:0/1348411947 conn(0x7fd96806c240 msgr2=0x7fd9680cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:55.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 shutdown_connections 2024-09-19T05:43:55.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.498+0000 7fd96d0d2640 1 -- 172.21.15.2:0/1348411947 wait complete. 2024-09-19T05:43:55.505 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:55.503+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:55.605 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:55.606 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:55.606 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:55.604+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:55.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:55.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.662+0000 7f3980ff9640 1 -- 172.21.15.2:0/2254972310 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v54) ==== 91+0+0 (secure 0 0 0) 0x7f397c07a4d0 con 0x7f39840b0ee0 2024-09-19T05:43:55.665 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:43:55.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.665+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f396406c2b0 msgr2=0x7f396406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.665+0000 7f398a168640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f396406c2b0 0x7f396406e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3974000f60 tx=0x7f3974007d70 comp rx=0 tx=0).stop 2024-09-19T05:43:55.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.665+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39840b0ee0 msgr2=0x7f3984157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:55.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.665+0000 7f398a168640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39840b0ee0 0x7f3984157ad0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f397c009eb0 tx=0x7f397c004a40 comp rx=0 tx=0).stop 2024-09-19T05:43:55.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 shutdown_connections 2024-09-19T05:43:55.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f396406c2b0 0x7f396406e770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3984071110 0x7f3984155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39840b0ee0 0x7f3984157ad0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 --2- 172.21.15.2:0/2254972310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f398406ea80 0x7f398414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:55.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 >> 172.21.15.2:0/2254972310 conn(0x7f398406c240 msgr2=0x7f39840af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:55.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 shutdown_connections 2024-09-19T05:43:55.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:55.666+0000 7f398a168640 1 -- 172.21.15.2:0/2254972310 wait complete. 2024-09-19T05:43:55.706 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:55.704+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:55.737 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:55.737 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:55.747 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:55.807 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:55.806+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:55.851 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:55.851 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:55.860 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:43:55.908 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:55.906+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:55.963 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:55.964 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:55.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:43:56.008 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:56.006+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:56.074 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.075 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:43:56.109 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:56.108+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:56.186 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.186 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.194 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:43:56.211 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:56.209+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:56.296 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.297 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:43:56.350 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:56.348+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:56.409 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.409 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.412 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:56.411+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:56.418 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:43:56.514 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:56.512+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:56.561 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.562 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:43:56.614 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:56.613+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:56.715 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:56.713+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:56.715 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.716 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:43:56.815 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:56.814+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:56.827 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.827 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:43:56.920 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:56.919+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:56.980 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:56.980 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:56.988 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:43:57.030 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:43:57.028+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:43:57.118 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:57.117+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:57.132 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:57.132 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:57.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:43:57.219 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:57.217+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:57.266 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:57.266 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:57.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:43:57.359 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:57.357+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:57.421 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:57.419+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:57.423 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:57.424 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:57.433 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:43:57.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:43:57.561 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:57.559+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:57.582 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:43:57.582 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:43:57.592 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:43:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.616+0000 7fc4e49a6640 1 Processor -- start 2024-09-19T05:43:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.616+0000 7fc4e49a6640 1 -- start start 2024-09-19T05:43:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4e49a6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 0x7fc4e015b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4e49a6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc4e00cc2e0 0x7fc4e00cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4e49a6640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e00b3100 con 0x7fc4e015e200 2024-09-19T05:43:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4e49a6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e015b070 con 0x7fc4e00cc2e0 2024-09-19T05:43:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4e49a6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e00ca940 con 0x7fc4e00ca560 2024-09-19T05:43:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4ddd9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fc4e015e200 0x7fc4e00cb700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45038/0 (socket says 172.21.15.2:45038) 2024-09-19T05:43:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4ddd9b640 1 -- 172.21.15.2:0/816873488 learned_addr learned my addr 172.21.15.2:0/816873488 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/816873488 >> v2:172.21.15.123:3300/0 conn(0x7fc4e00cc2e0 0x7fc4e00cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310006152 0 0) 0x7fc4e00b3100 con 0x7fc4e015e200 2024-09-19T05:43:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4de59c640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 0x7fc4e015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.617+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4c8003520 con 0x7fc4e015e200 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1586144605 0 0) 0x7fc4c8003520 con 0x7fc4e015e200 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 msgr2=0x7fc4e015b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 0x7fc4e015b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 >> v2:172.21.15.123:3300/0 conn(0x7fc4e00cc2e0 msgr2=0x7fc4e00cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 --2- 172.21.15.2:0/816873488 >> v2:172.21.15.123:3300/0 conn(0x7fc4e00cc2e0 0x7fc4e00cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e015e740 con 0x7fc4e015e200 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7fc4d4001670 con 0x7fc4e015e200 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 >> v1:172.21.15.2:6789/0 conn(0x7fc4e015e200 legacy=0x7fc4e00cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 0x7fc4c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 0x7fc4c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c8004d40 0x7fc4c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e015b070 con 0x7fc4c8004d40 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e00ca940 con 0x7fc4c80039b0 2024-09-19T05:43:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c800b9c0 con 0x7fc4e00ca560 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c8004d40 0x7fc4c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4de59c640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 0x7fc4c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 0x7fc4c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4de59c640 1 -- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 msgr2=0x7fc4c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4de59c640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00ca560 0x7fc4c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4de59c640 1 -- 172.21.15.2:0/816873488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c8004d40 msgr2=0x7fc4c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4de59c640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c8004d40 0x7fc4c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4de59c640 1 -- 172.21.15.2:0/816873488 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c8003520 con 0x7fc4c80039b0 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.618+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4c8004d40 0x7fc4c80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.619+0000 7fc4de59c640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 0x7fc4c8003dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009fc0 tx=0x7fc4c8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc88b83db5f9f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.619+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0009c10 con 0x7fc4c80039b0 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.619+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d0036070 con 0x7fc4c80039b0 2024-09-19T05:43:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.619+0000 7fc4dd59a640 1 -- 172.21.15.2:0/816873488 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0031880 con 0x7fc4c80039b0 2024-09-19T05:43:57.625 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:57.620+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:57.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.620+0000 7fc4e49a6640 1 -- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 msgr2=0x7fc4c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.620+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/816873488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 0x7fc4c8003dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009fc0 tx=0x7fc4c8005830 comp rx=0 tx=0).stop 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.620+0000 7fc4ded9d640 1 -- 172.21.15.2:0/816873488 reap_dead start 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.620+0000 7fc4e49a6640 1 -- 172.21.15.2:0/816873488 shutdown_connections 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.620+0000 7fc4e49a6640 1 -- 172.21.15.2:0/816873488 >> 172.21.15.2:0/816873488 conn(0x7fc4e006d8d0 msgr2=0x7fc4e006e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.620+0000 7fc4e49a6640 1 -- 172.21.15.2:0/816873488 shutdown_connections 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.620+0000 7fc4e49a6640 1 -- 172.21.15.2:0/816873488 wait complete. 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.621+0000 7fc4e49a6640 1 Processor -- start 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.621+0000 7fc4e49a6640 1 -- start start 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4e49a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 0x7fc4e00c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4e49a6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00c5500 0x7fc4e01f0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4e49a6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e01f0dc0 0x7fc4e01f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4e49a6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e015efe0 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4e49a6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e015ed00 con 0x7fc4c80039b0 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4e49a6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e015ac90 con 0x7fc4e00c5500 2024-09-19T05:43:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e01f0dc0 0x7fc4e01f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4ded9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e01f0dc0 0x7fc4e01f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39126/0 (socket says 172.21.15.2:39126) 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4ded9d640 1 -- 172.21.15.2:0/406583920 learned_addr learned my addr 172.21.15.2:0/406583920 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00c5500 0x7fc4e01f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 0x7fc4e00c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4ded9d640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00c5500 msgr2=0x7fc4e01f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.622+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4e00c5500 0x7fc4e01f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4ded9d640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 msgr2=0x7fc4e00c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c80039b0 0x7fc4e00c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4ded9d640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e00ce660 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e01f0dc0 0x7fc4e01f3df0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00d15e0 tx=0x7fc4d8011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8009690 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d801b070 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e00d0d40 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8004460 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.623+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e00ce8d0 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.624+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc4e015c100 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.625+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc4d8012040 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.625+0000 7fc4ceffd640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4c006c2e0 0x7fc4c006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.625+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7fc4d808b0b0 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.625+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v54) ==== 54+0+16 (secure 0 0 0) 0x7fc4d808d730 con 0x7fc4e01f0dc0 2024-09-19T05:43:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.625+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4c006c2e0 0x7fc4c006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.626+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc4e00c6b50 0x7fc4e00c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.626+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4e00cc0c0 con 0x7fc4e00c6b50 2024-09-19T05:43:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.626+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc4e00c6b50 0x7fc4e00c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.626+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4c006c2e0 0x7fc4c006e7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0035000 tx=0x7fc4d002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.626+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc4e00c6b50 0x7fc4e00c8c60 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.627+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e00cc0c0 con 0x7fc4e00c6b50 2024-09-19T05:43:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.642+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e015ce00 con 0x7fc4e00c6b50 2024-09-19T05:43:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.644+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e015ce00 con 0x7fc4e00c6b50 2024-09-19T05:43:57.646 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.644+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc4e01f4840 0x7fc4e0154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.644+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc4e015c840 con 0x7fc4e01f4840 2024-09-19T05:43:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.644+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc4e01f4840 0x7fc4e0154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.645+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc4e01f4840 0x7fc4e0154370 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.645+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e015c840 con 0x7fc4e01f4840 2024-09-19T05:43:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.660+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e015c560 con 0x7fc4e01f4840 2024-09-19T05:43:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.662+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e015c560 con 0x7fc4e01f4840 2024-09-19T05:43:57.664 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.662+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc4e0154dc0 0x7fc4e0062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.662+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc4e00d12e0 con 0x7fc4e0154dc0 2024-09-19T05:43:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.662+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc4e0154dc0 0x7fc4e0062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.662+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc4e0154dc0 0x7fc4e0062630 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.663+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e00d12e0 con 0x7fc4e0154dc0 2024-09-19T05:43:57.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.677+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e0062b70 con 0x7fc4e0154dc0 2024-09-19T05:43:57.680 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:43:57.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.679+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e0062b70 con 0x7fc4e0154dc0 2024-09-19T05:43:57.681 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:57.682 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:57.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.679+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc4e01595f0 0x7fc4e0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.679+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc4e0062b70 con 0x7fc4e01595f0 2024-09-19T05:43:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.679+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc4e01595f0 0x7fc4e0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.679+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc4e01595f0 0x7fc4e0159a50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.680+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e0062b70 con 0x7fc4e01595f0 2024-09-19T05:43:57.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.694+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e0062b70 con 0x7fc4e01595f0 2024-09-19T05:43:57.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.695+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e0062b70 con 0x7fc4e01595f0 2024-09-19T05:43:57.698 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:57.698 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:57.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.696+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc4e015a2c0 0x7fc4e002cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.696+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc4e0062b70 con 0x7fc4e015a2c0 2024-09-19T05:43:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.696+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc4e015a2c0 0x7fc4e002cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.697+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc4e015a2c0 0x7fc4e002cb50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.699 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.697+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e0062b70 con 0x7fc4e015a2c0 2024-09-19T05:43:57.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.711+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e0062b70 con 0x7fc4e015a2c0 2024-09-19T05:43:57.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.713+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e0062b70 con 0x7fc4e015a2c0 2024-09-19T05:43:57.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:43:57.716 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:43:57.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.713+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc4e002d5a0 0x7fc4e0207390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.713+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc4e002da00 con 0x7fc4e002d5a0 2024-09-19T05:43:57.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.713+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc4e002d5a0 0x7fc4e0207390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.714+0000 7fc4ddd9b640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc4e002d5a0 0x7fc4e0207390 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.715+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e002da00 con 0x7fc4e002d5a0 2024-09-19T05:43:57.722 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:57.721+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:57.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.728+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e002da00 con 0x7fc4e002d5a0 2024-09-19T05:43:57.731 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:57.731 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:57.731 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:57.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.730+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e002da00 con 0x7fc4e002d5a0 2024-09-19T05:43:57.733 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:43:57.733 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:43:57.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.730+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc4e0207de0 0x7fc4e020a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.731+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc4e020cfa0 con 0x7fc4e0207de0 2024-09-19T05:43:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.731+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc4e0207de0 0x7fc4e020a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.731+0000 7fc4ded9d640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc4e0207de0 0x7fc4e020a1d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.732+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e020cfa0 con 0x7fc4e0207de0 2024-09-19T05:43:57.742 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:43:57.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.749+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e020a710 con 0x7fc4e0207de0 2024-09-19T05:43:57.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.751+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e020a710 con 0x7fc4e0207de0 2024-09-19T05:43:57.757 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:43:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:43:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.751+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc4e020ad90 0x7fc4e020d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.751+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc4e02100e0 con 0x7fc4e020ad90 2024-09-19T05:43:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.751+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc4e020ad90 0x7fc4e020d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.752+0000 7fc4de59c640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc4e020ad90 0x7fc4e020d180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.752+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4e02100e0 con 0x7fc4e020ad90 2024-09-19T05:43:57.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.766+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fc4e020ee00 con 0x7fc4e020ad90 2024-09-19T05:43:57.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4ceffd640 1 -- 172.21.15.2:0/406583920 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4e020ee00 con 0x7fc4e020ad90 2024-09-19T05:43:57.770 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc4e00c6b50 msgr2=0x7fc4e00c8c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc4e00c6b50 0x7fc4e00c8c60 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc4e01f4840 msgr2=0x7fc4e0154370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc4e01f4840 0x7fc4e0154370 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc4e0154dc0 msgr2=0x7fc4e0062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc4e0154dc0 0x7fc4e0062630 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc4e01595f0 msgr2=0x7fc4e0159a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc4e01595f0 0x7fc4e0159a50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc4e015a2c0 msgr2=0x7fc4e002cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc4e015a2c0 0x7fc4e002cb50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4ded9d640 1 -- 172.21.15.2:0/406583920 reap_dead start 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc4e002d5a0 msgr2=0x7fc4e0207390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.768+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc4e002d5a0 0x7fc4e0207390 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc4e0207de0 msgr2=0x7fc4e020a1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4ded9d640 1 -- 172.21.15.2:0/406583920 reap_dead start 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc4e0207de0 0x7fc4e020a1d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc4e020ad90 msgr2=0x7fc4e020d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc4e020ad90 0x7fc4e020d180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4ded9d640 1 -- 172.21.15.2:0/406583920 reap_dead start 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4c006c2e0 msgr2=0x7fc4c006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4c006c2e0 0x7fc4c006e7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0035000 tx=0x7fc4d002f000 comp rx=0 tx=0).stop 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e01f0dc0 msgr2=0x7fc4e01f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e01f0dc0 0x7fc4e01f3df0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00d15e0 tx=0x7fc4d8011d70 comp rx=0 tx=0).stop 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 shutdown_connections 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc4c006c2e0 0x7fc4c006e7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 --2- 172.21.15.2:0/406583920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e01f0dc0 0x7fc4e01f3df0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 >> 172.21.15.2:0/406583920 conn(0x7fc4e006d8d0 msgr2=0x7fc4e0157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 shutdown_connections 2024-09-19T05:43:57.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.769+0000 7fc4e49a6640 1 -- 172.21.15.2:0/406583920 wait complete. 2024-09-19T05:43:57.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.786+0000 7f98d8dbe640 1 Processor -- start 2024-09-19T05:43:57.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.786+0000 7f98d8dbe640 1 -- start start 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d8dbe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f98d4151970 0x7f98d406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d8dbe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98d40ca850 0x7f98d415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d8dbe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d415e620 0x7f98d40cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d8dbe640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f98d415ba50 con 0x7f98d4151970 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d8dbe640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f98d406d260 con 0x7f98d40ca850 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d8dbe640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f98d4151d50 con 0x7f98d415e620 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f98d4151970 0x7f98d406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f98d4151970 0x7f98d406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39136/0 (socket says 172.21.15.2:39136) 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d259c640 1 -- 172.21.15.2:0/2302435726 learned_addr learned my addr 172.21.15.2:0/2302435726 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d1d9b640 1 --2- 172.21.15.2:0/2302435726 >> v2:172.21.15.123:3300/0 conn(0x7f98d40ca850 0x7f98d415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.787+0000 7f98d2d9d640 1 --2- 172.21.15.2:0/2302435726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d415e620 0x7f98d40cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.788+0000 7f98d259c640 1 -- 172.21.15.2:0/2302435726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d415e620 msgr2=0x7f98d40cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:57.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.788+0000 7f98d259c640 1 --2- 172.21.15.2:0/2302435726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d415e620 0x7f98d40cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.788+0000 7f98d259c640 1 -- 172.21.15.2:0/2302435726 >> v2:172.21.15.123:3300/0 conn(0x7f98d40ca850 msgr2=0x7f98d415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:57.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.788+0000 7f98d259c640 1 --2- 172.21.15.2:0/2302435726 >> v2:172.21.15.123:3300/0 conn(0x7f98d40ca850 0x7f98d415e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.788+0000 7f98d259c640 1 -- 172.21.15.2:0/2302435726 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d406d480 con 0x7f98d4151970 2024-09-19T05:43:57.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.788+0000 7f98d259c640 1 --2- 172.21.15.2:0/2302435726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d4151970 0x7f98d406cd20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98d40caee0 tx=0x7f98c402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f91945f5f02869c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d159a640 1 -- 172.21.15.2:0/2302435726 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4037070 con 0x7f98d4151970 2024-09-19T05:43:57.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d159a640 1 -- 172.21.15.2:0/2302435726 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c4032440 con 0x7f98d4151970 2024-09-19T05:43:57.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d159a640 1 -- 172.21.15.2:0/2302435726 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4032810 con 0x7f98d4151970 2024-09-19T05:43:57.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 -- 172.21.15.2:0/2302435726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d4151970 msgr2=0x7f98d406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/2302435726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d4151970 0x7f98d406cd20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f98d40caee0 tx=0x7f98c402a140 comp rx=0 tx=0).stop 2024-09-19T05:43:57.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 -- 172.21.15.2:0/2302435726 shutdown_connections 2024-09-19T05:43:57.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/2302435726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d415e620 0x7f98d40cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/2302435726 >> v2:172.21.15.123:3300/0 conn(0x7f98d40ca850 0x7f98d415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/2302435726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d4151970 0x7f98d406cd20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 -- 172.21.15.2:0/2302435726 >> 172.21.15.2:0/2302435726 conn(0x7f98d406bdf0 msgr2=0x7f98d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:57.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 -- 172.21.15.2:0/2302435726 shutdown_connections 2024-09-19T05:43:57.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.789+0000 7f98d8dbe640 1 -- 172.21.15.2:0/2302435726 wait complete. 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 Processor -- start 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 -- start start 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98d40ca850 0x7f98d41f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d4151970 0x7f98d415cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d415e620 0x7f98d415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f98d415b5c0 con 0x7f98d415e620 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98d406d760 con 0x7f98d40ca850 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.791+0000 7f98d8dbe640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f98d415af70 con 0x7f98d4151970 2024-09-19T05:43:57.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d415e620 0x7f98d415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d415e620 0x7f98d415d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39138/0 (socket says 172.21.15.2:39138) 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 -- 172.21.15.2:0/3438896303 learned_addr learned my addr 172.21.15.2:0/3438896303 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d1d9b640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d4151970 0x7f98d415cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d259c640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98d40ca850 0x7f98d41f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 -- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d4151970 msgr2=0x7f98d415cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d4151970 0x7f98d415cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 -- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98d40ca850 msgr2=0x7f98d41f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98d40ca850 0x7f98d41f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 -- 172.21.15.2:0/3438896303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d41f5140 con 0x7f98d415e620 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d1d9b640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d4151970 0x7f98d415cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:57.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d2d9d640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d415e620 0x7f98d415d420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00efc0 tx=0x7f98cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98caffd640 1 -- 172.21.15.2:0/3438896303 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc018020 con 0x7f98d415e620 2024-09-19T05:43:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98caffd640 1 -- 172.21.15.2:0/3438896303 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc014480 con 0x7f98d415e620 2024-09-19T05:43:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d41f5400 con 0x7f98d415e620 2024-09-19T05:43:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98caffd640 1 -- 172.21.15.2:0/3438896303 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc014870 con 0x7f98d415e620 2024-09-19T05:43:57.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.792+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d40c8610 con 0x7f98d415e620 2024-09-19T05:43:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.794+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d406c730 con 0x7f98d415e620 2024-09-19T05:43:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.794+0000 7f98caffd640 1 -- 172.21.15.2:0/3438896303 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f98cc005d10 con 0x7f98d415e620 2024-09-19T05:43:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.794+0000 7f98caffd640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f98b406c2b0 0x7f98b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.794+0000 7f98caffd640 1 -- 172.21.15.2:0/3438896303 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f98cc08bdc0 con 0x7f98d415e620 2024-09-19T05:43:57.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.794+0000 7f98d259c640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f98b406c2b0 0x7f98b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:57.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.795+0000 7f98d259c640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f98b406c2b0 0x7f98b406e770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98c4034000 tx=0x7f98c402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:57.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:57.799+0000 7f98caffd640 1 -- 172.21.15.2:0/3438896303 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f98d406c730 con 0x7f98d415e620 2024-09-19T05:43:57.824 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:57.822+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:57.884 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:57.884 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:57.884 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:57.884 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:57.884 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:57.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:43:57.935 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:57.933+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:58.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.015+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f98d4150a90 con 0x7f98d415e620 2024-09-19T05:43:58.026 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:58.025+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:58.045 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:58.045 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:58.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:58.045 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:58.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 95.163156375, 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:43:58.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:7 3.2a 3:557f550e:::10000000002.00000004:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7597080700000003, 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27494626900000002, 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 7 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282495+0000", 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4609999999999998e-06 2024-09-19T05:43:58.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286920+0000", 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044248229999999996 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286972+0000", 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1659000000000001e-05 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286986+0000", 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287117+0000", 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131328 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287358+0000", 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024141700000000001 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287543+0000", 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018436700000000001 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299025+0000", 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011482327000000001 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557305+0000", 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25828001099999998 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557360+0000", 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5074000000000002e-05 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557439+0000", 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9018999999999997e-05 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:8 3.1c 3:3ae0713d:::10000000002.00000005:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:58.058 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.296068+0000", 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7461333230000005, 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26154461800000001, 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 8 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.296068+0000", 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.296068+0000", 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.296072+0000", 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3229999999999999e-06 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301751+0000", 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056786409999999999 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301774+0000", 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3475000000000001e-05 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301792+0000", 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.301884+0000", 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2305999999999993e-05 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302046+0000", 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016148399999999999 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302196+0000", 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000149746 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.334012+0000", 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031816143999999998 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557498+0000", 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22348642599999999 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557572+0000", 2024-09-19T05:43:58.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3670000000000004e-05 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557612+0000", 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0459e-05 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:43 3.1a 3:5917aad8:::10000000002.00000028:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.337239+0000", 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7049615390000001, 2024-09-19T05:43:58.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.064239559000000002, 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 43 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337239+0000", 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337239+0000", 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337242+0000", 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.559e-06 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.350884+0000", 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013642101 2024-09-19T05:43:58.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.350918+0000", 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3729000000000002e-05 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.350933+0000", 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5072000000000001e-05 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.351549+0000", 2024-09-19T05:43:58.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061607800000000005 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.351723+0000", 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017419399999999999 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.351889+0000", 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016625 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.358475+0000", 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065857149999999998 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.401355+0000", 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042879408000000001 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.401422+0000", 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7373999999999996e-05 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.401479+0000", 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7079e-05 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:45 3.37 3:ec154225:::10000000002.0000002a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.356538+0000", 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6856633189999997, 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093610510999999993, 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 45 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.356538+0000", 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.356538+0000", 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.356541+0000", 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1259999999999998e-06 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369575+0000", 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013033805000000001 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369594+0000", 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9247000000000001e-05 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369606+0000", 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2062000000000001e-05 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369683+0000", 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7136000000000001e-05 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369835+0000", 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015142700000000001 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369987+0000", 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000152495 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.399199+0000", 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029212444000000001 2024-09-19T05:43:58.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.450027+0000", 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050827259 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.450088+0000", 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1193999999999997e-05 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.450148+0000", 2024-09-19T05:43:58.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0316000000000003e-05 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:47 3.37 3:ec3fc134:::10000000002.0000002c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.369644+0000", 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6725570190000001, 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15125028900000001, 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 47 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369644+0000", 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369644+0000", 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.369647+0000", 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8789999999999999e-06 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.371739+0000", 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020923449999999998 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.371764+0000", 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4794999999999999e-05 2024-09-19T05:43:58.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.371778+0000", 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3712e-05 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.377906+0000", 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006128451 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.378113+0000", 2024-09-19T05:43:58.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000206744 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.378281+0000", 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000168572 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.404124+0000", 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025842774999999998 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.520779+0000", 2024-09-19T05:43:58.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.116654297 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.520841+0000", 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2343999999999998e-05 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.520894+0000", 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3375000000000003e-05 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:59 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.402045+0000", 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6401560630000001, 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20849380200000001, 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 59 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.402045+0000", 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.402045+0000", 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.402048+0000", 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8550000000000002e-06 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406834+0000", 2024-09-19T05:43:58.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047862850000000004 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406855+0000", 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0975000000000002e-05 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406868+0000", 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2842e-05 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.406972+0000", 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104435 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.407208+0000", 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023539700000000001 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.407400+0000", 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019240899999999999 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.414059+0000", 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066590160000000002 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.610422+0000", 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19636292 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.610484+0000", 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1494000000000004e-05 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.610539+0000", 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5173999999999997e-05 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:91 3.2a 3:5747858d:::10000000002.00000058:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.411935+0000", 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6302664350000002, 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21859863900000001, 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 91 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.411935+0000", 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.411935+0000", 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.411937+0000", 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415027+0000", 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030898309999999999 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415290+0000", 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026321599999999998 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415302+0000", 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1488000000000001e-05 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415368+0000", 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6047e-05 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415532+0000", 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016381100000000001 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415790+0000", 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000258452 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489636+0000", 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073845375000000005 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630388+0000", 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.140752496 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630468+0000", 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9944999999999994e-05 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.630533+0000", 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5194e-05 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.095 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:92 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6265600520000003, 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37051118500000002, 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415644+0000", 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:43:58.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.422983+0000", 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073390779999999997 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423020+0000", 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7660000000000002e-05 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423036+0000", 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5359999999999999e-05 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423261+0000", 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225217 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423458+0000", 2024-09-19T05:43:58.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196555 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423737+0000", 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027922100000000002 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.435447+0000", 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011709855999999999 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.785961+0000", 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35051446400000003 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786086+0000", 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012537599999999999 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786152+0000", 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691000000000005e-05 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:97 3.1a 3:5b190c62:::10000000002.0000005e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.427800+0000", 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6144012700000001, 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.062014485000000001, 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 97 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.427800+0000", 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.427800+0000", 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.427803+0000", 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4130000000000002e-06 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429781+0000", 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019773500000000001 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429801+0000", 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429817+0000", 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5299e-05 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.431876+0000", 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00205879 2024-09-19T05:43:58.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.432072+0000", 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196851 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.432273+0000", 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020102600000000001 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.454877+0000", 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022603068 2024-09-19T05:43:58.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489656+0000", 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034779657999999998 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489736+0000", 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9587999999999998e-05 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489814+0000", 2024-09-19T05:43:58.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8474999999999997e-05 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:104 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6123337299999996, 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28556600799999998, 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:43:58.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429870+0000", 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-06 2024-09-19T05:43:58.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446619+0000", 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016749447000000001 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446650+0000", 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0218000000000001e-05 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446666+0000", 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6920999999999999e-05 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448857+0000", 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021902699999999998 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449011+0000", 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015441700000000001 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449168+0000", 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156527 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489828+0000", 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040659908000000002 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715285+0000", 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22545759500000001 2024-09-19T05:43:58.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715345+0000", 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0269999999999997e-05 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715433+0000", 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7814999999999999e-05 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:12 3.2c e53/38 3:354fa1eb:::10000000002.00000009:head v 53'1, mlcod=0'0)", 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6115512780000003, 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.124694371, 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430650+0000", 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1199999999999999e-07 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541667+0000", 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.111017229 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541684+0000", 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7183000000000001e-05 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541697+0000", 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2177999999999999e-05 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541807+0000", 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110747 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.541835+0000", 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7338000000000002e-05 2024-09-19T05:43:58.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.554544+0000", 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012709573 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555344+0000", 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00079991100000000005 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:108 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.595482713, 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26890963299999998, 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.120 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446721+0000", 2024-09-19T05:43:58.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8250000000000001e-06 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451943+0000", 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052222140000000002 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451961+0000", 2024-09-19T05:43:58.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8117e-05 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451970+0000", 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7420000000000003e-06 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473034+0000", 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021064282 2024-09-19T05:43:58.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473242+0000", 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020716699999999999 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473446+0000", 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204396 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489832+0000", 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016385994000000001 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715506+0000", 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22567427700000001 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715577+0000", 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0548000000000003e-05 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715628+0000", 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070999999999999e-05 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:139 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5786713070000005, 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.328123891, 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463532+0000", 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0339999999999999e-06 2024-09-19T05:43:58.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468331+0000", 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047995099999999999 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468365+0000", 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3667000000000001e-05 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468382+0000", 2024-09-19T05:43:58.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6651e-05 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468915+0000", 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053293299999999997 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469199+0000", 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028406900000000002 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469482+0000", 2024-09-19T05:43:58.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028390000000000002 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.485026+0000", 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015543509 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791538+0000", 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30651197400000002 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791602+0000", 2024-09-19T05:43:58.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3780000000000003e-05 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791654+0000", 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1863999999999997e-05 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:148 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:43:58.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5602212729999998, 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34695000399999998, 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 148 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:43:58.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481982+0000", 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6309999999999999e-06 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489765+0000", 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077821310000000003 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489778+0000", 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3604999999999999e-05 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489787+0000", 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489875+0000", 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8039000000000002e-05 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490101+0000", 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022636299999999999 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490426+0000", 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032453799999999997 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.506325+0000", 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015899149000000001 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828797+0000", 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32247239900000002 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828865+0000", 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7521999999999995e-05 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828930+0000", 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4795999999999994e-05 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:156 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5523660660000003, 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26935146300000001, 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:43:58.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489837+0000", 2024-09-19T05:43:58.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2689999999999998e-06 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495152+0000", 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053150209999999996 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495356+0000", 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203537 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495373+0000", 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496071+0000", 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069810200000000003 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496246+0000", 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017547299999999999 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496469+0000", 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022283200000000001 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.540840+0000", 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044371318999999999 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759082+0000", 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21824185500000001 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759137+0000", 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645000000000003e-05 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759186+0000", 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9499000000000002e-05 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:185 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5396862220000003, 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32688410400000001, 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 185 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:43:58.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502518+0000", 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.509973+0000", 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074553919999999999 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510031+0000", 2024-09-19T05:43:58.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8356999999999997e-05 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510046+0000", 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932e-05 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510135+0000", 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8548000000000007e-05 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510886+0000", 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075159700000000001 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511079+0000", 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019261500000000001 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532794+0000", 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021714989000000001 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829312+0000", 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29651747699999997 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829360+0000", 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8396000000000002e-05 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829399+0000", 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8961999999999997e-05 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.150 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:198 3.37 3:ec162ecd:::10000000002.000000c3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.518231+0000", 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5239698199999996, 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063948191000000001, 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.151 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 198 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518231+0000", 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518231+0000", 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518235+0000", 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.693e-06 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531247+0000", 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013012271000000001 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531283+0000", 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5550999999999999e-05 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531299+0000", 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6178999999999999e-05 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531471+0000", 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017213499999999999 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532503+0000", 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010324290000000001 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532706+0000", 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020218699999999999 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.554522+0000", 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021816397000000001 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.582075+0000", 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027552706999999999 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.582127+0000", 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.202e-05 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.582179+0000", 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2621999999999999e-05 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:257 3.21 e53/38 3:86aaefcc:::10000000002.000000fe:head v 53'5, mlcod=53'3)", 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.522406+0000", 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5197953479999997, 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.088968622999999997, 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522406+0000", 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522406+0000", 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522408+0000", 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9470000000000002e-06 2024-09-19T05:43:58.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570461+0000", 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.048053235999999999 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570477+0000", 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.658e-05 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570489+0000", 2024-09-19T05:43:58.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1989e-05 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570569+0000", 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9709999999999994e-05 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570636+0000", 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6829999999999995e-05 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.611243+0000", 2024-09-19T05:43:58.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040607074999999999 2024-09-19T05:43:58.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.611374+0000", 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013125600000000001 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:119 3.f e53/38 3:f3777bdc:::10000000002.00000074:head v 53'3, mlcod=0'0)", 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.641756+0000", 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4004446870000002, 2024-09-19T05:43:58.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.089892116999999994, 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.641756+0000", 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.641756+0000", 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.641758+0000", 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5340000000000001e-06 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715078+0000", 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073320084999999993 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715103+0000", 2024-09-19T05:43:58.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4929000000000001e-05 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715118+0000", 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4707e-05 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715319+0000", 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200965 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715347+0000", 2024-09-19T05:43:58.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8847999999999999e-05 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.731604+0000", 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016256900000000001 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.731648+0000", 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4149e-05 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.170 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:58.171 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:43:58.171 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:58.169+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:58.272 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:58.270+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:58.279 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:58.279 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:58.280 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:43:58.280 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:58.288 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:43:58.373 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:43:58.371+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:43:58.442 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:58.443 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:58.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:43:58.494 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:58.492+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:58.575 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:58.573+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:58.671 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:67 3.16 3:6a386151:::10000000002.00000040:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.343583+0000", 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.323247694, 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25356932300000001, 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 67 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343583+0000", 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343583+0000", 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343586+0000", 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.055e-06 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348022+0000", 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044362830000000001 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348101+0000", 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8364999999999999e-05 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348119+0000", 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8419000000000001e-05 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348552+0000", 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00043272500000000002 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348832+0000", 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028048999999999999 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.349057+0000", 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022431799999999999 2024-09-19T05:43:58.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.370068+0000", 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021011193000000001 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.597045+0000", 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22697705000000001 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.597098+0000", 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2871999999999998e-05 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.597152+0000", 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4552999999999998e-05 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:77 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.31867438, 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40221933999999998, 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 77 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348160+0000", 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0459999999999999e-06 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362035+0000", 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013874391999999999 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362063+0000", 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7804e-05 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362076+0000", 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3200000000000001e-05 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362162+0000", 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6469000000000002e-05 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362317+0000", 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015507100000000001 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362454+0000", 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013636799999999999 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.372327+0000", 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0098729219999999993 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750183+0000", 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37785643600000002 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750233+0000", 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9960999999999998e-05 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750376+0000", 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142671 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.684 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:87 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.284209454, 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39257614200000002, 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382623+0000", 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2529999999999999e-06 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390766+0000", 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081425790000000005 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390808+0000", 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2311999999999997e-05 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390827+0000", 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390955+0000", 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012825900000000001 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391148+0000", 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000192673 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391432+0000", 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028362599999999998 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.455303+0000", 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063870983000000006 2024-09-19T05:43:58.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775090+0000", 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31978768600000002 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775146+0000", 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5393000000000003e-05 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775197+0000", 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1699000000000001e-05 2024-09-19T05:43:58.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:94 3.5 3:a282a134:::10000000002.0000005b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.426351+0000", 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2404796229999997, 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951865500000001, 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 94 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.426351+0000", 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.426351+0000", 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.426355+0000", 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7639999999999999e-06 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429819+0000", 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034642869999999999 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436292+0000", 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064730129999999997 2024-09-19T05:43:58.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436316+0000", 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3966000000000001e-05 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436395+0000", 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9042999999999995e-05 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436567+0000", 2024-09-19T05:43:58.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000171519 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.436734+0000", 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016694 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.465303+0000", 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028569302000000001 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555768+0000", 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.090465569999999995 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555820+0000", 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1591000000000001e-05 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555870+0000", 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9660000000000002e-05 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:98 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2290001430000004, 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42340282099999998, 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 98 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437834+0000", 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.442377+0000", 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045437719999999997 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448982+0000", 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066041210000000001 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449009+0000", 2024-09-19T05:43:58.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7898999999999998e-05 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449078+0000", 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8301999999999998e-05 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449208+0000", 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013036300000000001 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449349+0000", 2024-09-19T05:43:58.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000140791 2024-09-19T05:43:58.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515702+0000", 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066353102999999997 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861115+0000", 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34541303600000001 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861183+0000", 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8238999999999995e-05 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861233+0000", 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0081999999999999e-05 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:140 3.20 3:06fc074c:::10000000002.00000089:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.466108+0000", 2024-09-19T05:43:58.705 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2007230130000002, 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.148574505, 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 140 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.706 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.466108+0000", 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.466108+0000", 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.466111+0000", 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4970000000000001e-06 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472886+0000", 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067752580000000001 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472908+0000", 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1637e-05 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472921+0000", 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3111e-05 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473382+0000", 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046065800000000002 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473609+0000", 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022706000000000001 2024-09-19T05:43:58.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.474321+0000", 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071182000000000005 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.500062+0000", 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025741151 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614497+0000", 2024-09-19T05:43:58.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.114435146 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614595+0000", 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8363000000000007e-05 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.614682+0000", 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6804000000000004e-05 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:143 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1938682399999996, 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40938193499999997, 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 143 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.712 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472965+0000", 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7879999999999998e-06 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486591+0000", 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013625417000000001 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486611+0000", 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0285e-05 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486624+0000", 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2763e-05 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501834+0000", 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01521081 2024-09-19T05:43:58.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504117+0000", 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022829479999999999 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504306+0000", 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018812599999999999 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515709+0000", 2024-09-19T05:43:58.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403529000000001 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882232+0000", 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36652316899999998 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882300+0000", 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7539000000000005e-05 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882344+0000", 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4561000000000002e-05 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:145 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1801805170000002, 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39312144199999999, 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:43:58.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486653+0000", 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0089999999999999e-06 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491674+0000", 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050208450000000003 2024-09-19T05:43:58.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491698+0000", 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4457999999999999e-05 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491721+0000", 2024-09-19T05:43:58.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2294999999999999e-05 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510737+0000", 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019016000000000002 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510880+0000", 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014287999999999999 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511053+0000", 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017286299999999999 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555682+0000", 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044629755 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879645+0000", 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.323962272 2024-09-19T05:43:58.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879715+0000", 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0400000000000004e-05 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879772+0000", 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6665e-05 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:150 3.e 3:7197baa1:::10000000002.00000093:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.491742+0000", 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1750883349999999, 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11303964800000001, 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 150 2024-09-19T05:43:58.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491742+0000", 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491742+0000", 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491746+0000", 2024-09-19T05:43:58.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.19e-06 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514720+0000", 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022974511 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514758+0000", 2024-09-19T05:43:58.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7523e-05 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514776+0000", 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8130000000000001e-05 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515792+0000", 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001016215 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.516183+0000", 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039145100000000002 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.516512+0000", 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032865299999999999 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555335+0000", 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038823158000000003 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604640+0000", 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049304459000000002 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604699+0000", 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9694999999999997e-05 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.604782+0000", 2024-09-19T05:43:58.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2663000000000005e-05 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:167 3.3a 3:5e20dd20:::10000000002.000000a4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.514813+0000", 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.152017936, 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29944009100000002, 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514813+0000", 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514813+0000", 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.514817+0000", 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7890000000000001e-06 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518589+0000", 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037720029999999999 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518610+0000", 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.143e-05 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518622+0000", 2024-09-19T05:43:58.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2247e-05 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522643+0000", 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040203679999999999 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.522829+0000", 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186455 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:58.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.523002+0000", 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017278899999999999 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555658+0000", 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032656213000000003 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.814128+0000", 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25847003600000001 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.814192+0000", 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3682000000000005e-05 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.814253+0000", 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1079000000000003e-05 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:170 3.3c 3:3ecd43ae:::10000000002.000000a7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.518652+0000", 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1481786669999998, 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28339853500000001, 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 170 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518652+0000", 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518652+0000", 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.518655+0000", 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9840000000000001e-06 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521398+0000", 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002743152 2024-09-19T05:43:58.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521420+0000", 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1549999999999999e-05 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521432+0000", 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2377e-05 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521509+0000", 2024-09-19T05:43:58.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6852000000000002e-05 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521667+0000", 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015837499999999999 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.521822+0000", 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015457599999999999 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.553336+0000", 2024-09-19T05:43:58.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031513855 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.801926+0000", 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.248589854 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.801985+0000", 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599000000000003e-05 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.802051+0000", 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5361e-05 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:175 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:43:58.744 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1396151339999996, 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35525569800000001, 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 175 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527221+0000", 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2299999999999999e-06 2024-09-19T05:43:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531495+0000", 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004274211 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556121+0000", 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024626329999999998 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556132+0000", 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0740000000000001e-05 2024-09-19T05:43:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556187+0000", 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5096000000000002e-05 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556372+0000", 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185059 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556560+0000", 2024-09-19T05:43:58.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000187803 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.562115+0000", 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055550449999999998 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882386+0000", 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32027064199999999 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882435+0000", 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8990999999999999e-05 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882471+0000", 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6551000000000003e-05 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:176 3.26 3:66aef6e9:::10000000002.000000ad:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.561465+0000", 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.105366106, 2024-09-19T05:43:58.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32111455300000002, 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 176 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.561465+0000", 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.561465+0000", 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.561468+0000", 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2279999999999998e-06 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569661+0000", 2024-09-19T05:43:58.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081929380000000003 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569698+0000", 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7387e-05 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569715+0000", 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6761000000000001e-05 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569805+0000", 2024-09-19T05:43:58.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0279999999999996e-05 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.569939+0000", 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013416799999999999 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.570107+0000", 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016782400000000001 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.589296+0000", 2024-09-19T05:43:58.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019189319999999999 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882511+0000", 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29321486099999999 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882549+0000", 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7620000000000002e-05 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882579+0000", 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0165999999999999e-05 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:201 3.26 3:64ff7f1d:::10000000002.000000c6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.588102+0000", 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0787289900000001, 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29461220900000001, 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.756 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 201 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.588102+0000", 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.588102+0000", 2024-09-19T05:43:58.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.588105+0000", 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3340000000000002e-06 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.599947+0000", 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011842020999999999 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.599974+0000", 2024-09-19T05:43:58.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7016e-05 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.599987+0000", 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2884999999999999e-05 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600062+0000", 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4686999999999998e-05 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600259+0000", 2024-09-19T05:43:58.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019709600000000001 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600439+0000", 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180652 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.658837+0000", 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058397772000000001 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882617+0000", 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22377992199999999 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882677+0000", 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9871999999999999e-05 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882714+0000", 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6952000000000001e-05 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.761 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:61 3.11 e53/20 3:8969f477:::10000000002.0000003a:head v 53'1, mlcod=0'0)", 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0684150360000002, 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26692470600000001, 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.598416+0000", 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9400000000000004e-07 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.860816+0000", 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26239986500000001 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.860910+0000", 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4100999999999999e-05 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.860926+0000", 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5460000000000001e-05 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861060+0000", 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013446000000000001 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861089+0000", 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8538e-05 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.865245+0000", 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041562079999999998 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.865340+0000", 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5279999999999996e-05 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.766 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:202 3.3c 3:3e5783f1:::10000000002.000000c7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.600017+0000", 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0668134269999996, 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22482739400000001, 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 202 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600017+0000", 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600017+0000", 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.600020+0000", 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9790000000000002e-06 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.605076+0000", 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050553270000000001 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.605091+0000", 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5926e-05 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.605101+0000", 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0019e-05 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.606538+0000", 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014364740000000001 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.606704+0000", 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016607300000000001 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.606870+0000", 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016643300000000001 2024-09-19T05:43:58.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.626640+0000", 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019769554000000002 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.824727+0000", 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19808663300000001 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.824792+0000", 2024-09-19T05:43:58.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5210999999999996e-05 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.824845+0000", 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2765e-05 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:204 3.26 3:6430d4f3:::10000000002.000000c9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.621455+0000", 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0453760719999998, 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26136332899999998, 2024-09-19T05:43:58.772 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 204 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621455+0000", 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621455+0000", 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.621458+0000", 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6490000000000001e-06 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638150+0000", 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016691620000000001 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638191+0000", 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1056999999999997e-05 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638218+0000", 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7064e-05 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638552+0000", 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033372099999999999 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.639010+0000", 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000458598 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.639458+0000", 2024-09-19T05:43:58.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00044719699999999998 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.726753+0000", 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087295090000000006 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882750+0000", 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15599700399999999 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882787+0000", 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7755e-05 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882818+0000", 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0574000000000002e-05 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:207 3.26 3:65689812:::10000000002.000000cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.638282+0000", 2024-09-19T05:43:58.782 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0285487870000001, 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24464840600000001, 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 207 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638282+0000", 2024-09-19T05:43:58.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638282+0000", 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.638286+0000", 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.561e-06 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651627+0000", 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013340250999999999 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651645+0000", 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7807000000000001e-05 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651655+0000", 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0705e-05 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.658901+0000", 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072462009999999999 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.659059+0000", 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015735599999999999 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.659257+0000", 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019826699999999999 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.729942+0000", 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.070684565000000005 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882855+0000", 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.152913733 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882898+0000", 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2595000000000001e-05 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882930+0000", 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2364999999999999e-05 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.788 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:211 3.26 3:65bb0d93:::10000000002.000000d0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.651679+0000", 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0151521140000002, 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.231370309, 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 211 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651679+0000", 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651679+0000", 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.651682+0000", 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1049999999999999e-06 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.661340+0000", 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0096578649999999999 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.661360+0000", 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0250000000000001e-05 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.661371+0000", 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0754000000000001e-05 2024-09-19T05:43:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.713738+0000", 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052367128999999998 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715955+0000", 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022169910000000002 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.716154+0000", 2024-09-19T05:43:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019889300000000001 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.738197+0000", 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022043618000000001 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882967+0000", 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14477016200000001 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.883018+0000", 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1097999999999998e-05 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.883049+0000", 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0443999999999999e-05 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:249 3.16 3:689177d4:::10000000002.000000f6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.685256+0000", 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.9815748810000002, 2024-09-19T05:43:58.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20163096899999999, 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 249 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.685256+0000", 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.685256+0000", 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.685259+0000", 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.146e-06 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687156+0000", 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00189657 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687174+0000", 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8909000000000001e-05 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.687188+0000", 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3309000000000001e-05 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.730187+0000", 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042999355000000003 2024-09-19T05:43:58.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.730356+0000", 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016865000000000001 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.730558+0000", 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020220500000000001 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.770736+0000", 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040177998999999999 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886787+0000", 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11605080399999999 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886844+0000", 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7083999999999998e-05 2024-09-19T05:43:58.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886887+0000", 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2938000000000001e-05 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:58.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:58.801 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.677+0000 7f98caffd640 1 -- 172.21.15.2:0/3438896303 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v55) ==== 101+0+0 (secure 0 0 0) 0x7f98d4150a90 con 0x7f98d415e620 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.688+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f98b406c2b0 msgr2=0x7f98b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.688+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f98b406c2b0 0x7f98b406e770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98c4034000 tx=0x7f98c402e000 comp rx=0 tx=0).stop 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.688+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d415e620 msgr2=0x7f98d415d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.688+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d415e620 0x7f98d415d420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00efc0 tx=0x7f98cc00c490 comp rx=0 tx=0).stop 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 shutdown_connections 2024-09-19T05:43:58.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f98b406c2b0 0x7f98b406e770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98d415e620 0x7f98d415d420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f98d4151970 0x7f98d415cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 --2- 172.21.15.2:0/3438896303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98d40ca850 0x7f98d41f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 >> 172.21.15.2:0/3438896303 conn(0x7f98d406bdf0 msgr2=0x7f98d4071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 shutdown_connections 2024-09-19T05:43:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:58.690+0000 7f98d8dbe640 1 -- 172.21.15.2:0/3438896303 wait complete. 2024-09-19T05:43:58.804 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:43:58.805 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:43:58.711+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:43:58.806 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:58.805+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:58.922 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:58.906+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:59.010 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:59.008+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:59.111 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:43:59.108+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:43:59.202 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:459 3.14 3:2878c2ad:::10000000002.000000c8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.857456+0000", 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.33956693199999999, 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.339597231, 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 459 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.857456+0000", 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.857456+0000", 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.857458+0000", 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1239999999999999e-06 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.861978+0000", 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045202360000000004 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.870353+0000", 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083748360000000001 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.870379+0000", 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6177999999999999e-05 2024-09-19T05:43:59.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.870561+0000", 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018186999999999999 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.870728+0000", 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016748500000000001 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.870988+0000", 2024-09-19T05:43:59.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025921000000000001 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117768+0000", 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24678082500000001 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:399 3.26 e55/20 3:6771a79d:::10000000002.0000008c:head v 55'12, mlcod=53'8)", 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.078374+0000", 2024-09-19T05:43:59.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.118648222, 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.118763431, 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.078374+0000", 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.078374+0000", 2024-09-19T05:43:59.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.078375+0000", 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2000000000000001e-07 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.080080+0000", 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017051270000000001 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.080100+0000", 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9817999999999999e-05 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.080114+0000", 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4562999999999999e-05 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127799+0000", 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047684703000000002 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127823+0000", 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3587e-05 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:331 3.30 3:0d2dbd17:::10000000002.00000048:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.44800952999999999, 2024-09-19T05:43:59.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44818338899999999, 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 331 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749016+0000", 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.754846+0000", 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058302459999999999 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756282+0000", 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001436295 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756297+0000", 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4921e-05 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756370+0000", 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2856999999999997e-05 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756495+0000", 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012511999999999999 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756711+0000", 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216233 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828851+0000", 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072139743000000006 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:464 3.11 3:898ed099:::10000000002.000000cd:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.876190+0000", 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.32083273699999998, 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32108990399999998, 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 464 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.876190+0000", 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.876190+0000", 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.876193+0000", 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.879169+0000", 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029768669999999998 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.879189+0000", 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9922999999999999e-05 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.879202+0000", 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2525e-05 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.177940+0000", 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29873776699999999 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178047+0000", 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107439 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178237+0000", 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019032399999999999 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:401 3.16 e55/20 3:6bf10b49:::10000000002.0000008e:head v 55'5, mlcod=53'2)", 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.080143+0000", 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.116879497, 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.117208853, 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.080143+0000", 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.080143+0000", 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.080144+0000", 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0130000000000001e-06 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.093683+0000", 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013539303000000001 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.093712+0000", 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8300999999999999e-05 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.093724+0000", 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2339e-05 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.144199+0000", 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050474968000000002 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.144229+0000", 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9884999999999999e-05 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.197311+0000", 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053082262999999998 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:431 3.26 e55/20 3:66bf65cf:::10000000002.000000ac:head v 55'13, mlcod=53'8)", 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.093756+0000", 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.103267054, 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10365400700000001, 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.093756+0000", 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.093756+0000", 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.093757+0000", 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2589999999999999e-06 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.097312+0000", 2024-09-19T05:43:59.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035553960000000002 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.097328+0000", 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5262000000000001e-05 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.097339+0000", 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1109e-05 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.159551+0000", 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.062212851999999999 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.159609+0000", 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7846999999999997e-05 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:346 3.11 3:8815798b:::10000000002.00000057:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:43:59.230 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.440695381, 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44114331200000001, 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 346 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:43:59.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756330+0000", 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759712+0000", 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033821889999999999 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759737+0000", 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5069e-05 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759752+0000", 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4372999999999999e-05 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759847+0000", 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5291999999999995e-05 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760098+0000", 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025112099999999998 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760422+0000", 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032433199999999998 2024-09-19T05:43:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855328+0000", 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094905274999999997 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:432 3.26 e55/20 3:66aef6e9:::10000000002.000000ad:head v 55'14, mlcod=53'8)", 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.097361+0000", 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.099661471000000001, 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10020802299999999, 2024-09-19T05:43:59.235 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.097361+0000", 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.097361+0000", 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.097362+0000", 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4900000000000001e-07 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.100990+0000", 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036279200000000002 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.101010+0000", 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0245e-05 2024-09-19T05:43:59.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.101024+0000", 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4219999999999999e-05 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185030+0000", 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.084006371999999996 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185054+0000", 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3961e-05 2024-09-19T05:43:59.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:457 3.26 e55/20 3:64ff7f1d:::10000000002.000000c6:head v 55'15, mlcod=53'8)", 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.101054+0000", 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.095969084999999996, 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.096574426000000005, 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.101054+0000", 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.101054+0000", 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.101054+0000", 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9800000000000002e-07 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.105550+0000", 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044950459999999999 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.105571+0000", 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1671999999999999e-05 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.105587+0000", 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5834000000000001e-05 2024-09-19T05:43:59.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.193091+0000", 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087504451999999996 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.193116+0000", 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4757999999999999e-05 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:477 3.14 3:2adf572d:::10000000002.000000da:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.894019+0000", 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.30300391799999998, 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30366818499999998, 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 477 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.894019+0000", 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.894019+0000", 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.894023+0000", 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2560000000000004e-06 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898256+0000", 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004233077 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898289+0000", 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3380000000000002e-05 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898302+0000", 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2318e-05 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.920391+0000", 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022089088999999999 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.920531+0000", 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014002400000000001 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.920719+0000", 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000187863 2024-09-19T05:43:59.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117789+0000", 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19707031899999999 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:460 3.26 e55/20)", 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.105625+0000", 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.091397939999999997, 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092137986000000005, 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.105625+0000", 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.105625+0000", 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.105626+0000", 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.598e-06 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.110414+0000", 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047880190000000001 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.110437+0000", 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2748999999999999e-05 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.110449+0000", 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.238e-05 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:463 3.26 e55/20)", 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.110482+0000", 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.086541032000000004, 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.087326613999999997, 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.110482+0000", 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.110482+0000", 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.110483+0000", 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.037e-06 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.113697+0000", 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032143459999999999 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.113715+0000", 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8076000000000001e-05 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.113724+0000", 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8289999999999997e-06 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:480 3.14 3:2b6a5f82:::10000000002.000000dd:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.899690+0000", 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.29733226200000001, 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29817837699999999, 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 480 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.899690+0000", 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.899690+0000", 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.899694+0000", 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.844e-06 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905395+0000", 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057008850000000002 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.907661+0000", 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002265488 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.907682+0000", 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1248999999999999e-05 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.954192+0000", 2024-09-19T05:43:59.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046509994999999998 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.954331+0000", 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013915299999999999 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.978900+0000", 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024569032000000001 2024-09-19T05:43:59.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.125908+0000", 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14700758899999999 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.258 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:361 3.30 3:0eedefab:::10000000002.00000066:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.42455986200000001, 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42549637099999998, 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 361 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772466+0000", 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8279999999999999e-06 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777394+0000", 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049281790000000004 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777413+0000", 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8913999999999999e-05 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777424+0000", 2024-09-19T05:43:59.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1192999999999999e-05 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799507+0000", 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022083230999999998 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799656+0000", 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014921300000000001 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799858+0000", 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020186999999999999 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830773+0000", 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030915148999999999 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:366 3.11 3:8b4c71c3:::10000000002.0000006b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.41957003999999998, 2024-09-19T05:43:59.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.420599008, 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 366 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777455+0000", 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7640000000000001e-06 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783373+0000", 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005917533 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783420+0000", 2024-09-19T05:43:59.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6709999999999998e-05 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783443+0000", 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3836999999999999e-05 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808013+0000", 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024569607 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808128+0000", 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011490299999999999 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808375+0000", 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024700299999999998 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937889+0000", 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951412500000001 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.268 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:368 3.14 3:280f29a3:::10000000002.0000006d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.41352466300000001, 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41464989099999999, 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 368 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783504+0000", 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7749999999999998e-06 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785589+0000", 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020852409999999998 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785606+0000", 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6609999999999999e-05 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785614+0000", 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4940000000000003e-06 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789369+0000", 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037552250000000001 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789485+0000", 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116019 2024-09-19T05:43:59.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789654+0000", 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016819299999999999 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828885+0000", 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039231901999999999 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:382 3.11 3:8818af48:::10000000002.0000007b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.40811674199999998, 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.409327465, 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 382 2024-09-19T05:43:59.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788911+0000", 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6609999999999999e-06 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796253+0000", 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073421060000000002 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796284+0000", 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1383000000000001e-05 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796299+0000", 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4793e-05 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832493+0000", 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036194021 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832612+0000", 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011929 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832778+0000", 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.066630+0000", 2024-09-19T05:43:59.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.233852226 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:497 3.30 3:0c580b6a:::10000000002.000000ee:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.924154+0000", 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.27286849899999999, 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27417829399999999, 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 497 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.924154+0000", 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.924154+0000", 2024-09-19T05:43:59.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.924157+0000", 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3069999999999998e-06 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.927680+0000", 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035227700000000002 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.927699+0000", 2024-09-19T05:43:59.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8870000000000001e-05 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.927731+0000", 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2190000000000002e-05 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.936830+0000", 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0090991149999999996 2024-09-19T05:43:59.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.936921+0000", 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0849999999999999e-05 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937076+0000", 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000155217 2024-09-19T05:43:59.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.058126+0000", 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.121049669 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:274 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.58482950899999997, 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58627646200000005, 2024-09-19T05:43:59.284 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 274 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612198+0000", 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.476e-06 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619545+0000", 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073473399999999999 2024-09-19T05:43:59.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619574+0000", 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8994000000000001e-05 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619585+0000", 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1103e-05 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619846+0000", 2024-09-19T05:43:59.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026047199999999998 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619979+0000", 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013353099999999999 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.620203+0000", 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000223945 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629556+0000", 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093532560000000008 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.289 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:503 3.9 3:91daa300:::10000000002.000000f4:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.931245+0000", 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.26577785100000001, 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26735597700000002, 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 503 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.931245+0000", 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.931245+0000", 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.931248+0000", 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2909999999999999e-06 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.936098+0000", 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048498819999999998 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.936112+0000", 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3549e-05 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.936121+0000", 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2399999999999996e-06 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175328+0000", 2024-09-19T05:43:59.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.239206839 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175421+0000", 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3449000000000006e-05 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175576+0000", 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000154888 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:413 3.12 e55/20 3:4b703321:::10000000002.0000009a:head v 55'12, mlcod=53'7)", 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.153113+0000", 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.043910012999999998, 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045605371999999998, 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.153113+0000", 2024-09-19T05:43:59.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.153113+0000", 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.153114+0000", 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1119999999999999e-06 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.163341+0000", 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010227004 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.163365+0000", 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3947999999999998e-05 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.163378+0000", 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3349999999999999e-05 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.183183+0000", 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019804954 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.183208+0000", 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4688000000000001e-05 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:403 3.9 3:91a8729a:::10000000002.00000090:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.298 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.38045169499999998, 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38222792100000003, 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 403 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816573+0000", 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.345e-06 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.820472+0000", 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038989060000000002 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827487+0000", 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070152189999999996 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827507+0000", 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9434000000000001e-05 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858059+0000", 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030552598 2024-09-19T05:43:59.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858252+0000", 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019281999999999999 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871029+0000", 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012776648999999999 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175256+0000", 2024-09-19T05:43:59.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30422741199999997 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:405 3.11 3:8834f03f:::10000000002.00000092:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.36948837699999998, 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37139984599999998, 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 405 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:43:59.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827537+0000", 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6620000000000001e-06 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830490+0000", 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029526209999999999 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830506+0000", 2024-09-19T05:43:59.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6891000000000001e-05 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830535+0000", 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8377e-05 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855366+0000", 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024831576000000001 2024-09-19T05:43:59.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855488+0000", 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012119499999999999 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855676+0000", 2024-09-19T05:43:59.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018832899999999999 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:293 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.54333082300000002, 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54534126900000002, 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 293 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653696+0000", 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.33e-06 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658892+0000", 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051954360000000003 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658913+0000", 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1705999999999999e-05 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658928+0000", 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4361e-05 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659326+0000", 2024-09-19T05:43:59.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000398253 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659446+0000", 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012050200000000001 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659655+0000", 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020868500000000001 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.734308+0000", 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074653061000000007 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:410 3.11 3:8988b152:::10000000002.00000097:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:43:59.314 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.36646620600000002, 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36857492200000003, 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 410 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830559+0000", 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835124+0000", 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045654520000000002 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835145+0000", 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0750999999999998e-05 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835179+0000", 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4013000000000001e-05 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937947+0000", 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102767884 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938050+0000", 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010270199999999999 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938225+0000", 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000175494 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:440 3.12 e55/20 3:4bdf515e:::10000000002.000000b5:head v 55'13, mlcod=53'7)", 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.163410+0000", 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033612802999999997, 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035847436000000003, 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.163410+0000", 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.163410+0000", 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.163411+0000", 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.06e-06 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.169078+0000", 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056670959999999999 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.169099+0000", 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0908999999999999e-05 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.169114+0000", 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5559000000000001e-05 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.190918+0000", 2024-09-19T05:43:59.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021803926000000001 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.191174+0000", 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000255784 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:416 3.9 3:93988fc9:::10000000002.0000009d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.835209+0000", 2024-09-19T05:43:59.323 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.36181327800000002, 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36420290700000002, 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 416 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835209+0000", 2024-09-19T05:43:59.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835209+0000", 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835213+0000", 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6389999999999998e-06 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839310+0000", 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040970670000000002 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839325+0000", 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5339999999999999e-05 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839340+0000", 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.464e-05 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.940381+0000", 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.101041269 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.940525+0000", 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000143559 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.940726+0000", 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000201447 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175298+0000", 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.234571536 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:511 3.9 3.45bc81c9 (undecoded) ondisk+write+known_if_redirected e55)", 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.980869+0000", 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21615336600000001, 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21864071600000001, 2024-09-19T05:43:59.328 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 511 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.980869+0000", 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.980869+0000", 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.980872+0000", 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2009999999999999e-06 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.983165+0000", 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022924859999999998 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.014730+0000", 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031565528000000002 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.014756+0000", 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5029e-05 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.331 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:419 3.30 3:0efeaece:::10000000002.000000a0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.839372+0000", 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.35765059100000002, 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36021957500000001, 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 419 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839372+0000", 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839372+0000", 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839377+0000", 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5689999999999998e-06 2024-09-19T05:43:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842914+0000", 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035371619999999999 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842931+0000", 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7053000000000001e-05 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842948+0000", 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7368e-05 2024-09-19T05:43:59.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.847233+0000", 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004284482 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.847351+0000", 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011814799999999999 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.847503+0000", 2024-09-19T05:43:59.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015198800000000001 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.056561+0000", 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20905768199999999 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4534.0:503 3.9 e55/20)", 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:59.185491+0000", 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011531316, 2024-09-19T05:43:59.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014212832, 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185491+0000", 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185491+0000", 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185493+0000", 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4470000000000001e-06 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185501+0000", 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1929999999999993e-06 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185514+0000", 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2952999999999999e-05 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.185523+0000", 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2129999999999997e-06 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:317 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:43:59.339 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.511673826, 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51441374200000001, 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 317 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:43:59.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685351+0000", 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3549999999999999e-06 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692638+0000", 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072869909999999996 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692666+0000", 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8010000000000001e-05 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692677+0000", 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0776e-05 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692800+0000", 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012327200000000001 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692936+0000", 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013544100000000001 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.693426+0000", 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00049003600000000001 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732131+0000", 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038705212000000003 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:420 3.11 3:88ab7984:::10000000002.000000a1:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.842977+0000", 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.35404545599999998, 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35689104599999999, 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 420 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842977+0000", 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842977+0000", 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842980+0000", 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0599999999999999e-06 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.846109+0000", 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031284529999999998 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.846150+0000", 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1029999999999998e-05 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.846159+0000", 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1970000000000002e-06 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.070208+0000", 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22404938799999999 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.070344+0000", 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013617800000000001 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.070506+0000", 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016109699999999999 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 32 2024-09-19T05:43:59.348 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:59.350 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:43:59.350 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:43:59.351 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 rewriters = 1150704.50 kB/sec 2024-09-19T05:43:59.351 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 rewriters = 1150454.68 kB/sec 2024-09-19T05:43:59.351 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 1150704.50 kB/sec 2024-09-19T05:43:59.351 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 1150704.50 kB/sec 2024-09-19T05:43:59.351 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 1150704.50 kB/sec 2024-09-19T05:43:59.351 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:43:59.351 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:43:59.348+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:43:59.450 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:59.448+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:59.499 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:59.499 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:43:59.499 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:43:59.499 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:43:59.500 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:59.508 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:43:59.551 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:43:59.549+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:11 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:43:59.620 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.3297872469999996, 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53368345399999995, 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287043+0000", 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294617+0000", 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075741280000000003 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294641+0000", 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3424000000000001e-05 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294656+0000", 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4679e-05 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294841+0000", 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185176 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295155+0000", 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031388199999999997 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295811+0000", 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065616700000000003 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.308016+0000", 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012205472 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820590+0000", 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51257333400000005 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820658+0000", 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8836000000000003e-05 2024-09-19T05:43:59.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820724+0000", 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5426999999999995e-05 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:18 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.2790762439999996, 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48181817700000001, 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 18 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337754+0000", 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346234+0000", 2024-09-19T05:43:59.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084795919999999993 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346309+0000", 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4950999999999994e-05 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346324+0000", 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5199e-05 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346456+0000", 2024-09-19T05:43:59.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013213799999999999 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346703+0000", 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024689500000000001 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346893+0000", 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019002400000000001 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382206+0000", 2024-09-19T05:43:59.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035313485999999998 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819443+0000", 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43723664899999998 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819516+0000", 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3021000000000004e-05 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819570+0000", 2024-09-19T05:43:59.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3696999999999998e-05 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:37 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.1995248419999998, 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.412251706, 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417306+0000", 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3790000000000002e-06 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421246+0000", 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039399869999999998 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421273+0000", 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6412999999999999e-05 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421284+0000", 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1856e-05 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421374+0000", 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9411000000000002e-05 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429518+0000", 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008143908 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429741+0000", 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022306100000000001 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.460095+0000", 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030354269999999999 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829462+0000", 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36936722900000002 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829510+0000", 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7698999999999999e-05 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829555+0000", 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4493000000000002e-05 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:53 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.1863574510000001, 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45610759200000001, 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 53 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430472+0000", 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1320000000000001e-06 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437581+0000", 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071082300000000001 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444289+0000", 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067080070000000002 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444322+0000", 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3034000000000003e-05 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445189+0000", 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086775100000000005 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445402+0000", 2024-09-19T05:43:59.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021237099999999999 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445599+0000", 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019734099999999999 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501230+0000", 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055630863000000003 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886462+0000", 2024-09-19T05:43:59.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38523158200000002 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886525+0000", 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3138000000000004e-05 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886578+0000", 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3143000000000003e-05 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:61 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.1594231180000003, 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.424715008, 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 61 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.644 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457409+0000", 2024-09-19T05:43:59.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461813+0000", 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044040060000000002 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461834+0000", 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0789e-05 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461876+0000", 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2268000000000003e-05 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461987+0000", 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110593 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467170+0000", 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051828849999999999 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467417+0000", 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024757199999999999 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490516+0000", 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023098802000000002 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.881965+0000", 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39144917200000001 2024-09-19T05:43:59.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882043+0000", 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7830000000000005e-05 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882120+0000", 2024-09-19T05:43:59.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6360999999999996e-05 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:267 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0202765890000001, 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52119119999999997, 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 267 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596555+0000", 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.596e-06 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597947+0000", 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001392517 2024-09-19T05:43:59.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597975+0000", 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7993000000000001e-05 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597989+0000", 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3927999999999999e-05 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598090+0000", 2024-09-19T05:43:59.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010057500000000001 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598219+0000", 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012952999999999999 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598408+0000", 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018851600000000001 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604394+0000", 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059861159999999997 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117572+0000", 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51317804499999997 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117624+0000", 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2126000000000002e-05 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117742+0000", 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011825800000000001 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:274 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.004634537, 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63191469899999997, 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 274 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:43:59.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612198+0000", 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.476e-06 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619545+0000", 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073473399999999999 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619574+0000", 2024-09-19T05:43:59.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8994000000000001e-05 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619585+0000", 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1103e-05 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619846+0000", 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026047199999999998 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619979+0000", 2024-09-19T05:43:59.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013353099999999999 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.620203+0000", 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000223945 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629556+0000", 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093532560000000008 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244028+0000", 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61447147300000005 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244065+0000", 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6968999999999998e-05 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244108+0000", 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3139999999999997e-05 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:293 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96313585099999999, 2024-09-19T05:43:59.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59051000200000003, 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 293 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653696+0000", 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.33e-06 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658892+0000", 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051954360000000003 2024-09-19T05:43:59.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658913+0000", 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1705999999999999e-05 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658928+0000", 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4361e-05 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659326+0000", 2024-09-19T05:43:59.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000398253 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659446+0000", 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012050200000000001 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659655+0000", 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020868500000000001 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.734308+0000", 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074653061000000007 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244147+0000", 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.50983838800000003 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244175+0000", 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8901999999999999e-05 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244202+0000", 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6378000000000001e-05 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:309 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.94740192000000001, 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.505676974, 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 309 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669428+0000", 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.593e-06 2024-09-19T05:43:59.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.676971+0000", 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075425839999999997 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677009+0000", 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8154000000000001e-05 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677039+0000", 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9788999999999999e-05 2024-09-19T05:43:59.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677223+0000", 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000184372 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677347+0000", 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123979 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677516+0000", 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016852499999999999 2024-09-19T05:43:59.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732124+0000", 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054607912000000002 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.174998+0000", 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44287454799999998 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175054+0000", 2024-09-19T05:43:59.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5476999999999998e-05 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175103+0000", 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9041e-05 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:313 3.2f 3:f7c72662:::10000000002.00000036:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.93970122199999995, 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47726204500000002, 2024-09-19T05:43:59.674 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 313 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677129+0000", 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3479999999999998e-06 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685213+0000", 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080837730000000007 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685237+0000", 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4311000000000002e-05 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685254+0000", 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7428999999999999e-05 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863622+0000", 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.178367422 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863724+0000", 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101938 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863861+0000", 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013715 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.152292+0000", 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.288431614 2024-09-19T05:43:59.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154270+0000", 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019771250000000001 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154330+0000", 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154388+0000", 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8717e-05 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:317 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.93147885399999997, 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56821626300000005, 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.681 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 317 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:43:59.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685351+0000", 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3549999999999999e-06 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692638+0000", 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072869909999999996 2024-09-19T05:43:59.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692666+0000", 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8010000000000001e-05 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692677+0000", 2024-09-19T05:43:59.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0776e-05 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692800+0000", 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012327200000000001 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692936+0000", 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013544100000000001 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.693426+0000", 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00049003600000000001 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732131+0000", 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038705212000000003 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253472+0000", 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52134096299999999 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253518+0000", 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5731e-05 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253565+0000", 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7475999999999997e-05 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:331 3.30 3:0d2dbd17:::10000000002.00000048:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86781455799999996, 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.495291814, 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 331 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749016+0000", 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.754846+0000", 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058302459999999999 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756282+0000", 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001436295 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756297+0000", 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4921e-05 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756370+0000", 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2856999999999997e-05 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756495+0000", 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012511999999999999 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756711+0000", 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216233 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828851+0000", 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072139743000000006 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244223+0000", 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41537228599999998 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244270+0000", 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244305+0000", 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4527999999999999e-05 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.693 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:346 3.11 3:8815798b:::10000000002.00000057:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86050040900000002, 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49739494099999998, 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 346 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:43:59.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756330+0000", 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759712+0000", 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033821889999999999 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759737+0000", 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5069e-05 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759752+0000", 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4372999999999999e-05 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759847+0000", 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5291999999999995e-05 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760098+0000", 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025112099999999998 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760422+0000", 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032433199999999998 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855328+0000", 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094905274999999997 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253622+0000", 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39829475600000003 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253679+0000", 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6047000000000001e-05 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253722+0000", 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3702999999999998e-05 2024-09-19T05:43:59.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:361 3.30 3:0eedefab:::10000000002.00000066:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.84436489000000003, 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47196730199999998, 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 361 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772466+0000", 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8279999999999999e-06 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777394+0000", 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049281790000000004 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777413+0000", 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8913999999999999e-05 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777424+0000", 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1192999999999999e-05 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799507+0000", 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022083230999999998 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799656+0000", 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014921300000000001 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799858+0000", 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020186999999999999 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830773+0000", 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030915148999999999 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244343+0000", 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41356947900000002 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244392+0000", 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9489e-05 2024-09-19T05:43:59.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244430+0000", 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7756999999999998e-05 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:366 3.11 3:8b4c71c3:::10000000002.0000006b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.83937506799999995, 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47662371999999997, 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 366 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777455+0000", 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7640000000000001e-06 2024-09-19T05:43:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783373+0000", 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005917533 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783420+0000", 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6709999999999998e-05 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783443+0000", 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3836999999999999e-05 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808013+0000", 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024569607 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808128+0000", 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011490299999999999 2024-09-19T05:43:59.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808375+0000", 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024700299999999998 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937889+0000", 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951412500000001 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253987+0000", 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31609763600000002 2024-09-19T05:43:59.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254037+0000", 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0383999999999997e-05 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254076+0000", 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9218000000000003e-05 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:368 3.14 3:280f29a3:::10000000002.0000006d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.83332969099999998, 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45471794900000001, 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 368 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783504+0000", 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7749999999999998e-06 2024-09-19T05:43:59.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785589+0000", 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020852409999999998 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785606+0000", 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6609999999999999e-05 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785614+0000", 2024-09-19T05:43:59.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4940000000000003e-06 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789369+0000", 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037552250000000001 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789485+0000", 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116019 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789654+0000", 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016819299999999999 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828885+0000", 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039231901999999999 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238106+0000", 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40922085699999999 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238164+0000", 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7373000000000001e-05 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238216+0000", 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2259999999999998e-05 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:382 3.11 3:8818af48:::10000000002.0000007b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.82792177, 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46533816500000003, 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 382 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788911+0000", 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6609999999999999e-06 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796253+0000", 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073421060000000002 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796284+0000", 2024-09-19T05:43:59.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1383000000000001e-05 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796299+0000", 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4793e-05 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832493+0000", 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036194021 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832612+0000", 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011929 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832778+0000", 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.066630+0000", 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.233852226 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254144+0000", 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18751385100000001 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254200+0000", 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6028e-05 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254244+0000", 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4373999999999999e-05 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:403 3.9 3:91a8729a:::10000000002.00000090:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80025672299999995, 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41822062799999998, 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 403 2024-09-19T05:43:59.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816573+0000", 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.345e-06 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.820472+0000", 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038989060000000002 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827487+0000", 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070152189999999996 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827507+0000", 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9434000000000001e-05 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858059+0000", 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030552598 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858252+0000", 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019281999999999999 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871029+0000", 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012776648999999999 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175256+0000", 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30422741199999997 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234683+0000", 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059426432000000001 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234738+0000", 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4923999999999998e-05 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234792+0000", 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3888999999999999e-05 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:405 3.11 3:8834f03f:::10000000002.00000092:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.730 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.789293405, 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42686835699999998, 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 405 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827537+0000", 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6620000000000001e-06 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830490+0000", 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029526209999999999 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830506+0000", 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6891000000000001e-05 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830535+0000", 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8377e-05 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855366+0000", 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024831576000000001 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855488+0000", 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012119499999999999 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855676+0000", 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018832899999999999 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233600+0000", 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37792366900000002 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254308+0000", 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020708684000000002 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254362+0000", 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3430000000000002e-05 2024-09-19T05:43:59.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254403+0000", 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0923000000000001e-05 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:410 3.11 3:8988b152:::10000000002.00000097:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.78627123399999999, 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42399440900000002, 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:43:59.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 410 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830559+0000", 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835124+0000", 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045654520000000002 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835145+0000", 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0750999999999998e-05 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835179+0000", 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4013000000000001e-05 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937947+0000", 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102767884 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938050+0000", 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010270199999999999 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938225+0000", 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000175494 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234859+0000", 2024-09-19T05:43:59.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.296633759 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254459+0000", 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019599979999999999 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254510+0000", 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1431000000000003e-05 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:43:59.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254551+0000", 2024-09-19T05:43:59.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0391e-05 2024-09-19T05:43:59.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:43:59.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:43:59.742 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:43:59.742 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:43:59.742 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:43:59.743 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-19T05:43:59.743 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-19T05:43:59.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:43:59.745 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:59.743+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:59.784 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:43:59.845 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:43:59.844+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:43:59.850 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:43:59.850 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:43:59.850 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:43:59.850 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:43:59.859 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:43:59.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.884+0000 7fc39d159640 1 Processor -- start 2024-09-19T05:43:59.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.884+0000 7fc39d159640 1 -- start start 2024-09-19T05:43:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39d159640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc398073560 0x7fc398157700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39d159640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398071a10 0x7fc398162080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39d159640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3980c3820 con 0x7fc398073940 2024-09-19T05:43:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39d159640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc3980c35d0 con 0x7fc398073560 2024-09-19T05:43:59.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39d159640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc398072a90 con 0x7fc398071a10 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc396d9d640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc398073560 0x7fc398157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39759e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398071a10 0x7fc398162080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39659c640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc398073940 0x7fc398151590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:41826/0 (socket says 172.21.15.2:41826) 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc396d9d640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc398073560 0x7fc398157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:38402/0 (socket says 172.21.15.2:38402) 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.885+0000 7fc39659c640 1 -- 172.21.15.2:0/2900762479 learned_addr learned my addr 172.21.15.2:0/2900762479 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.886+0000 7fc39759e640 1 -- 172.21.15.2:0/2900762479 >> v2:172.21.15.123:3300/0 conn(0x7fc398073560 msgr2=0x7fc398157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.886+0000 7fc39759e640 1 --2- 172.21.15.2:0/2900762479 >> v2:172.21.15.123:3300/0 conn(0x7fc398073560 0x7fc398157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.886+0000 7fc39759e640 1 -- 172.21.15.2:0/2900762479 >> v1:172.21.15.123:6789/0 conn(0x7fc398073940 legacy=0x7fc398151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.886+0000 7fc39759e640 1 -- 172.21.15.2:0/2900762479 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc398072440 con 0x7fc398071a10 2024-09-19T05:43:59.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.886+0000 7fc396d9d640 1 --2- 172.21.15.2:0/2900762479 >> v2:172.21.15.123:3300/0 conn(0x7fc398073560 0x7fc398157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:43:59.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.886+0000 7fc39759e640 1 --2- 172.21.15.2:0/2900762479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398071a10 0x7fc398162080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3980c3750 tx=0x7fc39002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2befca386800a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc395d9b640 1 -- 172.21.15.2:0/2900762479 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39002ae60 con 0x7fc398071a10 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc395d9b640 1 -- 172.21.15.2:0/2900762479 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc390007740 con 0x7fc398071a10 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc395d9b640 1 -- 172.21.15.2:0/2900762479 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390007b10 con 0x7fc398071a10 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 -- 172.21.15.2:0/2900762479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398071a10 msgr2=0x7fc398162080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 --2- 172.21.15.2:0/2900762479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398071a10 0x7fc398162080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3980c3750 tx=0x7fc39002a640 comp rx=0 tx=0).stop 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 -- 172.21.15.2:0/2900762479 shutdown_connections 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 --2- 172.21.15.2:0/2900762479 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398071a10 0x7fc398162080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 --2- 172.21.15.2:0/2900762479 >> v2:172.21.15.123:3300/0 conn(0x7fc398073560 0x7fc398157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 -- 172.21.15.2:0/2900762479 >> 172.21.15.2:0/2900762479 conn(0x7fc3980cc0d0 msgr2=0x7fc3980cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 -- 172.21.15.2:0/2900762479 shutdown_connections 2024-09-19T05:43:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7fc39d159640 1 -- 172.21.15.2:0/2900762479 wait complete. 2024-09-19T05:43:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7f73e12f1640 1 Processor -- start 2024-09-19T05:43:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.887+0000 7f73e12f1640 1 -- start start 2024-09-19T05:43:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.888+0000 7f73e12f1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c6b80 0x7f73dc0c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.888+0000 7f73e12f1640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73dc070cd0 con 0x7f73dc0c67a0 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.888+0000 7f73e12f1640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73dc070fb0 con 0x7f73dc0c82d0 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.888+0000 7f73e12f1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f73dc0708b0 con 0x7f73dc0c6b80 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.888+0000 7f73da7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c6b80 0x7f73dc0c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.888+0000 7f73daffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f73dc0c67a0 0x7f73dc0c72e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45052/0 (socket says 172.21.15.2:45052) 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.888+0000 7f73daffd640 1 -- 172.21.15.2:0/3542693470 learned_addr learned my addr 172.21.15.2:0/3542693470 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.889+0000 7f73da7fc640 1 -- 172.21.15.2:0/3542693470 >> v1:172.21.15.123:6789/0 conn(0x7f73dc0c82d0 legacy=0x7f73dc151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.889+0000 7f73d9ffb640 1 -- 172.21.15.2:0/3542693470 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3353785942 0 0) 0x7f73dc070cd0 con 0x7f73dc0c67a0 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.889+0000 7f73da7fc640 1 -- 172.21.15.2:0/3542693470 >> v1:172.21.15.2:6789/0 conn(0x7f73dc0c67a0 legacy=0x7f73dc0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.889+0000 7f73da7fc640 1 -- 172.21.15.2:0/3542693470 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc071290 con 0x7f73dc0c6b80 2024-09-19T05:43:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.889+0000 7fc39d159640 1 Processor -- start 2024-09-19T05:43:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.889+0000 7fc39d159640 1 -- start start 2024-09-19T05:43:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.889+0000 7f73da7fc640 1 --2- 172.21.15.2:0/3542693470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c6b80 0x7f73dc0c48f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009750 tx=0x7f73d002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=933c77d600cf886e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39d159640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398071a10 0x7fc3981f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39d159640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc398073560 0x7fc3981e1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39d159640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398073940 0x7fc398156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39d159640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc398157c40 con 0x7fc398071a10 2024-09-19T05:43:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39d159640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc39806f100 con 0x7fc398073560 2024-09-19T05:43:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39d159640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc398157dc0 con 0x7fc398073940 2024-09-19T05:43:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7f73d9ffb640 1 -- 172.21.15.2:0/3542693470 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0037070 con 0x7f73dc0c6b80 2024-09-19T05:43:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7f73d9ffb640 1 -- 172.21.15.2:0/3542693470 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73d002ae40 con 0x7f73dc0c6b80 2024-09-19T05:43:59.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7f73d9ffb640 1 -- 172.21.15.2:0/3542693470 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0032660 con 0x7f73dc0c6b80 2024-09-19T05:43:59.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc396d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398071a10 0x7fc3981f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc398073560 0x7fc3981e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc396d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398071a10 0x7fc3981f12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39146/0 (socket says 172.21.15.2:39146) 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39759e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398073940 0x7fc398156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc396d9d640 1 -- 172.21.15.2:0/4290743538 learned_addr learned my addr 172.21.15.2:0/4290743538 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39659c640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398073940 msgr2=0x7fc398156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39659c640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc398073940 0x7fc398156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39659c640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398071a10 msgr2=0x7fc3981f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39659c640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc398071a10 0x7fc3981f12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39659c640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3981e68c0 con 0x7fc398073560 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39659c640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc398073560 0x7fc3981e1690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc38c000f60 tx=0x7fc38c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c017070 con 0x7fc398073560 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38c005ce0 con 0x7fc398073560 2024-09-19T05:43:59.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3981e5bd0 con 0x7fc398073560 2024-09-19T05:43:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.890+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c007310 con 0x7fc398073560 2024-09-19T05:43:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3981e0e90 con 0x7fc398073560 2024-09-19T05:43:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7f73e12f1640 1 -- 172.21.15.2:0/3542693470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c6b80 msgr2=0x7f73dc0c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7f73e12f1640 1 --2- 172.21.15.2:0/3542693470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c6b80 0x7f73dc0c48f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009750 tx=0x7f73d002a130 comp rx=0 tx=0).stop 2024-09-19T05:43:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7f73e12f1640 1 -- 172.21.15.2:0/3542693470 shutdown_connections 2024-09-19T05:43:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7f73e12f1640 1 --2- 172.21.15.2:0/3542693470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c6b80 0x7f73dc0c48f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7f73e12f1640 1 -- 172.21.15.2:0/3542693470 >> 172.21.15.2:0/3542693470 conn(0x7f73dc0cd7e0 msgr2=0x7f73dc0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7f73e12f1640 1 -- 172.21.15.2:0/3542693470 shutdown_connections 2024-09-19T05:43:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7f73e12f1640 1 -- 172.21.15.2:0/3542693470 wait complete. 2024-09-19T05:43:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.891+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc39814eb50 con 0x7fc398073560 2024-09-19T05:43:59.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.892+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc38c01a070 con 0x7fc398073560 2024-09-19T05:43:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.892+0000 7f73e12f1640 1 Processor -- start 2024-09-19T05:43:59.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.892+0000 7f73e12f1640 1 -- start start 2024-09-19T05:43:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.892+0000 7fc37f7fe640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc38406c2e0 0x7fc38406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fc38c08a6c0 con 0x7fc398073560 2024-09-19T05:43:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v55) ==== 54+0+16 (secure 0 0 0) 0x7fc39814eb50 con 0x7fc398073560 2024-09-19T05:43:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc38406c2e0 0x7fc38406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc3981e6360 0x7fc39814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3980c3b90 con 0x7fc3981e6360 2024-09-19T05:43:59.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7fc39759e640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc3981e6360 0x7fc39814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73e12f1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c67a0 0x7f73dc146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73e12f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73dc0c6b80 0x7f73dc156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73e12f1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73dc0c82d0 0x7f73dc156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73e12f1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc0c4e30 con 0x7f73dc0c82d0 2024-09-19T05:43:59.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73e12f1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc070a80 con 0x7f73dc0c6b80 2024-09-19T05:43:59.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73e12f1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f73dc151a40 con 0x7f73dc0c67a0 2024-09-19T05:43:59.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc38406c2e0 0x7fc38406e7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc388009560 tx=0x7fc388009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73daffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c67a0 0x7f73dc146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73daffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c67a0 0x7f73dc146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:33376/0 (socket says 172.21.15.2:33376) 2024-09-19T05:43:59.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73da7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73dc0c6b80 0x7f73dc156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73daffd640 1 -- 172.21.15.2:0/2457205212 learned_addr learned my addr 172.21.15.2:0/2457205212 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:43:59.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.893+0000 7f73db7fe640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73dc0c82d0 0x7f73dc156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7fc39759e640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc3981e6360 0x7fc39814a9d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73daffd640 1 -- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73dc0c6b80 msgr2=0x7f73dc156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73daffd640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f73dc0c6b80 0x7f73dc156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73daffd640 1 -- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73dc0c82d0 msgr2=0x7f73dc156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73daffd640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73dc0c82d0 0x7f73dc156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73daffd640 1 -- 172.21.15.2:0/2457205212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc141460 con 0x7f73dc0c67a0 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73daffd640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c67a0 0x7f73dc146560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c4d0 tx=0x7f73cc00c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc018070 con 0x7f73dc0c67a0 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73cc0074e0 con 0x7f73dc0c67a0 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc155e60 con 0x7f73dc0c67a0 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004a20 con 0x7f73dc0c67a0 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc140ea0 con 0x7f73dc0c67a0 2024-09-19T05:43:59.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.894+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3980c3b90 con 0x7fc3981e6360 2024-09-19T05:43:59.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.895+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73dc0c48f0 con 0x7f73dc0c67a0 2024-09-19T05:43:59.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.896+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f73cc01b070 con 0x7f73dc0c67a0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73c37fe640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f73c806c2e0 0x7f73c806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f73cc08a750 con 0x7f73dc0c67a0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73c37fe640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f73c80744e0 0x7f73c8076900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73da7fc640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f73c806c2e0 0x7f73c806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73c8076fb0 con 0x7f73c80744e0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f73cc091050 con 0x7f73dc0c67a0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73db7fe640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f73c80744e0 0x7f73c8076900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73db7fe640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f73c80744e0 0x7f73c8076900 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.897+0000 7f73da7fc640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f73c806c2e0 0x7f73c806e7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73d0034000 tx=0x7f73d002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.898+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73c8076fb0 con 0x7f73c80744e0 2024-09-19T05:43:59.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.924+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc3980cc930 con 0x7fc3981e6360 2024-09-19T05:43:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.926+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3980cc930 con 0x7fc3981e6360 2024-09-19T05:43:59.928 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:43:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:43:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.926+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc3981e1060 0x7fc3981f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.926+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc39814f300 con 0x7fc3981e1060 2024-09-19T05:43:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.926+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc3981e1060 0x7fc3981f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.927+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc3981e1060 0x7fc3981f3d90 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.927+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc39814f300 con 0x7fc3981e1060 2024-09-19T05:43:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.931+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f73dc14e970 con 0x7f73c80744e0 2024-09-19T05:43:59.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.931+0000 7f73c37fe640 1 -- 172.21.15.2:0/2457205212 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f73dc14e970 con 0x7f73c80744e0 2024-09-19T05:43:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f73c80744e0 msgr2=0x7f73c8076900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f73c80744e0 0x7f73c8076900 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:43:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f73c806c2e0 msgr2=0x7f73c806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f73c806c2e0 0x7f73c806e7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73d0034000 tx=0x7f73d002e000 comp rx=0 tx=0).stop 2024-09-19T05:43:59.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c67a0 msgr2=0x7f73dc146560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:43:59.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 --2- 172.21.15.2:0/2457205212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f73dc0c67a0 0x7f73dc146560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c4d0 tx=0x7f73cc00c990 comp rx=0 tx=0).stop 2024-09-19T05:43:59.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73db7fe640 1 -- 172.21.15.2:0/2457205212 reap_dead start 2024-09-19T05:43:59.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 shutdown_connections 2024-09-19T05:43:59.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 >> 172.21.15.2:0/2457205212 conn(0x7f73dc0cd7e0 msgr2=0x7f73dc0cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:43:59.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 shutdown_connections 2024-09-19T05:43:59.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.932+0000 7f73e12f1640 1 -- 172.21.15.2:0/2457205212 wait complete. 2024-09-19T05:43:59.946 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-19T05:43:59.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.946+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc3980c5d50 con 0x7fc3981e1060 2024-09-19T05:43:59.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.948+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3980c5d50 con 0x7fc3981e1060 2024-09-19T05:43:59.950 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:43:59.951 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:43:59.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.948+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc3981f47e0 0x7fc398062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.948+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc3980c98f0 con 0x7fc3981f47e0 2024-09-19T05:43:59.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.948+0000 7fc39659c640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc3981f47e0 0x7fc398062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.949+0000 7fc39659c640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc3981f47e0 0x7fc398062630 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.952 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:43:59.950+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:43:59.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.950+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3980c98f0 con 0x7fc3981f47e0 2024-09-19T05:43:59.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.963+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc398062b70 con 0x7fc3981f47e0 2024-09-19T05:43:59.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.965+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc398062b70 con 0x7fc3981f47e0 2024-09-19T05:43:59.968 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:43:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:43:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.966+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc3981e6f80 0x7fc3981e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.966+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc3981e78c0 con 0x7fc3981e6f80 2024-09-19T05:43:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.966+0000 7fc39759e640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc3981e6f80 0x7fc3981e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.966+0000 7fc39759e640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc3981e6f80 0x7fc3981e7380 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.967+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3981e78c0 con 0x7fc3981e6f80 2024-09-19T05:43:59.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.981+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc3981e7e20 con 0x7fc3981e6f80 2024-09-19T05:43:59.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.982+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3981e7e20 con 0x7fc3981e6f80 2024-09-19T05:43:59.985 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:43:59.985 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:43:59.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.982+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc39802cb50 0x7fc39802cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:43:59.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.983+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc3981e78c0 con 0x7fc39802cb50 2024-09-19T05:43:59.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.983+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc39802cb50 0x7fc39802cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:43:59.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.983+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc39802cb50 0x7fc39802cf70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:43:59.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.984+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3981e78c0 con 0x7fc39802cb50 2024-09-19T05:44:00.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:43:59.999+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc3981e78c0 con 0x7fc39802cb50 2024-09-19T05:44:00.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.000+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3981e78c0 con 0x7fc39802cb50 2024-09-19T05:44:00.005 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:00.006 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:00.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.001+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc398205110 0x7fc398207500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.001+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc3981e7e80 con 0x7fc398205110 2024-09-19T05:44:00.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.001+0000 7fc39659c640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc398205110 0x7fc398207500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.001+0000 7fc39659c640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc398205110 0x7fc398207500 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.002+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3981e7e80 con 0x7fc398205110 2024-09-19T05:44:00.010 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.010 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.016+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc3981e7e80 con 0x7fc398205110 2024-09-19T05:44:00.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:00.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.018+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3981e7e80 con 0x7fc398205110 2024-09-19T05:44:00.020 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:00.021 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:00.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.018+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc398207f50 0x7fc39820a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.018+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc3981e7f50 con 0x7fc398207f50 2024-09-19T05:44:00.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.019+0000 7fc39759e640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc398207f50 0x7fc39820a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.019+0000 7fc39759e640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc398207f50 0x7fc39820a340 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.020+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3981e7f50 con 0x7fc398207f50 2024-09-19T05:44:00.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.034+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc3981e7f50 con 0x7fc398207f50 2024-09-19T05:44:00.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.035+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc3981e7f50 con 0x7fc398207f50 2024-09-19T05:44:00.038 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.036+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc39820ad90 0x7fc39820d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.036+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc39802dab0 con 0x7fc39820ad90 2024-09-19T05:44:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.036+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc39820ad90 0x7fc39820d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.036+0000 7fc396d9d640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc39820ad90 0x7fc39820d180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.037+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc39802dab0 con 0x7fc39820ad90 2024-09-19T05:44:00.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.049+0000 7efd0c6b7640 1 Processor -- start 2024-09-19T05:44:00.051 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.049+0000 7efd0c6b7640 1 -- start start 2024-09-19T05:44:00.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.050+0000 7efd0c6b7640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efd04154d60 0x7efd040c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.050+0000 7efd0c6b7640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd040739e0 con 0x7efd0406c040 2024-09-19T05:44:00.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.050+0000 7efd0c6b7640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd0415e160 con 0x7efd0415b280 2024-09-19T05:44:00.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.050+0000 7efd0c6b7640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efd0415ed70 con 0x7efd04154d60 2024-09-19T05:44:00.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.050+0000 7efd09c52640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7efd0406c040 0x7efd0415ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45064/0 (socket says 172.21.15.2:45064) 2024-09-19T05:44:00.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.050+0000 7efd09c52640 1 -- 172.21.15.2:0/777014741 learned_addr learned my addr 172.21.15.2:0/777014741 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:00.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7efd0a453640 1 --2- 172.21.15.2:0/777014741 >> v2:172.21.15.123:3300/0 conn(0x7efd04154d60 0x7efd040c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc39820d6c0 con 0x7fc39820ad90 2024-09-19T05:44:00.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7efd09451640 1 -- 172.21.15.2:0/777014741 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2225149766 0 0) 0x7efd040739e0 con 0x7efd0406c040 2024-09-19T05:44:00.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7efd09451640 1 -- 172.21.15.2:0/777014741 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcf0003520 con 0x7efd0406c040 2024-09-19T05:44:00.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7efd09451640 1 -- 172.21.15.2:0/777014741 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3623279148 0 0) 0x7efcf0003520 con 0x7efd0406c040 2024-09-19T05:44:00.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7efd0a453640 1 -- 172.21.15.2:0/777014741 >> v1:172.21.15.123:6789/0 conn(0x7efd0415b280 legacy=0x7efd041654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7efd0a453640 1 -- 172.21.15.2:0/777014741 >> v1:172.21.15.2:6789/0 conn(0x7efd0406c040 legacy=0x7efd0415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.051+0000 7efd0a453640 1 -- 172.21.15.2:0/777014741 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0415e440 con 0x7efd04154d60 2024-09-19T05:44:00.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.052+0000 7efd0a453640 1 --2- 172.21.15.2:0/777014741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd04154d60 0x7efd040c8060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd0415eef0 tx=0x7efcf802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc98d96ebb7f346 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.052+0000 7efd09451640 1 -- 172.21.15.2:0/777014741 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf8037070 con 0x7efd04154d60 2024-09-19T05:44:00.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.052+0000 7efd09451640 1 -- 172.21.15.2:0/777014741 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf802ac30 con 0x7efd04154d60 2024-09-19T05:44:00.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.052+0000 7efd09451640 1 -- 172.21.15.2:0/777014741 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00001670 con 0x7efd04154d60 2024-09-19T05:44:00.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.052+0000 7fc37f7fe640 1 -- 172.21.15.2:0/4290743538 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc39820d6c0 con 0x7fc39820ad90 2024-09-19T05:44:00.056 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:00.057 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:00.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.052+0000 7efd0c6b7640 1 -- 172.21.15.2:0/777014741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd04154d60 msgr2=0x7efd040c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.052+0000 7efd0c6b7640 1 --2- 172.21.15.2:0/777014741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd04154d60 0x7efd040c8060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd0415eef0 tx=0x7efcf802a1a0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7efd0c6b7640 1 -- 172.21.15.2:0/777014741 shutdown_connections 2024-09-19T05:44:00.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7efd0c6b7640 1 --2- 172.21.15.2:0/777014741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd04154d60 0x7efd040c8060 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7efd0c6b7640 1 -- 172.21.15.2:0/777014741 >> 172.21.15.2:0/777014741 conn(0x7efd0406f3d0 msgr2=0x7efd040707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7efd0c6b7640 1 -- 172.21.15.2:0/777014741 shutdown_connections 2024-09-19T05:44:00.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7efd0c6b7640 1 -- 172.21.15.2:0/777014741 wait complete. 2024-09-19T05:44:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc3981e6360 msgr2=0x7fc39814a9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fc3981e6360 0x7fc39814a9d0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc3981e1060 msgr2=0x7fc3981f3d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fc3981e1060 0x7fc3981f3d90 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc3981f47e0 msgr2=0x7fc398062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fc3981f47e0 0x7fc398062630 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc3981e6f80 msgr2=0x7fc3981e7380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fc3981e6f80 0x7fc3981e7380 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39759e640 1 -- 172.21.15.2:0/4290743538 reap_dead start 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc39802cb50 msgr2=0x7fc39802cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fc39802cb50 0x7fc39802cf70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc398205110 msgr2=0x7fc398207500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fc398205110 0x7fc398207500 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc398207f50 msgr2=0x7fc39820a340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fc398207f50 0x7fc39820a340 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc39820ad90 msgr2=0x7fc39820d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fc39820ad90 0x7fc39820d180 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39759e640 1 -- 172.21.15.2:0/4290743538 reap_dead start 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc38406c2e0 msgr2=0x7fc38406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc38406c2e0 0x7fc38406e7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc388009560 tx=0x7fc388009190 comp rx=0 tx=0).stop 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc398073560 msgr2=0x7fc3981e1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc398073560 0x7fc3981e1690 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc38c000f60 tx=0x7fc38c00c800 comp rx=0 tx=0).stop 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 shutdown_connections 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc38406c2e0 0x7fc38406e7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 --2- 172.21.15.2:0/4290743538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc398073560 0x7fc3981e1690 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 >> 172.21.15.2:0/4290743538 conn(0x7fc3980cc0d0 msgr2=0x7fc3980c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 shutdown_connections 2024-09-19T05:44:00.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.053+0000 7fc39d159640 1 -- 172.21.15.2:0/4290743538 wait complete. 2024-09-19T05:44:00.064 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:00.060+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:00.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.054+0000 7efd0c6b7640 1 Processor -- start 2024-09-19T05:44:00.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.054+0000 7efd0c6b7640 1 -- start start 2024-09-19T05:44:00.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0c6b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0406c040 0x7efd040cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0c6b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd04154d60 0x7efd040cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0c6b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd0415b280 0x7efd041577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0c6b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efd040c8b80 con 0x7efd04154d60 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0c6b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efd0415ea00 con 0x7efd0406c040 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0c6b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efd040c8d00 con 0x7efd0415b280 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd04154d60 0x7efd040cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd04154d60 0x7efd040cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39154/0 (socket says 172.21.15.2:39154) 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 -- 172.21.15.2:0/359793870 learned_addr learned my addr 172.21.15.2:0/359793870 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0a453640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0406c040 0x7efd040cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd0ac54640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd0415b280 0x7efd041577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 -- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd0415b280 msgr2=0x7efd041577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efd0415b280 0x7efd041577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 -- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0406c040 msgr2=0x7efd040cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd0406c040 0x7efd040cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.055+0000 7efd09c52640 1 -- 172.21.15.2:0/359793870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd041f1550 con 0x7efd04154d60 2024-09-19T05:44:00.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.056+0000 7efd09c52640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd04154d60 0x7efd040cb4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efd0000c750 tx=0x7efd0000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.056+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00017070 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.056+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd00005ad0 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.056+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd041509a0 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.056+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd00007070 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.056+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0414f870 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.057+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd040735c0 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.057+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efd0001a070 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.058+0000 7efcf6ffd640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efce806c2b0 0x7efce806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.058+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7efd0008af40 con 0x7efd04154d60 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.058+0000 7efcf6ffd640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efce80744b0 0x7efce80768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.058+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efce8076f80 con 0x7efce80744b0 2024-09-19T05:44:00.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.058+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efd0008b3d0 con 0x7efd04154d60 2024-09-19T05:44:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.058+0000 7efd0a453640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efce806c2b0 0x7efce806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.058+0000 7efd0ac54640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efce80744b0 0x7efce80768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.059+0000 7efd0ac54640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efce80744b0 0x7efce80768d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.059+0000 7efd0a453640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efce806c2b0 0x7efce806e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efcf8034000 tx=0x7efcf802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.059+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efce8076f80 con 0x7efce80744b0 2024-09-19T05:44:00.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7efd0414d6c0 con 0x7efce80744b0 2024-09-19T05:44:00.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efcf6ffd640 1 -- 172.21.15.2:0/359793870 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7efd0414d6c0 con 0x7efce80744b0 2024-09-19T05:44:00.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efce80744b0 msgr2=0x7efce80768d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efce80744b0 0x7efce80768d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efce806c2b0 msgr2=0x7efce806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efce806c2b0 0x7efce806e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efcf8034000 tx=0x7efcf802e000 comp rx=0 tx=0).stop 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd04154d60 msgr2=0x7efd040cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 --2- 172.21.15.2:0/359793870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd04154d60 0x7efd040cb4d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efd0000c750 tx=0x7efd0000cc10 comp rx=0 tx=0).stop 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0ac54640 1 -- 172.21.15.2:0/359793870 reap_dead start 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 shutdown_connections 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 >> 172.21.15.2:0/359793870 conn(0x7efd0406f3d0 msgr2=0x7efd0415d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 shutdown_connections 2024-09-19T05:44:00.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.076+0000 7efd0c6b7640 1 -- 172.21.15.2:0/359793870 wait complete. 2024-09-19T05:44:00.089 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-19T05:44:00.167 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.167 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.173 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:00.171+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:00.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:00.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.231+0000 7f438259f640 1 Processor -- start 2024-09-19T05:44:00.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.231+0000 7f438259f640 1 -- start start 2024-09-19T05:44:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.231+0000 7f438259f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.231+0000 7f438259f640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f437c0caf40 0x7f437c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f438259f640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437c0ccbc0 con 0x7f437c155000 2024-09-19T05:44:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f438259f640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f437c0cc600 con 0x7f437c0caf40 2024-09-19T05:44:00.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f438259f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f437c0cc8e0 con 0x7f437c154c20 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f437bfff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f4380b3c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f437c0caf40 0x7f437c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f437b7fe640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f437c155000 0x7f437c073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:41848/0 (socket says 172.21.15.2:41848) 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f437b7fe640 1 -- 172.21.15.2:0/3936909702 learned_addr learned my addr 172.21.15.2:0/3936909702 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f437affd640 1 -- 172.21.15.2:0/3936909702 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147696609 0 0) 0x7f437c0ccbc0 con 0x7f437c155000 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f437affd640 1 -- 172.21.15.2:0/3936909702 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4364003520 con 0x7f437c155000 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f4380b3c640 1 -- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 msgr2=0x7f437c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f4380b3c640 1 --2- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f4380b3c640 1 -- 172.21.15.2:0/3936909702 >> v1:172.21.15.123:6789/0 conn(0x7f437c155000 legacy=0x7f437c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f4380b3c640 1 -- 172.21.15.2:0/3936909702 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c0cbc60 con 0x7f437c0caf40 2024-09-19T05:44:00.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.232+0000 7f437bfff640 1 --2- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:00.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.233+0000 7f4380b3c640 1 --2- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c0caf40 0x7f437c15aae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4374009d90 tx=0x7f437402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e67b68f7c67ed79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.233+0000 7f437affd640 1 -- 172.21.15.2:0/3936909702 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374007740 con 0x7f437c0caf40 2024-09-19T05:44:00.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.233+0000 7f437affd640 1 -- 172.21.15.2:0/3936909702 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43740078e0 con 0x7f437c0caf40 2024-09-19T05:44:00.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.233+0000 7f437affd640 1 -- 172.21.15.2:0/3936909702 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374007cb0 con 0x7f437c0caf40 2024-09-19T05:44:00.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 -- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c0caf40 msgr2=0x7f437c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 --2- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c0caf40 0x7f437c15aae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4374009d90 tx=0x7f437402a660 comp rx=0 tx=0).stop 2024-09-19T05:44:00.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 -- 172.21.15.2:0/3936909702 shutdown_connections 2024-09-19T05:44:00.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 --2- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c0caf40 0x7f437c15aae0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 --2- 172.21.15.2:0/3936909702 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 -- 172.21.15.2:0/3936909702 >> 172.21.15.2:0/3936909702 conn(0x7f437c06d0e0 msgr2=0x7f437c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 -- 172.21.15.2:0/3936909702 shutdown_connections 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.234+0000 7f438259f640 1 -- 172.21.15.2:0/3936909702 wait complete. 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.235+0000 7f438259f640 1 Processor -- start 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.235+0000 7f438259f640 1 -- start start 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f438259f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f437c0caf40 0x7f437c151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f438259f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c1e2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f438259f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c155000 0x7f437c1e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f438259f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f437c073a50 con 0x7f437c0caf40 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f438259f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f437c0cb430 con 0x7f437c155000 2024-09-19T05:44:00.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f438259f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f437c073bd0 con 0x7f437c154c20 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f437c0caf40 0x7f437c151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f437c0caf40 0x7f437c151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39164/0 (socket says 172.21.15.2:39164) 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 -- 172.21.15.2:0/1118417444 learned_addr learned my addr 172.21.15.2:0/1118417444 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437b7fe640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c1e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f4380b3c640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c155000 0x7f437c1e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 -- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 msgr2=0x7f437c1e2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f437c154c20 0x7f437c1e2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 -- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c155000 msgr2=0x7f437c1e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f437c155000 0x7f437c1e2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 -- 172.21.15.2:0/1118417444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c0b5620 con 0x7f437c0caf40 2024-09-19T05:44:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.236+0000 7f437bfff640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f437c0caf40 0x7f437c151210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f436c00ec10 tx=0x7f436c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.237+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c017070 con 0x7f437c0caf40 2024-09-19T05:44:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.237+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f436c00cc30 con 0x7f437c0caf40 2024-09-19T05:44:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.237+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c0b5900 con 0x7f437c0caf40 2024-09-19T05:44:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.237+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c013470 con 0x7f437c0caf40 2024-09-19T05:44:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.237+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c0b5ec0 con 0x7f437c0caf40 2024-09-19T05:44:00.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.237+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f437c0c5770 con 0x7f437c0caf40 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.238+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f436c01a070 con 0x7f437c0caf40 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f4378ff9640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f436806c2e0 0x7f436806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f436c08bb50 con 0x7f437c0caf40 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f4378ff9640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f43680744e0 0x7f4368076900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f436c0093e0 con 0x7f43680744e0 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f436c08bfe0 con 0x7f437c0caf40 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f437b7fe640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f436806c2e0 0x7f436806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f4380b3c640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f43680744e0 0x7f4368076900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f437b7fe640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f436806c2e0 0x7f436806e7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f437c1e3c10 tx=0x7f4370009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.239+0000 7f4380b3c640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f43680744e0 0x7f4368076900 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.240+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f436c0093e0 con 0x7f43680744e0 2024-09-19T05:44:00.249 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:00.247+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:00.258 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.256+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f437c0b32c0 con 0x7f43680744e0 2024-09-19T05:44:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.256+0000 7f4378ff9640 1 -- 172.21.15.2:0/1118417444 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f437c0b32c0 con 0x7f43680744e0 2024-09-19T05:44:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f43680744e0 msgr2=0x7f4368076900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.259 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f43680744e0 0x7f4368076900 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f436806c2e0 msgr2=0x7f436806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f436806c2e0 0x7f436806e7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f437c1e3c10 tx=0x7f4370009040 comp rx=0 tx=0).stop 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f437c0caf40 msgr2=0x7f437c151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 --2- 172.21.15.2:0/1118417444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f437c0caf40 0x7f437c151210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f436c00ec10 tx=0x7f436c00c520 comp rx=0 tx=0).stop 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f4380b3c640 1 -- 172.21.15.2:0/1118417444 reap_dead start 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 shutdown_connections 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 >> 172.21.15.2:0/1118417444 conn(0x7f437c06d0e0 msgr2=0x7f437c153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 shutdown_connections 2024-09-19T05:44:00.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.257+0000 7f438259f640 1 -- 172.21.15.2:0/1118417444 wait complete. 2024-09-19T05:44:00.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-19T05:44:00.280 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.280 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:00.351 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:00.349+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:00.391 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.391 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.399 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:00.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.411+0000 7f2d8689b640 1 Processor -- start 2024-09-19T05:44:00.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.411+0000 7f2d8689b640 1 -- start start 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d8689b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2d8015b2a0 0x7f2d800718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d8689b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cbc50 0x7f2d80071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d8689b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d800ca910 con 0x7f2d800cb870 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d8689b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8006d930 con 0x7f2d8015b2a0 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d8689b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2d800caed0 con 0x7f2d800cbc50 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7f7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2d8015b2a0 0x7f2d800718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d84e38640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cbc50 0x7f2d80071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7ffff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2d800cb870 0x7f2d8015ab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:41856/0 (socket says 172.21.15.2:41856) 2024-09-19T05:44:00.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7ffff640 1 -- 172.21.15.2:0/2169895139 learned_addr learned my addr 172.21.15.2:0/2169895139 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:00.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7f7fe640 1 -- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cbc50 msgr2=0x7f2d80071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:00.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7f7fe640 1 --2- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cbc50 0x7f2d80071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7f7fe640 1 -- 172.21.15.2:0/2169895139 >> v1:172.21.15.123:6789/0 conn(0x7f2d800cb870 legacy=0x7f2d8015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7f7fe640 1 -- 172.21.15.2:0/2169895139 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d800cabf0 con 0x7f2d8015b2a0 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d7effd640 1 -- 172.21.15.2:0/2169895139 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2222126624 0 0) 0x7f2d800ca910 con 0x7f2d800cb870 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.412+0000 7f2d84e38640 1 --2- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cbc50 0x7f2d80071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.413+0000 7f2d7f7fe640 1 --2- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d8015b2a0 0x7f2d800718a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d740097a0 tx=0x7f2d7402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5270722c5eef2186 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.413+0000 7f2d7effd640 1 -- 172.21.15.2:0/2169895139 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74037070 con 0x7f2d8015b2a0 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.413+0000 7f2d7effd640 1 -- 172.21.15.2:0/2169895139 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d74032440 con 0x7f2d8015b2a0 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.413+0000 7f2d7effd640 1 -- 172.21.15.2:0/2169895139 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74032810 con 0x7f2d8015b2a0 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 -- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d8015b2a0 msgr2=0x7f2d800718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 --2- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d8015b2a0 0x7f2d800718a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d740097a0 tx=0x7f2d7402a180 comp rx=0 tx=0).stop 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 -- 172.21.15.2:0/2169895139 shutdown_connections 2024-09-19T05:44:00.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 --2- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cbc50 0x7f2d80071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 --2- 172.21.15.2:0/2169895139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d8015b2a0 0x7f2d800718a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 -- 172.21.15.2:0/2169895139 >> 172.21.15.2:0/2169895139 conn(0x7f2d8006bdf0 msgr2=0x7f2d8006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 -- 172.21.15.2:0/2169895139 shutdown_connections 2024-09-19T05:44:00.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.414+0000 7f2d8689b640 1 -- 172.21.15.2:0/2169895139 wait complete. 2024-09-19T05:44:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.415+0000 7f2d8689b640 1 Processor -- start 2024-09-19T05:44:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.415+0000 7f2d8689b640 1 -- start start 2024-09-19T05:44:00.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d8689b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cb870 0x7f2d80141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d8689b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d800cbc50 0x7f2d800c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d8689b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d8015b2a0 0x7f2d800ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d8689b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8006d130 con 0x7f2d8015b2a0 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d8689b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8006d4a0 con 0x7f2d800cbc50 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d8689b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2d800ca420 con 0x7f2d800cb870 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d8015b2a0 0x7f2d800ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d8015b2a0 0x7f2d800ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39174/0 (socket says 172.21.15.2:39174) 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 -- 172.21.15.2:0/2790551021 learned_addr learned my addr 172.21.15.2:0/2790551021 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d7ffff640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cb870 0x7f2d80141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d7f7fe640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d800cbc50 0x7f2d800c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 -- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cb870 msgr2=0x7f2d80141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d800cb870 0x7f2d80141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 -- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d800cbc50 msgr2=0x7f2d800c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d800cbc50 0x7f2d800c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.416+0000 7f2d84e38640 1 -- 172.21.15.2:0/2790551021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d80159860 con 0x7f2d8015b2a0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.417+0000 7f2d84e38640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d8015b2a0 0x7f2d800ce8b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009ea0 tx=0x7f2d78009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.417+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78018030 con 0x7f2d8015b2a0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.417+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7800cde0 con 0x7f2d8015b2a0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.417+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78003bb0 con 0x7f2d8015b2a0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.417+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d800cc580 con 0x7f2d8015b2a0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.417+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d80142f50 con 0x7f2d8015b2a0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.418+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d8015ab90 con 0x7f2d8015b2a0 2024-09-19T05:44:00.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.418+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2d7801b070 con 0x7f2d8015b2a0 2024-09-19T05:44:00.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.419+0000 7f2d7cff9640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2d6c06c2b0 0x7f2d6c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.419+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f2d7808bed0 con 0x7f2d8015b2a0 2024-09-19T05:44:00.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.419+0000 7f2d7cff9640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2d6c0744b0 0x7f2d6c0768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.419+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d7800be50 con 0x7f2d6c0744b0 2024-09-19T05:44:00.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.419+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2d7808e5d0 con 0x7f2d8015b2a0 2024-09-19T05:44:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.419+0000 7f2d7ffff640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2d6c06c2b0 0x7f2d6c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.419+0000 7f2d7f7fe640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2d6c0744b0 0x7f2d6c0768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.420+0000 7f2d7ffff640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2d6c06c2b0 0x7f2d6c06e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000f60 tx=0x7f2d70007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.420+0000 7f2d7f7fe640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2d6c0744b0 0x7f2d6c0768d0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.420+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d7800be50 con 0x7f2d6c0744b0 2024-09-19T05:44:00.438 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.436+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f2d80153000 con 0x7f2d6c0744b0 2024-09-19T05:44:00.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d7cff9640 1 -- 172.21.15.2:0/2790551021 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f2d80153000 con 0x7f2d6c0744b0 2024-09-19T05:44:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2d6c0744b0 msgr2=0x7f2d6c0768d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f2d6c0744b0 0x7f2d6c0768d0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2d6c06c2b0 msgr2=0x7f2d6c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2d6c06c2b0 0x7f2d6c06e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000f60 tx=0x7f2d70007b50 comp rx=0 tx=0).stop 2024-09-19T05:44:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d8015b2a0 msgr2=0x7f2d800ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.440 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 --2- 172.21.15.2:0/2790551021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d8015b2a0 0x7f2d800ce8b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009ea0 tx=0x7f2d78009f90 comp rx=0 tx=0).stop 2024-09-19T05:44:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d84e38640 1 -- 172.21.15.2:0/2790551021 reap_dead start 2024-09-19T05:44:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 shutdown_connections 2024-09-19T05:44:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 >> 172.21.15.2:0/2790551021 conn(0x7f2d8006bdf0 msgr2=0x7f2d80071670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 shutdown_connections 2024-09-19T05:44:00.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.437+0000 7f2d8689b640 1 -- 172.21.15.2:0/2790551021 wait complete. 2024-09-19T05:44:00.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-19T05:44:00.451 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:00.449+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:00.544 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.545 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.552 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:00.550+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:00.553 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.551+0000 7f24b2578640 1 Processor -- start 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.551+0000 7f24b2578640 1 -- start start 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.551+0000 7f24b2578640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24ac0c6760 0x7f24ac073e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.551+0000 7f24b2578640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24ac0c6b40 0x7f24ac0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24b2578640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24ac072410 con 0x7f24ac0b2bb0 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24b2578640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f24ac0c9760 con 0x7f24ac0c6760 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24b2578640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f24ac0b2720 con 0x7f24ac0c6b40 2024-09-19T05:44:00.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24abfff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24ac0c6760 0x7f24ac073e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24b0b15640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f24ac0b2bb0 0x7f24ac15c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45074/0 (socket says 172.21.15.2:45074) 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24b0b15640 1 -- 172.21.15.2:0/3526935143 learned_addr learned my addr 172.21.15.2:0/3526935143 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24ab7fe640 1 --2- 172.21.15.2:0/3526935143 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0c6b40 0x7f24ac0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24abfff640 1 -- 172.21.15.2:0/3526935143 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0c6b40 msgr2=0x7f24ac0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24abfff640 1 --2- 172.21.15.2:0/3526935143 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0c6b40 0x7f24ac0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24abfff640 1 -- 172.21.15.2:0/3526935143 >> v1:172.21.15.2:6789/0 conn(0x7f24ac0b2bb0 legacy=0x7f24ac15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.552+0000 7f24abfff640 1 -- 172.21.15.2:0/3526935143 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac072680 con 0x7f24ac0c6760 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24abfff640 1 --2- 172.21.15.2:0/3526935143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6760 0x7f24ac073e70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0cac90 tx=0x7f249c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3d88cd9acc0ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24aaffd640 1 -- 172.21.15.2:0/3526935143 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c037070 con 0x7f24ac0c6760 2024-09-19T05:44:00.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24aaffd640 1 -- 172.21.15.2:0/3526935143 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c032440 con 0x7f24ac0c6760 2024-09-19T05:44:00.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24aaffd640 1 -- 172.21.15.2:0/3526935143 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c032810 con 0x7f24ac0c6760 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24b2578640 1 -- 172.21.15.2:0/3526935143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6760 msgr2=0x7f24ac073e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24b2578640 1 --2- 172.21.15.2:0/3526935143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6760 0x7f24ac073e70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0cac90 tx=0x7f249c02a230 comp rx=0 tx=0).stop 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24b2578640 1 -- 172.21.15.2:0/3526935143 shutdown_connections 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24b2578640 1 --2- 172.21.15.2:0/3526935143 >> v2:172.21.15.123:3300/0 conn(0x7f24ac0c6b40 0x7f24ac0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24b2578640 1 --2- 172.21.15.2:0/3526935143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6760 0x7f24ac073e70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.553+0000 7f24b2578640 1 -- 172.21.15.2:0/3526935143 >> 172.21.15.2:0/3526935143 conn(0x7f24ac0cd470 msgr2=0x7f24ac0c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.554+0000 7f24b2578640 1 -- 172.21.15.2:0/3526935143 shutdown_connections 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.554+0000 7f24b2578640 1 -- 172.21.15.2:0/3526935143 wait complete. 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.555+0000 7f24b2578640 1 Processor -- start 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.555+0000 7f24b2578640 1 -- start start 2024-09-19T05:44:00.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.555+0000 7f24b2578640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0b2bb0 0x7f24ac0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.555+0000 7f24b2578640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0c6760 0x7f24ac0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b2578640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6b40 0x7f24ac0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b2578640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac0ca600 con 0x7f24ac0c6b40 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b2578640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac072960 con 0x7f24ac0b2bb0 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b2578640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f24ac158fc0 con 0x7f24ac0c6760 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6b40 0x7f24ac0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6b40 0x7f24ac0acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39198/0 (socket says 172.21.15.2:39198) 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 -- 172.21.15.2:0/3861741106 learned_addr learned my addr 172.21.15.2:0/3861741106 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24abfff640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0b2bb0 0x7f24ac0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24ab7fe640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0c6760 0x7f24ac0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 -- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0c6760 msgr2=0x7f24ac0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24ac0c6760 0x7f24ac0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 -- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0b2bb0 msgr2=0x7f24ac0ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24ac0b2bb0 0x7f24ac0ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 -- 172.21.15.2:0/3861741106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac1ee290 con 0x7f24ac0c6b40 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b0b15640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6b40 0x7f24ac0acda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24a400cbf0 tx=0x7f24a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4017070 con 0x7f24ac0c6b40 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a400ace0 con 0x7f24ac0c6b40 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac1ee530 con 0x7f24ac0c6b40 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4012600 con 0x7f24ac0c6b40 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.556+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac0abd80 con 0x7f24ac0c6b40 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.557+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24ac0c76e0 con 0x7f24ac0c6b40 2024-09-19T05:44:00.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.558+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f24a401a070 con 0x7f24ac0c6b40 2024-09-19T05:44:00.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.558+0000 7f24a8ff9640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f249806c2b0 0x7f249806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.558+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f24a408b0a0 con 0x7f24ac0c6b40 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.558+0000 7f24abfff640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f249806c2b0 0x7f249806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.559+0000 7f24a8ff9640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2498074450 0x7f2498076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.559+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2498076f20 con 0x7f2498074450 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.559+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f24a4092050 con 0x7f24ac0c6b40 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.559+0000 7f24ab7fe640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2498074450 0x7f2498076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.559+0000 7f24abfff640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f249806c2b0 0x7f249806e770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f249c034000 tx=0x7f249c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.559+0000 7f24ab7fe640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2498074450 0x7f2498076870 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.560+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2498076f20 con 0x7f2498074450 2024-09-19T05:44:00.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.576+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f24ac0cbaf0 con 0x7f2498074450 2024-09-19T05:44:00.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24a8ff9640 1 -- 172.21.15.2:0/3861741106 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f24ac0cbaf0 con 0x7f2498074450 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2498074450 msgr2=0x7f2498076870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2498074450 0x7f2498076870 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f249806c2b0 msgr2=0x7f249806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f249806c2b0 0x7f249806e770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f249c034000 tx=0x7f249c02e000 comp rx=0 tx=0).stop 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6b40 msgr2=0x7f24ac0acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 --2- 172.21.15.2:0/3861741106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24ac0c6b40 0x7f24ac0acda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f24a400cbf0 tx=0x7f24a400a430 comp rx=0 tx=0).stop 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b0b15640 1 -- 172.21.15.2:0/3861741106 reap_dead start 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 shutdown_connections 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 >> 172.21.15.2:0/3861741106 conn(0x7f24ac0cd470 msgr2=0x7f24ac0cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 shutdown_connections 2024-09-19T05:44:00.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.577+0000 7f24b2578640 1 -- 172.21.15.2:0/3861741106 wait complete. 2024-09-19T05:44:00.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-19T05:44:00.653 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:00.652+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:00.659 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.659 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:00.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.732+0000 7f52ea3e4640 1 Processor -- start 2024-09-19T05:44:00.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.732+0000 7f52ea3e4640 1 -- start start 2024-09-19T05:44:00.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52ea3e4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f52e4155000 0x7f52e40cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52ea3e4640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e40ccbc0 con 0x7f52e40caf40 2024-09-19T05:44:00.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52ea3e4640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e40cc600 con 0x7f52e4154c20 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52ea3e4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f52e40cc8e0 con 0x7f52e4155000 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e8981640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f52e40caf40 0x7f52e415aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45078/0 (socket says 172.21.15.2:45078) 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e8981640 1 -- 172.21.15.2:0/2802496058 learned_addr learned my addr 172.21.15.2:0/2802496058 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e37fe640 1 --2- 172.21.15.2:0/2802496058 >> v2:172.21.15.123:3300/0 conn(0x7f52e4155000 0x7f52e40cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e2ffd640 1 -- 172.21.15.2:0/2802496058 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970593737 0 0) 0x7f52e40ccbc0 con 0x7f52e40caf40 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e2ffd640 1 -- 172.21.15.2:0/2802496058 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52cc003520 con 0x7f52e40caf40 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e37fe640 1 -- 172.21.15.2:0/2802496058 >> v1:172.21.15.123:6789/0 conn(0x7f52e4154c20 legacy=0x7f52e4073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e37fe640 1 -- 172.21.15.2:0/2802496058 >> v1:172.21.15.2:6789/0 conn(0x7f52e40caf40 legacy=0x7f52e415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.733+0000 7f52e37fe640 1 -- 172.21.15.2:0/2802496058 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e40cbc60 con 0x7f52e4155000 2024-09-19T05:44:00.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.734+0000 7f52e37fe640 1 --2- 172.21.15.2:0/2802496058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 0x7f52e40cd9e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009790 tx=0x7f52d802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d247d8988c9e0dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.734+0000 7f52e2ffd640 1 -- 172.21.15.2:0/2802496058 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8037070 con 0x7f52e4155000 2024-09-19T05:44:00.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.734+0000 7f52e2ffd640 1 -- 172.21.15.2:0/2802496058 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8032440 con 0x7f52e4155000 2024-09-19T05:44:00.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.734+0000 7f52e2ffd640 1 -- 172.21.15.2:0/2802496058 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d80327f0 con 0x7f52e4155000 2024-09-19T05:44:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.735+0000 7f52ea3e4640 1 -- 172.21.15.2:0/2802496058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 msgr2=0x7f52e40cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.735+0000 7f52ea3e4640 1 --2- 172.21.15.2:0/2802496058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 0x7f52e40cd9e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009790 tx=0x7f52d802a210 comp rx=0 tx=0).stop 2024-09-19T05:44:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.735+0000 7f52ea3e4640 1 -- 172.21.15.2:0/2802496058 shutdown_connections 2024-09-19T05:44:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.735+0000 7f52ea3e4640 1 --2- 172.21.15.2:0/2802496058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 0x7f52e40cd9e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.735+0000 7f52ea3e4640 1 -- 172.21.15.2:0/2802496058 >> 172.21.15.2:0/2802496058 conn(0x7f52e406d0e0 msgr2=0x7f52e406d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.735+0000 7f52ea3e4640 1 -- 172.21.15.2:0/2802496058 shutdown_connections 2024-09-19T05:44:00.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.735+0000 7f52ea3e4640 1 -- 172.21.15.2:0/2802496058 wait complete. 2024-09-19T05:44:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.736+0000 7f52ea3e4640 1 Processor -- start 2024-09-19T05:44:00.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.736+0000 7f52ea3e4640 1 -- start start 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52ea3e4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e40caf40 0x7f52e4155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52ea3e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e4154c20 0x7f52e41e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52ea3e4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 0x7f52e41e3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52ea3e4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52e4073a50 con 0x7f52e4154c20 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52ea3e4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52e40cb430 con 0x7f52e4155000 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52ea3e4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f52e4073bd0 con 0x7f52e40caf40 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e4154c20 0x7f52e41e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e4154c20 0x7f52e41e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39208/0 (socket says 172.21.15.2:39208) 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 -- 172.21.15.2:0/1740659950 learned_addr learned my addr 172.21.15.2:0/1740659950 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e8981640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 0x7f52e41e3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e3fff640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e40caf40 0x7f52e4155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 -- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e40caf40 msgr2=0x7f52e4155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f52e40caf40 0x7f52e4155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 -- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 msgr2=0x7f52e41e3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52e4155000 0x7f52e41e3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 -- 172.21.15.2:0/1740659950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e41f3ec0 con 0x7f52e4154c20 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.737+0000 7f52e37fe640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e4154c20 0x7f52e41e34b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52d8000c00 tx=0x7f52d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.738+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8043070 con 0x7f52e4154c20 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.738+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8038040 con 0x7f52e4154c20 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.738+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8037050 con 0x7f52e4154c20 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.738+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e41f4160 con 0x7f52e4154c20 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.738+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e41f4720 con 0x7f52e4154c20 2024-09-19T05:44:00.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.738+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52e40c5770 con 0x7f52e4154c20 2024-09-19T05:44:00.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.739+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f52d8046070 con 0x7f52e4154c20 2024-09-19T05:44:00.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e0ff9640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f52d006c2b0 0x7f52d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7f52d80ab440 con 0x7f52e4154c20 2024-09-19T05:44:00.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e3fff640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f52d006c2b0 0x7f52d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e0ff9640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f52d0074450 0x7f52d0076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52d8032440 con 0x7f52d0074450 2024-09-19T05:44:00.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f52d80b2050 con 0x7f52e4154c20 2024-09-19T05:44:00.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e8981640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f52d0074450 0x7f52d0076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e3fff640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f52d006c2b0 0x7f52d006e770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000f60 tx=0x7f52d4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.740+0000 7f52e8981640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f52d0074450 0x7f52d0076870 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.741+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52d8032440 con 0x7f52d0074450 2024-09-19T05:44:00.754 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:00.752+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:00.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.757+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f52e40b5fd0 con 0x7f52d0074450 2024-09-19T05:44:00.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52e0ff9640 1 -- 172.21.15.2:0/1740659950 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f52e40b5fd0 con 0x7f52d0074450 2024-09-19T05:44:00.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f52d0074450 msgr2=0x7f52d0076870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52ea3e4640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f52d0074450 0x7f52d0076870 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f52d006c2b0 msgr2=0x7f52d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52ea3e4640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f52d006c2b0 0x7f52d006e770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52d4000f60 tx=0x7f52d4009040 comp rx=0 tx=0).stop 2024-09-19T05:44:00.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e4154c20 msgr2=0x7f52e41e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52ea3e4640 1 --2- 172.21.15.2:0/1740659950 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52e4154c20 0x7f52e41e34b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f52d8000c00 tx=0x7f52d80046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.758+0000 7f52e8981640 1 -- 172.21.15.2:0/1740659950 reap_dead start 2024-09-19T05:44:00.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.759+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 shutdown_connections 2024-09-19T05:44:00.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.759+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 >> 172.21.15.2:0/1740659950 conn(0x7f52e406d0e0 msgr2=0x7f52e4153dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.759+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 shutdown_connections 2024-09-19T05:44:00.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.759+0000 7f52ea3e4640 1 -- 172.21.15.2:0/1740659950 wait complete. 2024-09-19T05:44:00.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-19T05:44:00.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:44:00.812 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.812 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.821 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:00.865 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:00.864+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.913+0000 7ff8585cf640 1 Processor -- start 2024-09-19T05:44:00.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.913+0000 7fcda0dc4640 1 Processor -- start 2024-09-19T05:44:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.913+0000 7ff8585cf640 1 -- start start 2024-09-19T05:44:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.913+0000 7fcda0dc4640 1 -- start start 2024-09-19T05:44:00.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.914+0000 7ff8585cf640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff8500caf40 0x7ff8500cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.914+0000 7ff8585cf640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8500ccbc0 con 0x7ff850155000 2024-09-19T05:44:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.914+0000 7fcda0dc4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcd9c073890 0x7fcd9c0c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.914+0000 7ff8585cf640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8500cc600 con 0x7ff850154c20 2024-09-19T05:44:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.914+0000 7ff8585cf640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ff8500cc8e0 con 0x7ff8500caf40 2024-09-19T05:44:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcda0dc4640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9c0c4070 con 0x7fcd9c0724c0 2024-09-19T05:44:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcda0dc4640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd9c1579d0 con 0x7fcd9c073c70 2024-09-19T05:44:00.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcda0dc4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c157cb0 con 0x7fcd9c073890 2024-09-19T05:44:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff856b6c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff8500caf40 0x7ff8500cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff855b6a640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff850155000 0x7ff85015aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45086/0 (socket says 172.21.15.2:45086) 2024-09-19T05:44:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff855b6a640 1 -- 172.21.15.2:0/2343026192 learned_addr learned my addr 172.21.15.2:0/2343026192 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcd9ad9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fcd9c0724c0 0x7fcd9c0728f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45098/0 (socket says 172.21.15.2:45098) 2024-09-19T05:44:00.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcd9ad9d640 1 -- 172.21.15.2:0/2847505164 learned_addr learned my addr 172.21.15.2:0/2847505164 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcd9a59c640 1 --2- 172.21.15.2:0/2847505164 >> v2:172.21.15.123:3300/0 conn(0x7fcd9c073890 0x7fcd9c0c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff855369640 1 -- 172.21.15.2:0/2343026192 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1423622927 0 0) 0x7ff8500ccbc0 con 0x7ff850155000 2024-09-19T05:44:00.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff855369640 1 -- 172.21.15.2:0/2343026192 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff83c003520 con 0x7ff850155000 2024-09-19T05:44:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff855369640 1 -- 172.21.15.2:0/2343026192 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951916564 0 0) 0x7ff8500cc600 con 0x7ff850154c20 2024-09-19T05:44:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff855369640 1 -- 172.21.15.2:0/2343026192 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8500ccbc0 con 0x7ff850154c20 2024-09-19T05:44:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff856b6c640 1 -- 172.21.15.2:0/2343026192 >> v1:172.21.15.123:6789/0 conn(0x7ff850154c20 legacy=0x7ff850073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff856b6c640 1 -- 172.21.15.2:0/2343026192 >> v1:172.21.15.2:6789/0 conn(0x7ff850155000 legacy=0x7ff85015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcd9a59c640 1 -- 172.21.15.2:0/2847505164 >> v1:172.21.15.123:6789/0 conn(0x7fcd9c073c70 legacy=0x7fcd9c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcd9a59c640 1 -- 172.21.15.2:0/2847505164 >> v1:172.21.15.2:6789/0 conn(0x7fcd9c0724c0 legacy=0x7fcd9c0728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7ff856b6c640 1 -- 172.21.15.2:0/2343026192 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8500cbc60 con 0x7ff8500caf40 2024-09-19T05:44:00.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcd9959a640 1 -- 172.21.15.2:0/2847505164 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529953176 0 0) 0x7fcd9c0c4070 con 0x7fcd9c0724c0 2024-09-19T05:44:00.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.915+0000 7fcd9a59c640 1 -- 172.21.15.2:0/2847505164 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c157f90 con 0x7fcd9c073890 2024-09-19T05:44:00.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.916+0000 7ff856b6c640 1 --2- 172.21.15.2:0/2343026192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8500caf40 0x7ff8500cd9e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff848009d90 tx=0x7ff84802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febb6c4b8a9b448 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.916+0000 7fcd9a59c640 1 --2- 172.21.15.2:0/2847505164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c0c3950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009720 tx=0x7fcd8c02a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2693abb4b7b5ed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.916+0000 7ff855369640 1 -- 172.21.15.2:0/2343026192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848007740 con 0x7ff8500caf40 2024-09-19T05:44:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.916+0000 7ff855369640 1 -- 172.21.15.2:0/2343026192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8480078e0 con 0x7ff8500caf40 2024-09-19T05:44:00.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.916+0000 7ff855369640 1 -- 172.21.15.2:0/2343026192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848007c90 con 0x7ff8500caf40 2024-09-19T05:44:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcd9959a640 1 -- 172.21.15.2:0/2847505164 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c037070 con 0x7fcd9c073890 2024-09-19T05:44:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcd9959a640 1 -- 172.21.15.2:0/2847505164 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd8c032440 con 0x7fcd9c073890 2024-09-19T05:44:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcd9959a640 1 -- 172.21.15.2:0/2847505164 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c02ae60 con 0x7fcd9c073890 2024-09-19T05:44:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7ff8585cf640 1 -- 172.21.15.2:0/2343026192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8500caf40 msgr2=0x7ff8500cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7ff8585cf640 1 --2- 172.21.15.2:0/2343026192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8500caf40 0x7ff8500cd9e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff848009d90 tx=0x7ff84802a710 comp rx=0 tx=0).stop 2024-09-19T05:44:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcda0dc4640 1 -- 172.21.15.2:0/2847505164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 msgr2=0x7fcd9c0c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/2847505164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c0c3950 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009720 tx=0x7fcd8c02a150 comp rx=0 tx=0).stop 2024-09-19T05:44:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7ff8585cf640 1 -- 172.21.15.2:0/2343026192 shutdown_connections 2024-09-19T05:44:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7ff8585cf640 1 --2- 172.21.15.2:0/2343026192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8500caf40 0x7ff8500cd9e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7ff8585cf640 1 -- 172.21.15.2:0/2343026192 >> 172.21.15.2:0/2343026192 conn(0x7ff85006d0e0 msgr2=0x7ff85006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcda0dc4640 1 -- 172.21.15.2:0/2847505164 shutdown_connections 2024-09-19T05:44:00.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/2847505164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c0c3950 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcda0dc4640 1 -- 172.21.15.2:0/2847505164 >> 172.21.15.2:0/2847505164 conn(0x7fcd9c06bc10 msgr2=0x7fcd9c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7ff8585cf640 1 -- 172.21.15.2:0/2343026192 shutdown_connections 2024-09-19T05:44:00.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7ff8585cf640 1 -- 172.21.15.2:0/2343026192 wait complete. 2024-09-19T05:44:00.928 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:00.928 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:00.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcda0dc4640 1 -- 172.21.15.2:0/2847505164 shutdown_connections 2024-09-19T05:44:00.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.917+0000 7fcda0dc4640 1 -- 172.21.15.2:0/2847505164 wait complete. 2024-09-19T05:44:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.919+0000 7ff8585cf640 1 Processor -- start 2024-09-19T05:44:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.919+0000 7ff8585cf640 1 -- start start 2024-09-19T05:44:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.919+0000 7fcda0dc4640 1 Processor -- start 2024-09-19T05:44:00.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.919+0000 7fcda0dc4640 1 -- start start 2024-09-19T05:44:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.920+0000 7ff8585cf640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8500caf40 0x7ff8500c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.920+0000 7ff8585cf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850154c20 0x7ff8500c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff8585cf640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff850155000 0x7ff8500c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff8585cf640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff850073a50 con 0x7ff850155000 2024-09-19T05:44:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff8585cf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500cb430 con 0x7ff850154c20 2024-09-19T05:44:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff8585cf640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff850073bd0 con 0x7ff8500caf40 2024-09-19T05:44:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcda0dc4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd9c0724c0 0x7fcd9c1eec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcda0dc4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcda0dc4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9c073c70 0x7fcd9c1f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcda0dc4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c071bc0 con 0x7fcd9c0724c0 2024-09-19T05:44:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff856b6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff850155000 0x7ff8500c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff856b6c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff850155000 0x7ff8500c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39232/0 (socket says 172.21.15.2:39232) 2024-09-19T05:44:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff856b6c640 1 -- 172.21.15.2:0/2494320645 learned_addr learned my addr 172.21.15.2:0/2494320645 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcda0dc4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c158550 con 0x7fcd9c073890 2024-09-19T05:44:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcda0dc4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c151670 con 0x7fcd9c073c70 2024-09-19T05:44:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff855b6a640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850154c20 0x7ff8500c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff85636b640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8500caf40 0x7ff8500c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd9a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd9c0724c0 0x7fcd9c1eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd9ad9d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9c073c70 0x7fcd9c1f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:38526/0 (socket says 172.21.15.2:38526) 2024-09-19T05:44:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 -- 172.21.15.2:0/3451014118 learned_addr learned my addr 172.21.15.2:0/3451014118 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff855b6a640 1 -- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8500caf40 msgr2=0x7ff8500c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff855b6a640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff8500caf40 0x7ff8500c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff855b6a640 1 -- 172.21.15.2:0/2494320645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff850155000 msgr2=0x7ff8500c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff855b6a640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff850155000 0x7ff8500c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff855b6a640 1 -- 172.21.15.2:0/2494320645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8501f1880 con 0x7ff850154c20 2024-09-19T05:44:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff856b6c640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff850155000 0x7ff8500c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 -- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9c073c70 msgr2=0x7fcd9c1f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9c073c70 0x7fcd9c1f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 -- 172.21.15.2:0/3451014118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd9c0724c0 msgr2=0x7fcd9c1eec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd9c0724c0 0x7fcd9c1eec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 -- 172.21.15.2:0/3451014118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c1f2560 con 0x7fcd9c073890 2024-09-19T05:44:00.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd9a59c640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd9c0724c0 0x7fcd9c1eec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:00.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff855b6a640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850154c20 0x7ff8500c9490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff84000c3f0 tx=0x7ff84000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7fcd99d9b640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c156620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c570 tx=0x7fcd9000ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.921+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840017070 con 0x7ff850154c20 2024-09-19T05:44:00.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8400074e0 con 0x7ff850154c20 2024-09-19T05:44:00.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8501e3080 con 0x7ff850154c20 2024-09-19T05:44:00.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8400059e0 con 0x7ff850154c20 2024-09-19T05:44:00.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8501e3640 con 0x7ff850154c20 2024-09-19T05:44:00.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7fcd8affd640 1 -- 172.21.15.2:0/3451014118 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90017070 con 0x7fcd9c073890 2024-09-19T05:44:00.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7fcd8affd640 1 -- 172.21.15.2:0/3451014118 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd90005480 con 0x7fcd9c073890 2024-09-19T05:44:00.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c152940 con 0x7fcd9c073890 2024-09-19T05:44:00.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7fcd8affd640 1 -- 172.21.15.2:0/3451014118 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd900037f0 con 0x7fcd9c073890 2024-09-19T05:44:00.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c152f00 con 0x7fcd9c073890 2024-09-19T05:44:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.922+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8500c5770 con 0x7ff850154c20 2024-09-19T05:44:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.923+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c0c3950 con 0x7fcd9c073890 2024-09-19T05:44:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.923+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff84001a070 con 0x7ff850154c20 2024-09-19T05:44:00.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.923+0000 7fcd8affd640 1 -- 172.21.15.2:0/3451014118 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcd9001a070 con 0x7fcd9c073890 2024-09-19T05:44:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff846ffd640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff83406c2b0 0x7ff83406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7ff84008a6e0 con 0x7ff850154c20 2024-09-19T05:44:00.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7fcd8affd640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcd7c06c2b0 0x7fcd7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff846ffd640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff834074450 0x7ff834076870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff840010e50 con 0x7ff834074450 2024-09-19T05:44:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff840091050 con 0x7ff850154c20 2024-09-19T05:44:00.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff85636b640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff83406c2b0 0x7ff83406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7fcd9a59c640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcd7c06c2b0 0x7fcd7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7fcd8affd640 1 -- 172.21.15.2:0/3451014118 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fcd9008a8b0 con 0x7fcd9c073890 2024-09-19T05:44:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff856b6c640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff834074450 0x7ff834076870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.924+0000 7ff85636b640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff83406c2b0 0x7ff83406e770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff84c001010 tx=0x7ff84c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.925+0000 7ff856b6c640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff834074450 0x7ff834076870 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.925+0000 7fcd9a59c640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcd7c06c2b0 0x7fcd7c06e770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c034000 tx=0x7fcd8c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.925+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff840010e50 con 0x7ff834074450 2024-09-19T05:44:00.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.929+0000 7fcd8affd640 1 -- 172.21.15.2:0/3451014118 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcd90058930 con 0x7fcd9c073890 2024-09-19T05:44:00.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:00.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.943+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7ff8500cac00 con 0x7ff834074450 2024-09-19T05:44:00.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff846ffd640 1 -- 172.21.15.2:0/2494320645 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7ff8500cac00 con 0x7ff834074450 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff834074450 msgr2=0x7ff834076870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff834074450 0x7ff834076870 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff83406c2b0 msgr2=0x7ff83406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff83406c2b0 0x7ff83406e770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff84c001010 tx=0x7ff84c009040 comp rx=0 tx=0).stop 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850154c20 msgr2=0x7ff8500c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 --2- 172.21.15.2:0/2494320645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff850154c20 0x7ff8500c9490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff84000c3f0 tx=0x7ff84000c8b0 comp rx=0 tx=0).stop 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff856b6c640 1 -- 172.21.15.2:0/2494320645 reap_dead start 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 shutdown_connections 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 >> 172.21.15.2:0/2494320645 conn(0x7ff85006d0e0 msgr2=0x7ff850073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:00.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 shutdown_connections 2024-09-19T05:44:00.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:00.944+0000 7ff8585cf640 1 -- 172.21.15.2:0/2494320645 wait complete. 2024-09-19T05:44:00.956 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:00.954+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:00.957 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-19T05:44:01.056 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:01.055+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:01.088 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:01.088 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:01.097 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:01.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.098+0000 7fd1b31fd640 1 Processor -- start 2024-09-19T05:44:01.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.098+0000 7fd1b31fd640 1 -- start start 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.099+0000 7fd1b31fd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd1ac06d370 0x7fd1ac0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.099+0000 7fd1b31fd640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ac0719f0 con 0x7fd1ac15e2a0 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.099+0000 7fd1b31fd640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ac071cd0 con 0x7fd1ac06cf90 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.099+0000 7fd1b31fd640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ac0cb4f0 con 0x7fd1ac06d370 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.099+0000 7fd1abfff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd1ac06d370 0x7fd1ac0c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.099+0000 7fd1b179a640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd1ac15e2a0 0x7fd1ac0c9990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45106/0 (socket says 172.21.15.2:45106) 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.099+0000 7fd1b179a640 1 -- 172.21.15.2:0/3740043945 learned_addr learned my addr 172.21.15.2:0/3740043945 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.100+0000 7fd1abfff640 1 -- 172.21.15.2:0/3740043945 >> v1:172.21.15.123:6789/0 conn(0x7fd1ac06cf90 legacy=0x7fd1ac0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.100+0000 7fd1abfff640 1 -- 172.21.15.2:0/3740043945 >> v1:172.21.15.2:6789/0 conn(0x7fd1ac15e2a0 legacy=0x7fd1ac0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.100+0000 7fd1ab7fe640 1 -- 172.21.15.2:0/3740043945 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937678672 0 0) 0x7fd1ac0719f0 con 0x7fd1ac15e2a0 2024-09-19T05:44:01.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.100+0000 7fd1abfff640 1 -- 172.21.15.2:0/3740043945 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac071fb0 con 0x7fd1ac06d370 2024-09-19T05:44:01.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.100+0000 7fd1abfff640 1 --2- 172.21.15.2:0/3740043945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0c5350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd1980097a0 tx=0x7fd19802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d394d9f25491c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.100+0000 7fd1ab7fe640 1 -- 172.21.15.2:0/3740043945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198037070 con 0x7fd1ac06d370 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1ab7fe640 1 -- 172.21.15.2:0/3740043945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd198032440 con 0x7fd1ac06d370 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1ab7fe640 1 -- 172.21.15.2:0/3740043945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198032810 con 0x7fd1ac06d370 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3740043945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 msgr2=0x7fd1ac0c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1b31fd640 1 --2- 172.21.15.2:0/3740043945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0c5350 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd1980097a0 tx=0x7fd19802a230 comp rx=0 tx=0).stop 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3740043945 shutdown_connections 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1b31fd640 1 --2- 172.21.15.2:0/3740043945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0c5350 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3740043945 >> 172.21.15.2:0/3740043945 conn(0x7fd1ac06bc10 msgr2=0x7fd1ac06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3740043945 shutdown_connections 2024-09-19T05:44:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.101+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3740043945 wait complete. 2024-09-19T05:44:01.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.102+0000 7fd1b31fd640 1 Processor -- start 2024-09-19T05:44:01.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.102+0000 7fd1b31fd640 1 -- start start 2024-09-19T05:44:01.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b31fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac06cf90 0x7fd1ac1e5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b31fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b31fd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac15e2a0 0x7fd1ac1e6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b31fd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac0b0ae0 con 0x7fd1ac06d370 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b31fd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac072290 con 0x7fd1ac06cf90 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b31fd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ac0b0c60 con 0x7fd1ac15e2a0 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39268/0 (socket says 172.21.15.2:39268) 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1abfff640 1 -- 172.21.15.2:0/3905182186 learned_addr learned my addr 172.21.15.2:0/3905182186 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b0f99640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac06cf90 0x7fd1ac1e5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.103+0000 7fd1b179a640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac15e2a0 0x7fd1ac1e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1abfff640 1 -- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac15e2a0 msgr2=0x7fd1ac1e6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1abfff640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd1ac15e2a0 0x7fd1ac1e6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1abfff640 1 -- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac06cf90 msgr2=0x7fd1ac1e5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1abfff640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1ac06cf90 0x7fd1ac1e5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1abfff640 1 -- 172.21.15.2:0/3905182186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac1e4ac0 con 0x7fd1ac06d370 2024-09-19T05:44:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1abfff640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0b2e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1980097a0 tx=0x7fd1980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198037050 con 0x7fd1ac06d370 2024-09-19T05:44:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd198038040 con 0x7fd1ac06d370 2024-09-19T05:44:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac1e6300 con 0x7fd1ac06d370 2024-09-19T05:44:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198004490 con 0x7fd1ac06d370 2024-09-19T05:44:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.104+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac1e5020 con 0x7fd1ac06d370 2024-09-19T05:44:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.105+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1ac0c5350 con 0x7fd1ac06d370 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.106+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd198030900 con 0x7fd1ac06d370 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.106+0000 7fd1a97fa640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd19006c2b0 0x7fd19006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.106+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5709+0+0 (secure 0 0 0) 0x7fd1980abf10 con 0x7fd1ac06d370 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.106+0000 7fd1b0f99640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd19006c2b0 0x7fd19006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.106+0000 7fd1a97fa640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd1900744b0 0x7fd1900768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.106+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd19802be50 con 0x7fd1900744b0 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.106+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd1980ae610 con 0x7fd1ac06d370 2024-09-19T05:44:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.107+0000 7fd1b179a640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd1900744b0 0x7fd1900768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.107+0000 7fd1b0f99640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd19006c2b0 0x7fd19006e770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0000f60 tx=0x7fd1a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.107+0000 7fd1b179a640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd1900744b0 0x7fd1900768d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.107+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd19802be50 con 0x7fd1900744b0 2024-09-19T05:44:01.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.120+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fcd9c0c8600 con 0x7fcd9c073890 2024-09-19T05:44:01.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fd1ac0cc0b0 con 0x7fd1900744b0 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1a97fa640 1 -- 172.21.15.2:0/3905182186 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fd1ac0cc0b0 con 0x7fd1900744b0 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd1900744b0 msgr2=0x7fd1900768d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1b31fd640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd1900744b0 0x7fd1900768d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd19006c2b0 msgr2=0x7fd19006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1b31fd640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd19006c2b0 0x7fd19006e770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0000f60 tx=0x7fd1a0009040 comp rx=0 tx=0).stop 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 msgr2=0x7fd1ac0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.124+0000 7fd1b31fd640 1 --2- 172.21.15.2:0/3905182186 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1ac06d370 0x7fd1ac0b2e00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd1980097a0 tx=0x7fd1980046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.125+0000 7fd1b179a640 1 -- 172.21.15.2:0/3905182186 reap_dead start 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.125+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 shutdown_connections 2024-09-19T05:44:01.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.125+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 >> 172.21.15.2:0/3905182186 conn(0x7fd1ac06bc10 msgr2=0x7fd1ac0c40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:01.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.125+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 shutdown_connections 2024-09-19T05:44:01.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.125+0000 7fd1b31fd640 1 -- 172.21.15.2:0/3905182186 wait complete. 2024-09-19T05:44:01.180 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:01.179+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:01.240 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:01.240 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:01.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:01.290 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:01.288+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:01.359 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:01.358+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:01.391 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:01.391 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:01.400 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:01.461 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:01.459+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:01.510 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:01.510 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:01.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:01.562 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:01.560+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:01.663 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:01.661+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:01.667 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:01.667 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:01.675 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.679+0000 7fcd8affd640 1 -- 172.21.15.2:0/3451014118 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v56) ==== 87+0+0 (secure 0 0 0) 0x7fcd9005cc30 con 0x7fcd9c073890 2024-09-19T05:44:01.681 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:44:01.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcd7c06c2b0 msgr2=0x7fcd7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcd7c06c2b0 0x7fcd7c06e770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c034000 tx=0x7fcd8c02e000 comp rx=0 tx=0).stop 2024-09-19T05:44:01.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 msgr2=0x7fcd9c156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c156620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c570 tx=0x7fcd9000ca30 comp rx=0 tx=0).stop 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 shutdown_connections 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcd7c06c2b0 0x7fcd7c06e770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd9c073890 0x7fcd9c156620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcd9c073c70 0x7fcd9c1f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 --2- 172.21.15.2:0/3451014118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd9c0724c0 0x7fcd9c1eec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 >> 172.21.15.2:0/3451014118 conn(0x7fcd9c06bc10 msgr2=0x7fcd9c0c6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 shutdown_connections 2024-09-19T05:44:01.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:01.681+0000 7fcda0dc4640 1 -- 172.21.15.2:0/3451014118 wait complete. 2024-09-19T05:44:01.764 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:01.762+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:01.789 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:01.789 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:01.799 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:01.865 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:01.863+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:01.915 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:01.915 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:01.919 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:01.919 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 readers = 7157174.50 kB/sec 2024-09-19T05:44:01.919 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 readers = 7148635.55 kB/sec 2024-09-19T05:44:01.919 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 7157174.50 kB/sec 2024-09-19T05:44:01.919 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 7157174.50 kB/sec 2024-09-19T05:44:01.919 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 7157174.50 kB/sec 2024-09-19T05:44:01.919 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:44:01.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:01.965 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:01.964+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:02.066 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:02.064+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:02.072 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:02.074 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:02.074 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:02.082 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:02.167 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:02.165+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:02.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.170+0000 7f5c34d26640 1 Processor -- start 2024-09-19T05:44:02.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.170+0000 7f5c34d26640 1 -- start start 2024-09-19T05:44:02.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c34d26640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c3015aa90 0x7f5c3015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c34d26640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c3015ebe0 con 0x7f5c300cd8c0 2024-09-19T05:44:02.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c34d26640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c300b2e00 con 0x7f5c300cd4e0 2024-09-19T05:44:02.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c34d26640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f5c3015eec0 con 0x7f5c3015aa90 2024-09-19T05:44:02.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c2ed9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5c300cd8c0 0x7f5c300c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55542/0 (socket says 172.21.15.2:55542) 2024-09-19T05:44:02.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c2ed9d640 1 -- 172.21.15.2:0/2045336550 learned_addr learned my addr 172.21.15.2:0/2045336550 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:02.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c2dd9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c3015aa90 0x7f5c3015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c2dd9b640 1 -- 172.21.15.2:0/2045336550 >> v1:172.21.15.123:6789/0 conn(0x7f5c300cd4e0 legacy=0x7f5c300b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c2dd9b640 1 -- 172.21.15.2:0/2045336550 >> v1:172.21.15.2:6789/0 conn(0x7f5c300cd8c0 legacy=0x7f5c300c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.172+0000 7f5c2dd9b640 1 -- 172.21.15.2:0/2045336550 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c300b30e0 con 0x7f5c3015aa90 2024-09-19T05:44:02.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.173+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/2045336550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3015aa90 0x7f5c3015df30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5c240097a0 tx=0x7f5c2402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60419edb67710b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.173+0000 7f5c2d59a640 1 -- 172.21.15.2:0/2045336550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24037070 con 0x7f5c3015aa90 2024-09-19T05:44:02.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.173+0000 7f5c2d59a640 1 -- 172.21.15.2:0/2045336550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c24032440 con 0x7f5c3015aa90 2024-09-19T05:44:02.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.173+0000 7f5c2d59a640 1 -- 172.21.15.2:0/2045336550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24032810 con 0x7f5c3015aa90 2024-09-19T05:44:02.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.174+0000 7f5c34d26640 1 -- 172.21.15.2:0/2045336550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3015aa90 msgr2=0x7f5c3015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.174+0000 7f5c34d26640 1 --2- 172.21.15.2:0/2045336550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3015aa90 0x7f5c3015df30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5c240097a0 tx=0x7f5c2402a230 comp rx=0 tx=0).stop 2024-09-19T05:44:02.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.174+0000 7f5c34d26640 1 -- 172.21.15.2:0/2045336550 shutdown_connections 2024-09-19T05:44:02.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.174+0000 7f5c34d26640 1 --2- 172.21.15.2:0/2045336550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3015aa90 0x7f5c3015df30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.174+0000 7f5c34d26640 1 -- 172.21.15.2:0/2045336550 >> 172.21.15.2:0/2045336550 conn(0x7f5c30067b30 msgr2=0x7f5c30067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:02.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.174+0000 7f5c34d26640 1 -- 172.21.15.2:0/2045336550 shutdown_connections 2024-09-19T05:44:02.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.174+0000 7f5c34d26640 1 -- 172.21.15.2:0/2045336550 wait complete. 2024-09-19T05:44:02.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.176+0000 7f5c34d26640 1 Processor -- start 2024-09-19T05:44:02.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.176+0000 7f5c34d26640 1 -- start start 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c34d26640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c300cd4e0 0x7f5c301f1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c34d26640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c300cd8c0 0x7f5c301e3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c34d26640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3015aa90 0x7f5c300c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c34d26640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c300b42d0 con 0x7f5c300cd8c0 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c34d26640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c300b36a0 con 0x7f5c300cd4e0 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c34d26640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5c300b4450 con 0x7f5c3015aa90 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c300cd8c0 0x7f5c301e3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c300cd8c0 0x7f5c301e3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55096/0 (socket says 172.21.15.2:55096) 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c300cd4e0 0x7f5c301f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 -- 172.21.15.2:0/708945691 learned_addr learned my addr 172.21.15.2:0/708945691 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2ed9d640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3015aa90 0x7f5c300c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3015aa90 msgr2=0x7f5c300c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3015aa90 0x7f5c300c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c300cd4e0 msgr2=0x7f5c301f1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c300cd4e0 0x7f5c301f1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c300c8f70 con 0x7f5c300cd8c0 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c300cd4e0 0x7f5c301f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.177+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c300cd8c0 0x7f5c301e3d70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5c240098c0 tx=0x7f5c240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.178+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24037040 con 0x7f5c300cd8c0 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.178+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c24042070 con 0x7f5c300cd8c0 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.178+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c301595f0 con 0x7f5c300cd8c0 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.178+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24030bd0 con 0x7f5c300cd8c0 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.178+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c301f0790 con 0x7f5c300cd8c0 2024-09-19T05:44:02.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.179+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5c300b3bc0 con 0x7f5c300cd8c0 2024-09-19T05:44:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.180+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5c24047460 con 0x7f5c300cd8c0 2024-09-19T05:44:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.180+0000 7f5c1effd640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1006c2e0 0x7f5c1006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.180+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f5c240abff0 con 0x7f5c300cd8c0 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.181+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v56) ==== 54+0+16 (secure 0 0 0) 0x7f5c24032e20 con 0x7f5c300cd8c0 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.181+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1006c2e0 0x7f5c1006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.181+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5c301f09d0 0x7f5c300c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.181+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c30067f10 con 0x7f5c301f09d0 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.181+0000 7f5c2ed9d640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5c301f09d0 0x7f5c300c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.181+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1006c2e0 0x7f5c1006e7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c20001b70 tx=0x7f5c20009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.181+0000 7f5c2ed9d640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5c301f09d0 0x7f5c300c6710 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.182+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c30067f10 con 0x7f5c301f09d0 2024-09-19T05:44:02.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.196+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c301e6d80 con 0x7f5c301f09d0 2024-09-19T05:44:02.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.198+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c301e6d80 con 0x7f5c301f09d0 2024-09-19T05:44:02.200 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:02.201 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:02.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.198+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5c3015a680 0x7f5c30156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.198+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5c300c7cd0 con 0x7f5c3015a680 2024-09-19T05:44:02.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.198+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5c3015a680 0x7f5c30156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.198+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5c3015a680 0x7f5c30156e00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.199+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c300c7cd0 con 0x7f5c3015a680 2024-09-19T05:44:02.216 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.214+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c30157340 con 0x7f5c3015a680 2024-09-19T05:44:02.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.217+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c30157340 con 0x7f5c3015a680 2024-09-19T05:44:02.219 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:02.220 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:02.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.217+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5c301f4100 0x7f5c301f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.217+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5c30157910 con 0x7f5c301f4100 2024-09-19T05:44:02.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.217+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5c301f4100 0x7f5c301f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.217+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5c301f4100 0x7f5c301f4520 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.218+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c30157910 con 0x7f5c301f4100 2024-09-19T05:44:02.233 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.232+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c30062900 con 0x7f5c301f4100 2024-09-19T05:44:02.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.233+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c30062900 con 0x7f5c301f4100 2024-09-19T05:44:02.236 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:02.237 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:02.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.233+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5c300cb2c0 0x7f5c300cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.233+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5c30062880 con 0x7f5c300cb2c0 2024-09-19T05:44:02.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.234+0000 7f5c2ed9d640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5c300cb2c0 0x7f5c300cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.234+0000 7f5c2ed9d640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5c300cb2c0 0x7f5c300cb6c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.234+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c30062880 con 0x7f5c300cb2c0 2024-09-19T05:44:02.238 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:02.238 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:02.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:02.238 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:02.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.356134780999994, 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:44:02.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:44:02.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:7 3.2a 3:557f550e:::10000000002.00000004:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.9526864760000002, 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27494626900000002, 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 7 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282495+0000", 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4609999999999998e-06 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286920+0000", 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044248229999999996 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286972+0000", 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1659000000000001e-05 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286986+0000", 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287117+0000", 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131328 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287358+0000", 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024141700000000001 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287543+0000", 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018436700000000001 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299025+0000", 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011482327000000001 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557305+0000", 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25828001099999998 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557360+0000", 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5074000000000002e-05 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557439+0000", 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9018999999999997e-05 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:92 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.8195384580000002, 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37051118500000002, 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:02.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415644+0000", 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.422983+0000", 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073390779999999997 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423020+0000", 2024-09-19T05:44:02.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7660000000000002e-05 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423036+0000", 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5359999999999999e-05 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423261+0000", 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225217 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423458+0000", 2024-09-19T05:44:02.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196555 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423737+0000", 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027922100000000002 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.435447+0000", 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011709855999999999 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.785961+0000", 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35051446400000003 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786086+0000", 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012537599999999999 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786152+0000", 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691000000000005e-05 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:104 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.8053121359999995, 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28556600799999998, 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429870+0000", 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-06 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446619+0000", 2024-09-19T05:44:02.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016749447000000001 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446650+0000", 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0218000000000001e-05 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446666+0000", 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6920999999999999e-05 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448857+0000", 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021902699999999998 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449011+0000", 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015441700000000001 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449168+0000", 2024-09-19T05:44:02.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156527 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489828+0000", 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040659908000000002 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715285+0000", 2024-09-19T05:44:02.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22545759500000001 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715345+0000", 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0269999999999997e-05 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715433+0000", 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7814999999999999e-05 2024-09-19T05:44:02.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:108 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7884611190000008, 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26890963299999998, 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446721+0000", 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8250000000000001e-06 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451943+0000", 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052222140000000002 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451961+0000", 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8117e-05 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451970+0000", 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7420000000000003e-06 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473034+0000", 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021064282 2024-09-19T05:44:02.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473242+0000", 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020716699999999999 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473446+0000", 2024-09-19T05:44:02.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204396 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489832+0000", 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016385994000000001 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715506+0000", 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22567427700000001 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715577+0000", 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0548000000000003e-05 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715628+0000", 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070999999999999e-05 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:139 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7716497130000004, 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.328123891, 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463532+0000", 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0339999999999999e-06 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468331+0000", 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047995099999999999 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468365+0000", 2024-09-19T05:44:02.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3667000000000001e-05 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468382+0000", 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6651e-05 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468915+0000", 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053293299999999997 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469199+0000", 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028406900000000002 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469482+0000", 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028390000000000002 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.485026+0000", 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015543509 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791538+0000", 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30651197400000002 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791602+0000", 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3780000000000003e-05 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791654+0000", 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1863999999999997e-05 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:148 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7531996789999997, 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34695000399999998, 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 148 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.280 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481982+0000", 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6309999999999999e-06 2024-09-19T05:44:02.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489765+0000", 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077821310000000003 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489778+0000", 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3604999999999999e-05 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489787+0000", 2024-09-19T05:44:02.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489875+0000", 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8039000000000002e-05 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490101+0000", 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022636299999999999 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490426+0000", 2024-09-19T05:44:02.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032453799999999997 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.506325+0000", 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015899149000000001 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828797+0000", 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32247239900000002 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828865+0000", 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7521999999999995e-05 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828930+0000", 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4795999999999994e-05 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:156 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7453444719999993, 2024-09-19T05:44:02.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26935146300000001, 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489837+0000", 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2689999999999998e-06 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495152+0000", 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053150209999999996 2024-09-19T05:44:02.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495356+0000", 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203537 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495373+0000", 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496071+0000", 2024-09-19T05:44:02.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069810200000000003 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496246+0000", 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017547299999999999 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496469+0000", 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022283200000000001 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.540840+0000", 2024-09-19T05:44:02.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044371318999999999 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759082+0000", 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21824185500000001 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759137+0000", 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645000000000003e-05 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759186+0000", 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9499000000000002e-05 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:185 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7326646280000002, 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32688410400000001, 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.291 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 185 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502518+0000", 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.509973+0000", 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074553919999999999 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510031+0000", 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8356999999999997e-05 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510046+0000", 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932e-05 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510135+0000", 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8548000000000007e-05 2024-09-19T05:44:02.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510886+0000", 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075159700000000001 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511079+0000", 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019261500000000001 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532794+0000", 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021714989000000001 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829312+0000", 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29651747699999997 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829360+0000", 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8396000000000002e-05 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829399+0000", 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8961999999999997e-05 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:264 3.1c 3:3ae0713d:::10000000002.00000005:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:02.297 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6427459180000001, 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29060550499999999, 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 264 2024-09-19T05:44:02.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592436+0000", 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5629999999999999e-06 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595027+0000", 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025912600000000002 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595055+0000", 2024-09-19T05:44:02.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8164999999999999e-05 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595069+0000", 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3953e-05 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595156+0000", 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6981e-05 2024-09-19T05:44:02.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595260+0000", 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010363 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595417+0000", 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015732100000000001 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.605722+0000", 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010305099999999999 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882929+0000", 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27720663499999998 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882988+0000", 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9237000000000002e-05 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.883039+0000", 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0659999999999999e-05 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:315 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.603785598, 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55644120600000002, 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 315 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:02.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631396+0000", 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5490000000000001e-06 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636257+0000", 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048607859999999998 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636331+0000", 2024-09-19T05:44:02.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3992000000000006e-05 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636355+0000", 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3813999999999999e-05 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636445+0000", 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0180999999999996e-05 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636569+0000", 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123991 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636849+0000", 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000279664 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659414+0000", 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022565053000000002 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187715+0000", 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52830067000000003 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187783+0000", 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8509000000000004e-05 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187835+0000", 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1996999999999997e-05 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:347 3.2a 3:5747858d:::10000000002.00000058:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5957127719999997, 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48789592399999998, 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 347 2024-09-19T05:44:02.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639470+0000", 2024-09-19T05:44:02.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.145e-06 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644022+0000", 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045525940000000001 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646323+0000", 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023007879999999998 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646342+0000", 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8766999999999998e-05 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646435+0000", 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3394999999999996e-05 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646564+0000", 2024-09-19T05:44:02.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012848800000000001 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646738+0000", 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017418499999999999 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.741685+0000", 2024-09-19T05:44:02.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094946825999999998 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127231+0000", 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.385546526 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127301+0000", 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9741999999999997e-05 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127363+0000", 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1468000000000002e-05 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:348 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5888102819999999, 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54159870300000001, 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 348 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646372+0000", 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9110000000000002e-06 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.678772+0000", 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032399869999999997 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681572+0000", 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002799714 2024-09-19T05:44:02.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681592+0000", 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0298000000000001e-05 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682777+0000", 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011847190000000001 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682909+0000", 2024-09-19T05:44:02.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000132146 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.683102+0000", 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019280499999999999 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.702631+0000", 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019529271000000001 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187923+0000", 2024-09-19T05:44:02.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48529252 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187948+0000", 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5004999999999999e-05 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187968+0000", 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9443999999999999e-05 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:360 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5454338669999999, 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43778335400000001, 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 360 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689748+0000", 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7690000000000001e-06 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692563+0000", 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028147329999999998 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692593+0000", 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9476999999999999e-05 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692616+0000", 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3340999999999999e-05 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723279+0000", 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030662759000000001 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723430+0000", 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151247 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723629+0000", 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199426 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793213+0000", 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069583435999999999 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127443+0000", 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33423067000000001 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127491+0000", 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8000999999999997e-05 2024-09-19T05:44:02.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127529+0000", 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7494999999999999e-05 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:364 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5425301500000002, 2024-09-19T05:44:02.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43506039200000002, 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 364 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692652+0000", 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9280000000000002e-06 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697100+0000", 2024-09-19T05:44:02.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004447678 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701619+0000", 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045192829999999998 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701638+0000", 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8365000000000001e-05 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742561+0000", 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040923560999999997 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742686+0000", 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012482899999999999 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742839+0000", 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015348 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808887+0000", 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066047260999999996 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127632+0000", 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31874497099999999 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127672+0000", 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0244999999999998e-05 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127710+0000", 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7790999999999998e-05 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:395 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5297883059999999, 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48275294299999999, 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 395 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705394+0000", 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7800000000000001e-06 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720169+0000", 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014775346 2024-09-19T05:44:02.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720185+0000", 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5721000000000001e-05 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720194+0000", 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4609999999999997e-06 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720294+0000", 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100005 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720392+0000", 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7547999999999995e-05 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720531+0000", 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000139099 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.767968+0000", 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047436846999999997 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188062+0000", 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42009412899999998 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188107+0000", 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5269000000000003e-05 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188144+0000", 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6738e-05 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:404 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.51435114, 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46750329699999998, 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 404 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720832+0000", 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8469999999999998e-06 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726318+0000", 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005485775 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726490+0000", 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172339 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726505+0000", 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4467999999999999e-05 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743219+0000", 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016714046 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743320+0000", 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101548 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743459+0000", 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013899499999999999 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.775025+0000", 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031565658000000003 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188247+0000", 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.413221583 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188297+0000", 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0586e-05 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188332+0000", 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4452000000000002e-05 2024-09-19T05:44:02.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:412 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.496732336, 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40023217100000003, 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 412 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738450+0000", 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7e-06 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.758682+0000", 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020232357999999999 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772625+0000", 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013943232 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.774996+0000", 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023703890000000001 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793240+0000", 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018244646999999999 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793347+0000", 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106276 2024-09-19T05:44:02.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793511+0000", 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016406099999999999 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808895+0000", 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015384296 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138570+0000", 2024-09-19T05:44:02.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32967460500000001 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138630+0000", 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0087999999999998e-05 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138679+0000", 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9518999999999998e-05 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:441 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4410237879999999, 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39435722000000001, 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 441 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.355 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794159+0000", 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.579e-06 2024-09-19T05:44:02.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.802843+0000", 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0086840579999999997 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.807986+0000", 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051424890000000001 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808000+0000", 2024-09-19T05:44:02.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808113+0000", 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011294900000000001 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808200+0000", 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6792999999999994e-05 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808612+0000", 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041192799999999999 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827676+0000", 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019063875000000001 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188432+0000", 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36075675699999998 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188479+0000", 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6149000000000001e-05 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188513+0000", 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4310999999999997e-05 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:454 3.37 3:ec162ecd:::10000000002.000000c3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:02.360 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.402260316, 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32323945999999998, 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 454 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:02.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832922+0000", 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6280000000000001e-06 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839384+0000", 2024-09-19T05:44:02.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064625180000000004 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850436+0000", 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011051365 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850464+0000", 2024-09-19T05:44:02.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8719000000000001e-05 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850532+0000", 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7212000000000006e-05 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850663+0000", 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013160799999999999 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850877+0000", 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000213716 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871083+0000", 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020205864 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156086+0000", 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.285003532 2024-09-19T05:44:02.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156125+0000", 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8569999999999998e-05 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156159+0000", 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3728e-05 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.366 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.367 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.367 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:02.369 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:02.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.249+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c30062880 con 0x7f5c300cb2c0 2024-09-19T05:44:02.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.250+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c30062880 con 0x7f5c300cb2c0 2024-09-19T05:44:02.370 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.251+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5c3002c640 0x7f5c3002ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.251+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5c30062880 con 0x7f5c3002c640 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.251+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5c3002c640 0x7f5c3002ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.251+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5c3002c640 0x7f5c3002ca40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.252+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c30062880 con 0x7f5c3002c640 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.268+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c30062880 con 0x7f5c3002c640 2024-09-19T05:44:02.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.270+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c30062880 con 0x7f5c3002c640 2024-09-19T05:44:02.372 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:02.373 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:02.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.270+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5c3002d490 0x7f5c30206f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.270+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5c3002d8f0 con 0x7f5c3002d490 2024-09-19T05:44:02.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.271+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5c3002d490 0x7f5c30206f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.271+0000 7f5c2dd9b640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5c3002d490 0x7f5c30206f80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.272+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c3002d8f0 con 0x7f5c3002d490 2024-09-19T05:44:02.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.285+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c3002d8f0 con 0x7f5c3002d490 2024-09-19T05:44:02.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.288+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c3002d8f0 con 0x7f5c3002d490 2024-09-19T05:44:02.374 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:02.375 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.288+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5c302079d0 0x7f5c30209dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.288+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5c3020cb90 con 0x7f5c302079d0 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.288+0000 7f5c2ed9d640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5c302079d0 0x7f5c30209dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.289+0000 7f5c2ed9d640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5c302079d0 0x7f5c30209dc0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.289+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c3020cb90 con 0x7f5c302079d0 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.303+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c3020a300 con 0x7f5c302079d0 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.305+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c3020a300 con 0x7f5c302079d0 2024-09-19T05:44:02.376 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:02.377 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:02.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.305+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5c3020aa20 0x7f5c3020ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:02.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.305+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f5c3020ff00 con 0x7f5c3020aa20 2024-09-19T05:44:02.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.305+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5c3020aa20 0x7f5c3020ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:02.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.306+0000 7f5c2e59c640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5c3020aa20 0x7f5c3020ae40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:02.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.307+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c3020ff00 con 0x7f5c3020aa20 2024-09-19T05:44:02.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.320+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5c3020b490 con 0x7f5c3020aa20 2024-09-19T05:44:02.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c1effd640 1 -- 172.21.15.2:0/708945691 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5c3020b490 con 0x7f5c3020aa20 2024-09-19T05:44:02.378 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5c301f09d0 msgr2=0x7f5c300c6710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f5c301f09d0 0x7f5c300c6710 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5c3015a680 msgr2=0x7f5c30156e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f5c3015a680 0x7f5c30156e00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5c301f4100 msgr2=0x7f5c301f4520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f5c301f4100 0x7f5c301f4520 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5c300cb2c0 msgr2=0x7f5c300cb6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f5c300cb2c0 0x7f5c300cb6c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5c3002c640 msgr2=0x7f5c3002ca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c2ed9d640 1 -- 172.21.15.2:0/708945691 reap_dead start 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f5c3002c640 0x7f5c3002ca40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c2ed9d640 1 -- 172.21.15.2:0/708945691 reap_dead start 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5c3002d490 msgr2=0x7f5c30206f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f5c3002d490 0x7f5c30206f80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5c302079d0 msgr2=0x7f5c30209dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f5c302079d0 0x7f5c30209dc0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5c3020aa20 msgr2=0x7f5c3020ae40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f5c3020aa20 0x7f5c3020ae40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1006c2e0 msgr2=0x7f5c1006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1006c2e0 0x7f5c1006e7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c20001b70 tx=0x7f5c20009040 comp rx=0 tx=0).stop 2024-09-19T05:44:02.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c300cd8c0 msgr2=0x7f5c301e3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:02.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 --2- 172.21.15.2:0/708945691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c300cd8c0 0x7f5c301e3d70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5c240098c0 tx=0x7f5c240046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:02.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c2ed9d640 1 -- 172.21.15.2:0/708945691 reap_dead start 2024-09-19T05:44:02.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c2ed9d640 1 -- 172.21.15.2:0/708945691 reap_dead start 2024-09-19T05:44:02.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 shutdown_connections 2024-09-19T05:44:02.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 >> 172.21.15.2:0/708945691 conn(0x7f5c30067b30 msgr2=0x7f5c3006d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:02.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 shutdown_connections 2024-09-19T05:44:02.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:02.323+0000 7f5c34d26640 1 -- 172.21.15.2:0/708945691 wait complete. 2024-09-19T05:44:02.382 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:02.366+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:02.469 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:02.467+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:02.517 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:02.517 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:02.517 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:02.517 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:02.526 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:02.570 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:02.568+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:02.670 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:02.668+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:02.673 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:02.673 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:02.673 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:02.674 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:02.674 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:02.683 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:02.771 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:02.769+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:77 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.4827381039999992, 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40221933999999998, 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 77 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348160+0000", 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0459999999999999e-06 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362035+0000", 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013874391999999999 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362063+0000", 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7804e-05 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362076+0000", 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3200000000000001e-05 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362162+0000", 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6469000000000002e-05 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362317+0000", 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015507100000000001 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362454+0000", 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013636799999999999 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.372327+0000", 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0098729219999999993 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750183+0000", 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37785643600000002 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750233+0000", 2024-09-19T05:44:02.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9960999999999998e-05 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750376+0000", 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142671 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:87 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.840 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.4482731780000009, 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39257614200000002, 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382623+0000", 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2529999999999999e-06 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390766+0000", 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081425790000000005 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390808+0000", 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2311999999999997e-05 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390827+0000", 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390955+0000", 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012825900000000001 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391148+0000", 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000192673 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391432+0000", 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028362599999999998 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.455303+0000", 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063870983000000006 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775090+0000", 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31978768600000002 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775146+0000", 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5393000000000003e-05 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775197+0000", 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1699000000000001e-05 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.846 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:98 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3930638670000004, 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42340282099999998, 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.847 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 98 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437834+0000", 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.442377+0000", 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045437719999999997 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448982+0000", 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066041210000000001 2024-09-19T05:44:02.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449009+0000", 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7898999999999998e-05 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449078+0000", 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8301999999999998e-05 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449208+0000", 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013036300000000001 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449349+0000", 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000140791 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515702+0000", 2024-09-19T05:44:02.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066353102999999997 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861115+0000", 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34541303600000001 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861183+0000", 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8238999999999995e-05 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861233+0000", 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0081999999999999e-05 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:143 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:02.853 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3579319640000005, 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40938193499999997, 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 143 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.854 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472965+0000", 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7879999999999998e-06 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486591+0000", 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013625417000000001 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486611+0000", 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0285e-05 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486624+0000", 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2763e-05 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501834+0000", 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01521081 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504117+0000", 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022829479999999999 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504306+0000", 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018812599999999999 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515709+0000", 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403529000000001 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882232+0000", 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36652316899999998 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882300+0000", 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7539000000000005e-05 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882344+0000", 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4561000000000002e-05 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:145 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3442442410000002, 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39312144199999999, 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:02.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486653+0000", 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0089999999999999e-06 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491674+0000", 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050208450000000003 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491698+0000", 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4457999999999999e-05 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491721+0000", 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2294999999999999e-05 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510737+0000", 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019016000000000002 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510880+0000", 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014287999999999999 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511053+0000", 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017286299999999999 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555682+0000", 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044629755 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879645+0000", 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.323962272 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879715+0000", 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0400000000000004e-05 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879772+0000", 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6665e-05 2024-09-19T05:44:02.868 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:175 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3036788579999996, 2024-09-19T05:44:02.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35525569800000001, 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 175 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527221+0000", 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2299999999999999e-06 2024-09-19T05:44:02.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531495+0000", 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004274211 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556121+0000", 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024626329999999998 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556132+0000", 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0740000000000001e-05 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556187+0000", 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5096000000000002e-05 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556372+0000", 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185059 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556560+0000", 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000187803 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.562115+0000", 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055550449999999998 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882386+0000", 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32027064199999999 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882435+0000", 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8990999999999999e-05 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882471+0000", 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6551000000000003e-05 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:298 3.26 3:64018a55:::10000000002.00000027:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2185870640000003, 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59287720300000002, 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 298 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:02.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612310+0000", 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.757e-06 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619160+0000", 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068500490000000004 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619231+0000", 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0642000000000006e-05 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619249+0000", 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8544000000000001e-05 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619441+0000", 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019125500000000001 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619600+0000", 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015895800000000001 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619776+0000", 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017654300000000001 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.630552+0000", 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010775523 2024-09-19T05:44:02.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205074+0000", 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57452220799999998 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205132+0000", 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8038999999999998e-05 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205185+0000", 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2685000000000001e-05 2024-09-19T05:44:02.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:323 3.16 3:6a386151:::10000000002.00000040:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.186358169, 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.567029063, 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 323 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644539+0000", 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5040000000000001e-06 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649094+0000", 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045552429999999996 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649120+0000", 2024-09-19T05:44:02.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6176000000000001e-05 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649138+0000", 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8046e-05 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649227+0000", 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8505999999999996e-05 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649330+0000", 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010301700000000001 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649482+0000", 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015256300000000001 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.719941+0000", 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.070459037000000002 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211483+0000", 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49154185500000003 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211531+0000", 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8026999999999999e-05 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211565+0000", 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4088999999999998e-05 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.887 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:269 3.2 e54/20 3:438a76aa:::10000000002.0000000a:head v 54'6, mlcod=53'4)", 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1862554769999996, 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37357711900000001, 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644640+0000", 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.192e-06 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976810+0000", 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33216997700000001 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976833+0000", 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2878e-05 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976847+0000", 2024-09-19T05:44:02.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3644e-05 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976972+0000", 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125487 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.977007+0000", 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5027000000000002e-05 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.017830+0000", 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040822590999999998 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.018216+0000", 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038632299999999999 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:333 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1801279989999998, 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39671826999999998, 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 333 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650769+0000", 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0110000000000001e-06 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657005+0000", 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062355090000000002 2024-09-19T05:44:02.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657046+0000", 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.074e-05 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657054+0000", 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3310000000000004e-06 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657132+0000", 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7696999999999998e-05 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657256+0000", 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012425000000000001 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:02.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657471+0000", 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021535099999999999 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.715589+0000", 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058117380000000003 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047372+0000", 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33178333599999998 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047417+0000", 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5423000000000002e-05 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047485+0000", 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7242000000000004e-05 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:343 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:02.900 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1666789199999998, 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39137022999999999, 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 343 2024-09-19T05:44:02.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664218+0000", 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666946+0000", 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027284710000000001 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666962+0000", 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5620999999999998e-05 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666973+0000", 2024-09-19T05:44:02.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1199000000000001e-05 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667053+0000", 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9689000000000002e-05 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667166+0000", 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011285699999999999 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667509+0000", 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00034300599999999997 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.675954+0000", 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084449580000000007 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055481+0000", 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37952754500000002 2024-09-19T05:44:02.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055535+0000", 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055586+0000", 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0358000000000001e-05 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:354 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0874877730000003, 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.468021299, 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 354 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.908 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743409+0000", 2024-09-19T05:44:02.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7369999999999998e-06 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.747060+0000", 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036510430000000001 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768120+0000", 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021059484 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768139+0000", 2024-09-19T05:44:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9085000000000001e-05 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768289+0000", 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150252 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768387+0000", 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8169000000000001e-05 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768573+0000", 2024-09-19T05:44:02.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185973 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.776933+0000", 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083599629999999998 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211285+0000", 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43435194300000002 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211363+0000", 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7985000000000006e-05 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211428+0000", 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4665000000000005e-05 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:399 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:02.913 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0459018970000002, 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.455193454, 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 399 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784997+0000", 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.791076+0000", 2024-09-19T05:44:02.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00607888 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793492+0000", 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024155940000000001 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793539+0000", 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7246999999999997e-05 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793694+0000", 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000155589 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793818+0000", 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012335499999999999 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794028+0000", 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000209822 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839319+0000", 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045291168999999999 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240115+0000", 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40079620599999999 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240147+0000", 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1782000000000002e-05 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240186+0000", 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9168999999999997e-05 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:401 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0335740869999999, 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41436097300000002, 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.919 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 401 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797324+0000", 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6239999999999999e-06 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803827+0000", 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065027219999999998 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803845+0000", 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8717999999999999e-05 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803858+0000", 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2599000000000001e-05 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803963+0000", 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010488 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.804102+0000", 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013866900000000001 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.804299+0000", 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000197624 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.861232+0000", 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056932356000000003 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211606+0000", 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35037407999999998 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211650+0000", 2024-09-19T05:44:02.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4629000000000002e-05 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211681+0000", 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1072000000000003e-05 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:431 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.925 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.9771813499999999, 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38660652099999998, 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 431 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853715+0000", 2024-09-19T05:44:02.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.95e-06 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862660+0000", 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008945138 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862684+0000", 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4102999999999999e-05 2024-09-19T05:44:02.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862697+0000", 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2899e-05 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862819+0000", 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012190500000000001 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863007+0000", 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018809600000000001 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.865417+0000", 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002410316 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967698+0000", 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102280148 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240225+0000", 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27252755200000001 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240282+0000", 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6935999999999998e-05 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240320+0000", 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7478000000000003e-05 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:432 3.26 3:66aef6e9:::10000000002.000000ad:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.9681624449999999, 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37769895199999998, 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 432 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862736+0000", 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1749999999999998e-06 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868431+0000", 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005694569 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868467+0000", 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5809000000000002e-05 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868482+0000", 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.52e-05 2024-09-19T05:44:02.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896790+0000", 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028308262000000001 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896919+0000", 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012855399999999999 2024-09-19T05:44:02.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.897240+0000", 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032181899999999999 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037309+0000", 2024-09-19T05:44:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.140068843 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240357+0000", 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20304760699999999 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240400+0000", 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2849000000000003e-05 2024-09-19T05:44:02.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240431+0000", 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1265e-05 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:457 3.26 3:64ff7f1d:::10000000002.000000c6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.939 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.9448645760000001, 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35451085300000001, 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 457 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886033+0000", 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889815+0000", 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037826800000000001 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889868+0000", 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2592000000000001e-05 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889881+0000", 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3251000000000001e-05 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912234+0000", 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022353101 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912416+0000", 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000181695 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912603+0000", 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186536 2024-09-19T05:44:02.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037342+0000", 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.124739117 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240463+0000", 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.203121157 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240505+0000", 2024-09-19T05:44:02.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2406e-05 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240541+0000", 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5488999999999999e-05 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:460 3.26 3:6430d4f3:::10000000002.000000c9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.93501995, 2024-09-19T05:44:02.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34478341699999998, 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 460 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895877+0000", 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.779e-06 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898920+0000", 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030423379999999999 2024-09-19T05:44:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898937+0000", 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7782999999999999e-05 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898948+0000", 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665e-05 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953289+0000", 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054340672999999999 2024-09-19T05:44:02.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953400+0000", 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111527 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953557+0000", 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015719599999999999 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.130946+0000", 2024-09-19T05:44:02.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17738901100000001 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240578+0000", 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10963127 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240625+0000", 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7429000000000003e-05 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240658+0000", 2024-09-19T05:44:02.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2746000000000001e-05 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:463 3.26 3:65689812:::10000000002.000000cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.9319190289999999, 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34245139299999999, 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 463 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898979+0000", 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2610000000000002e-06 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902351+0000", 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033725270000000002 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902372+0000", 2024-09-19T05:44:02.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0699e-05 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902388+0000", 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6461e-05 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967734+0000", 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065346107 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967874+0000", 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013911199999999999 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.968092+0000", 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218438 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.204745+0000", 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23665298800000001 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241370+0000", 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036625433999999998 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241402+0000", 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1523999999999998e-05 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241427+0000", 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4842e-05 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:467 3.26 3:65bb0d93:::10000000002.000000d0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.928474606, 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34422604099999998, 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 467 2024-09-19T05:44:02.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:02.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902423+0000", 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.117e-06 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908646+0000", 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062233660000000001 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908664+0000", 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7880000000000002e-05 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:02.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908679+0000", 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5129e-05 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211711+0000", 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30303182099999998 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211794+0000", 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2541999999999997e-05 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211940+0000", 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014651900000000001 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241451+0000", 2024-09-19T05:44:02.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029510593000000002 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246560+0000", 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051091019999999999 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246605+0000", 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123999999999997e-05 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:02.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246646+0000", 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0847999999999999e-05 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:02.963 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:02.965 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:02.966 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:02.963+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:03.065 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:03.063+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:03.113 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:03.113 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:03.113 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:03.113 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:03.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:03.207 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:03.205+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:03.266 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:03.264+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:03.273 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:03.273 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:03.273 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:03.273 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:03.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:03.367 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:03.366+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:03.428 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:03.428 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:03.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:03.428 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:03.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:11 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.138013751000001, 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53368345399999995, 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287043+0000", 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294617+0000", 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075741280000000003 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294641+0000", 2024-09-19T05:44:03.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3424000000000001e-05 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294656+0000", 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4679e-05 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294841+0000", 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185176 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295155+0000", 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031388199999999997 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295811+0000", 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065616700000000003 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.308016+0000", 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012205472 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820590+0000", 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51257333400000005 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820658+0000", 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8836000000000003e-05 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820724+0000", 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5426999999999995e-05 2024-09-19T05:44:03.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:18 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.087302748000001, 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48181817700000001, 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 18 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337754+0000", 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346234+0000", 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084795919999999993 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346309+0000", 2024-09-19T05:44:03.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4950999999999994e-05 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346324+0000", 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5199e-05 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346456+0000", 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013213799999999999 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346703+0000", 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024689500000000001 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346893+0000", 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019002400000000001 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382206+0000", 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035313485999999998 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819443+0000", 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43723664899999998 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819516+0000", 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3021000000000004e-05 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819570+0000", 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3696999999999998e-05 2024-09-19T05:44:03.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:37 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.007751345999999, 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.412251706, 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417306+0000", 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3790000000000002e-06 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421246+0000", 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039399869999999998 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421273+0000", 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6412999999999999e-05 2024-09-19T05:44:03.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421284+0000", 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1856e-05 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421374+0000", 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9411000000000002e-05 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429518+0000", 2024-09-19T05:44:03.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008143908 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429741+0000", 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022306100000000001 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.460095+0000", 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030354269999999999 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829462+0000", 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36936722900000002 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829510+0000", 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7698999999999999e-05 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829555+0000", 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4493000000000002e-05 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:53 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.9945839549999995, 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45610759200000001, 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 53 2024-09-19T05:44:03.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430472+0000", 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1320000000000001e-06 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437581+0000", 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071082300000000001 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444289+0000", 2024-09-19T05:44:03.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067080070000000002 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444322+0000", 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3034000000000003e-05 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445189+0000", 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086775100000000005 2024-09-19T05:44:03.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445402+0000", 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021237099999999999 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445599+0000", 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019734099999999999 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501230+0000", 2024-09-19T05:44:03.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055630863000000003 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886462+0000", 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38523158200000002 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886525+0000", 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3138000000000004e-05 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886578+0000", 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3143000000000003e-05 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:61 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.453 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.9676496219999997, 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.424715008, 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 61 2024-09-19T05:44:03.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457409+0000", 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461813+0000", 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044040060000000002 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461834+0000", 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0789e-05 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461876+0000", 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2268000000000003e-05 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461987+0000", 2024-09-19T05:44:03.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110593 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467170+0000", 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051828849999999999 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467417+0000", 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024757199999999999 2024-09-19T05:44:03.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490516+0000", 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023098802000000002 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.881965+0000", 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39144917200000001 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882043+0000", 2024-09-19T05:44:03.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7830000000000005e-05 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882120+0000", 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6360999999999996e-05 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:267 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8285030930000001, 2024-09-19T05:44:03.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52119119999999997, 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 267 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596555+0000", 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.596e-06 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597947+0000", 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001392517 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597975+0000", 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7993000000000001e-05 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597989+0000", 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3927999999999999e-05 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598090+0000", 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010057500000000001 2024-09-19T05:44:03.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598219+0000", 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012952999999999999 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598408+0000", 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018851600000000001 2024-09-19T05:44:03.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604394+0000", 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059861159999999997 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117572+0000", 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51317804499999997 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117624+0000", 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2126000000000002e-05 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117742+0000", 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011825800000000001 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:274 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:03.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.8128610409999997, 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63191469899999997, 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 274 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:03.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612198+0000", 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.476e-06 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619545+0000", 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073473399999999999 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619574+0000", 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8994000000000001e-05 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619585+0000", 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1103e-05 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619846+0000", 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026047199999999998 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619979+0000", 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013353099999999999 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.620203+0000", 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000223945 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629556+0000", 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093532560000000008 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244028+0000", 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61447147300000005 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244065+0000", 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6968999999999998e-05 2024-09-19T05:44:03.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244108+0000", 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3139999999999997e-05 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:293 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7713623549999999, 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59051000200000003, 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 293 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653696+0000", 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.33e-06 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658892+0000", 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051954360000000003 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658913+0000", 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1705999999999999e-05 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658928+0000", 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4361e-05 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659326+0000", 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000398253 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659446+0000", 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012050200000000001 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659655+0000", 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020868500000000001 2024-09-19T05:44:03.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.734308+0000", 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074653061000000007 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244147+0000", 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.50983838800000003 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244175+0000", 2024-09-19T05:44:03.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8901999999999999e-05 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244202+0000", 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6378000000000001e-05 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:309 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7556284240000002, 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.505676974, 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 309 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669428+0000", 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.593e-06 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.676971+0000", 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075425839999999997 2024-09-19T05:44:03.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677009+0000", 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8154000000000001e-05 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677039+0000", 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9788999999999999e-05 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677223+0000", 2024-09-19T05:44:03.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000184372 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677347+0000", 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123979 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677516+0000", 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016852499999999999 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732124+0000", 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054607912000000002 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.174998+0000", 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44287454799999998 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175054+0000", 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5476999999999998e-05 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175103+0000", 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9041e-05 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:313 3.2f 3:f7c72662:::10000000002.00000036:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7479277260000003, 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47726204500000002, 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 313 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677129+0000", 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3479999999999998e-06 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685213+0000", 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080837730000000007 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685237+0000", 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4311000000000002e-05 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685254+0000", 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7428999999999999e-05 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863622+0000", 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.178367422 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863724+0000", 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101938 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863861+0000", 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013715 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.152292+0000", 2024-09-19T05:44:03.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.288431614 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154270+0000", 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019771250000000001 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154330+0000", 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154388+0000", 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8717e-05 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:317 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.7397053580000001, 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56821626300000005, 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 317 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685351+0000", 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3549999999999999e-06 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692638+0000", 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072869909999999996 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692666+0000", 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8010000000000001e-05 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692677+0000", 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0776e-05 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692800+0000", 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012327200000000001 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692936+0000", 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013544100000000001 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.693426+0000", 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00049003600000000001 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732131+0000", 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038705212000000003 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253472+0000", 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52134096299999999 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253518+0000", 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5731e-05 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253565+0000", 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7475999999999997e-05 2024-09-19T05:44:03.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:331 3.30 3:0d2dbd17:::10000000002.00000048:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6760410620000004, 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.495291814, 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.503 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 331 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749016+0000", 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.754846+0000", 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058302459999999999 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756282+0000", 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001436295 2024-09-19T05:44:03.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756297+0000", 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4921e-05 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756370+0000", 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2856999999999997e-05 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756495+0000", 2024-09-19T05:44:03.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012511999999999999 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756711+0000", 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216233 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828851+0000", 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072139743000000006 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244223+0000", 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41537228599999998 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244270+0000", 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244305+0000", 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4527999999999999e-05 2024-09-19T05:44:03.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:346 3.11 3:8815798b:::10000000002.00000057:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6687269130000004, 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49739494099999998, 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 346 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756330+0000", 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759712+0000", 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033821889999999999 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759737+0000", 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5069e-05 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759752+0000", 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4372999999999999e-05 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759847+0000", 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5291999999999995e-05 2024-09-19T05:44:03.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760098+0000", 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025112099999999998 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760422+0000", 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032433199999999998 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855328+0000", 2024-09-19T05:44:03.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094905274999999997 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253622+0000", 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39829475600000003 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253679+0000", 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6047000000000001e-05 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253722+0000", 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3702999999999998e-05 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:361 3.30 3:0eedefab:::10000000002.00000066:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6525913939999999, 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47196730199999998, 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 361 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772466+0000", 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8279999999999999e-06 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777394+0000", 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049281790000000004 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777413+0000", 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8913999999999999e-05 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777424+0000", 2024-09-19T05:44:03.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1192999999999999e-05 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799507+0000", 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022083230999999998 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799656+0000", 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014921300000000001 2024-09-19T05:44:03.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799858+0000", 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020186999999999999 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830773+0000", 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030915148999999999 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244343+0000", 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41356947900000002 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244392+0000", 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9489e-05 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244430+0000", 2024-09-19T05:44:03.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7756999999999998e-05 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:366 3.11 3:8b4c71c3:::10000000002.0000006b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6476015720000001, 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47662371999999997, 2024-09-19T05:44:03.522 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 366 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:03.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777455+0000", 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7640000000000001e-06 2024-09-19T05:44:03.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783373+0000", 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005917533 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783420+0000", 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6709999999999998e-05 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783443+0000", 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3836999999999999e-05 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808013+0000", 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024569607 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808128+0000", 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011490299999999999 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808375+0000", 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024700299999999998 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937889+0000", 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951412500000001 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253987+0000", 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31609763600000002 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254037+0000", 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0383999999999997e-05 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254076+0000", 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9218000000000003e-05 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:368 3.14 3:280f29a3:::10000000002.0000006d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.529 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6415561949999997, 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45471794900000001, 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 368 2024-09-19T05:44:03.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:03.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783504+0000", 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7749999999999998e-06 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785589+0000", 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020852409999999998 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785606+0000", 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6609999999999999e-05 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785614+0000", 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4940000000000003e-06 2024-09-19T05:44:03.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789369+0000", 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037552250000000001 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789485+0000", 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116019 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789654+0000", 2024-09-19T05:44:03.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016819299999999999 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828885+0000", 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039231901999999999 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238106+0000", 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40922085699999999 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238164+0000", 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7373000000000001e-05 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238216+0000", 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2259999999999998e-05 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:382 3.11 3:8818af48:::10000000002.0000007b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:03.536 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.636148274, 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46533816500000003, 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 382 2024-09-19T05:44:03.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788911+0000", 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6609999999999999e-06 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796253+0000", 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073421060000000002 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796284+0000", 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1383000000000001e-05 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796299+0000", 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4793e-05 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832493+0000", 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036194021 2024-09-19T05:44:03.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832612+0000", 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011929 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832778+0000", 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.066630+0000", 2024-09-19T05:44:03.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.233852226 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254144+0000", 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18751385100000001 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254200+0000", 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6028e-05 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254244+0000", 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4373999999999999e-05 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:403 3.9 3:91a8729a:::10000000002.00000090:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.6084832269999998, 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41822062799999998, 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 403 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816573+0000", 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.345e-06 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.820472+0000", 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038989060000000002 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827487+0000", 2024-09-19T05:44:03.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070152189999999996 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827507+0000", 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9434000000000001e-05 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858059+0000", 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030552598 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858252+0000", 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019281999999999999 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871029+0000", 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012776648999999999 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175256+0000", 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30422741199999997 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234683+0000", 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059426432000000001 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234738+0000", 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4923999999999998e-05 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234792+0000", 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3888999999999999e-05 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.548 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:405 3.11 3:8834f03f:::10000000002.00000092:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5975199089999998, 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42686835699999998, 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 405 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827537+0000", 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6620000000000001e-06 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830490+0000", 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029526209999999999 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830506+0000", 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6891000000000001e-05 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830535+0000", 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8377e-05 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855366+0000", 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024831576000000001 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855488+0000", 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012119499999999999 2024-09-19T05:44:03.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855676+0000", 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018832899999999999 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233600+0000", 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37792366900000002 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254308+0000", 2024-09-19T05:44:03.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020708684000000002 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254362+0000", 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3430000000000002e-05 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254403+0000", 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0923000000000001e-05 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:410 3.11 3:8988b152:::10000000002.00000097:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5944977380000003, 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42399440900000002, 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 410 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:03.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830559+0000", 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835124+0000", 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045654520000000002 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835145+0000", 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0750999999999998e-05 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835179+0000", 2024-09-19T05:44:03.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4013000000000001e-05 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937947+0000", 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102767884 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938050+0000", 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010270199999999999 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938225+0000", 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000175494 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234859+0000", 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.296633759 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254459+0000", 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019599979999999999 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254510+0000", 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1431000000000003e-05 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254551+0000", 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0391e-05 2024-09-19T05:44:03.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.561 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:03.561 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:03.561 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:03.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:03.563 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:03.561+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:03.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:44:03.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:44:03.683 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:03.683 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:03.683 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:03.683 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:03.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:03.695 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:44:03.700 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:03.698+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:03.763 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:03.761+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:03.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.801+0000 7f341cf8d640 1 Processor -- start 2024-09-19T05:44:03.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.801+0000 7f341cf8d640 1 -- start start 2024-09-19T05:44:03.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341cf8d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f341815c150 0x7f34180c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341cf8d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34180c3a60 con 0x7f3418152b70 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341cf8d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34180c3d40 con 0x7f341815c530 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341cf8d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f34180c4c00 con 0x7f341815c150 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341659c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f341815c150 0x7f34180c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f3416d9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3418152b70 0x7f341806f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55558/0 (socket says 172.21.15.2:55558) 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f3416d9d640 1 -- 172.21.15.2:0/860974570 learned_addr learned my addr 172.21.15.2:0/860974570 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341659c640 1 -- 172.21.15.2:0/860974570 >> v1:172.21.15.123:6789/0 conn(0x7f341815c530 legacy=0x7f3418152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341659c640 1 -- 172.21.15.2:0/860974570 >> v1:172.21.15.2:6789/0 conn(0x7f3418152b70 legacy=0x7f341806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341559a640 1 -- 172.21.15.2:0/860974570 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876535599 0 0) 0x7f34180c3a60 con 0x7f3418152b70 2024-09-19T05:44:03.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.802+0000 7f341659c640 1 -- 172.21.15.2:0/860974570 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34180c4ee0 con 0x7f341815c150 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.803+0000 7f341659c640 1 --2- 172.21.15.2:0/860974570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c150 0x7f34180c6e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3410009d90 tx=0x7f341002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12931e28a6f5c5e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.803+0000 7f341559a640 1 -- 172.21.15.2:0/860974570 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410007740 con 0x7f341815c150 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.803+0000 7f341559a640 1 -- 172.21.15.2:0/860974570 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34100078e0 con 0x7f341815c150 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.803+0000 7f341559a640 1 -- 172.21.15.2:0/860974570 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410007cb0 con 0x7f341815c150 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.804+0000 7f341cf8d640 1 -- 172.21.15.2:0/860974570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c150 msgr2=0x7f34180c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.804+0000 7f341cf8d640 1 --2- 172.21.15.2:0/860974570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c150 0x7f34180c6e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3410009d90 tx=0x7f341002a6f0 comp rx=0 tx=0).stop 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.804+0000 7f341cf8d640 1 -- 172.21.15.2:0/860974570 shutdown_connections 2024-09-19T05:44:03.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.804+0000 7f341cf8d640 1 --2- 172.21.15.2:0/860974570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c150 0x7f34180c6e90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.804+0000 7f341cf8d640 1 -- 172.21.15.2:0/860974570 >> 172.21.15.2:0/860974570 conn(0x7f341806bc10 msgr2=0x7f341806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.804+0000 7f341cf8d640 1 -- 172.21.15.2:0/860974570 shutdown_connections 2024-09-19T05:44:03.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.804+0000 7f341cf8d640 1 -- 172.21.15.2:0/860974570 wait complete. 2024-09-19T05:44:03.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.805+0000 7f341cf8d640 1 Processor -- start 2024-09-19T05:44:03.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.805+0000 7f341cf8d640 1 -- start start 2024-09-19T05:44:03.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f341cf8d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3418152b70 0x7f3418157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:03.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f341cf8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f341815c150 0x7f3418158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:03.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f341cf8d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c530 0x7f34181f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f341cf8d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f34180c7570 con 0x7f341815c530 2024-09-19T05:44:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f341cf8d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34180c51c0 con 0x7f3418152b70 2024-09-19T05:44:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f341cf8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f34180c76f0 con 0x7f341815c150 2024-09-19T05:44:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c530 0x7f34181f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c530 0x7f34181f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55108/0 (socket says 172.21.15.2:55108) 2024-09-19T05:44:03.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 -- 172.21.15.2:0/214242037 learned_addr learned my addr 172.21.15.2:0/214242037 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f341659c640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3418152b70 0x7f3418157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3415d9b640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f341815c150 0x7f3418158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 -- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f341815c150 msgr2=0x7f3418158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f341815c150 0x7f3418158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 -- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3418152b70 msgr2=0x7f3418157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3418152b70 0x7f3418157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.806+0000 7f3416d9d640 1 -- 172.21.15.2:0/214242037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3418156a10 con 0x7f341815c530 2024-09-19T05:44:03.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.807+0000 7f3416d9d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c530 0x7f34181f1850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f340800c480 tx=0x7f340800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.807+0000 7f340effd640 1 -- 172.21.15.2:0/214242037 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408018070 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.807+0000 7f340effd640 1 -- 172.21.15.2:0/214242037 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34080074e0 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.807+0000 7f340effd640 1 -- 172.21.15.2:0/214242037 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408004a20 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.807+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34180cb670 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.807+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34180cbbf0 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.808+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34180c6e90 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.808+0000 7f340effd640 1 -- 172.21.15.2:0/214242037 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f340801b070 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.809+0000 7f340effd640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33f806c2b0 0x7f33f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.809+0000 7f340effd640 1 -- 172.21.15.2:0/214242037 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f340808a690 con 0x7f341815c530 2024-09-19T05:44:03.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.809+0000 7f341659c640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33f806c2b0 0x7f33f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:03.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.809+0000 7f341659c640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33f806c2b0 0x7f33f806e770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3410034000 tx=0x7f341002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:03.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:03.813+0000 7f340effd640 1 -- 172.21.15.2:0/214242037 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3418156a10 con 0x7f341815c530 2024-09-19T05:44:03.839 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:03.839 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:03.840 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:03.840 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:03.840 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:03.849 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:03.864 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:03.863+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:03.954 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:03.954 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:03.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:13 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.638116175, 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41922241599999999, 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 13 2024-09-19T05:44:03.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313315+0000", 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.978e-06 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329516+0000", 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016201601999999999 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329535+0000", 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9157e-05 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329545+0000", 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9909999999999997e-06 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329609+0000", 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3931999999999997e-05 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329764+0000", 2024-09-19T05:44:03.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015456700000000001 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329895+0000", 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013116700000000001 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.336041+0000", 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061460580000000003 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732421+0000", 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.396379484 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732484+0000", 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3100000000000002e-05 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732535+0000", 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1379999999999999e-05 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:64 3.2 3:40abce4f:::10000000002.0000003d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.961 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.489486689, 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29680736000000002, 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 64 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461945+0000", 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6860000000000002e-06 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478410+0000", 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016465418999999999 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478422+0000", 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1732e-05 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478431+0000", 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8890000000000008e-06 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478503+0000", 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.25e-05 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478662+0000", 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000158696 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478774+0000", 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011198500000000001 2024-09-19T05:44:03.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481734+0000", 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002960018 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758602+0000", 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27686762799999998 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758681+0000", 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9566000000000004e-05 2024-09-19T05:44:03.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758750+0000", 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8240999999999999e-05 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:82 3.2b 3:d50470f2:::10000000002.0000004f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.459692237000001, 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29488188799999998, 2024-09-19T05:44:03.967 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 82 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491754+0000", 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7186000000000001e-05 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.506963+0000", 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015209675000000001 2024-09-19T05:44:03.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.506991+0000", 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7872e-05 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507003+0000", 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507080+0000", 2024-09-19T05:44:03.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7327e-05 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507223+0000", 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014263500000000001 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507439+0000", 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000215873 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.511608+0000", 2024-09-19T05:44:03.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041688610000000003 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786537+0000", 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27492934899999999 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786576+0000", 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8575000000000003e-05 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:03.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786619+0000", 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2815999999999998e-05 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:132 3.2b 3:d52309a9:::10000000002.00000081:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.378818137, 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21416674099999999, 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.973 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 132 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572614+0000", 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585573+0000", 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012959205999999999 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585590+0000", 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7326999999999999e-05 2024-09-19T05:44:03.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585605+0000", 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5061e-05 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585665+0000", 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9991999999999998e-05 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585800+0000", 2024-09-19T05:44:03.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013432599999999999 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585932+0000", 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013247199999999999 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.588624+0000", 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026916520000000001 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786688+0000", 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.198064771 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786738+0000", 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9951999999999999e-05 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786777+0000", 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9053e-05 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:53 3.9 e53/20 3:9387ba5e:::10000000002.00000032:head v 53'1, mlcod=0'0)", 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.356510513, 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28964351700000002, 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594919+0000", 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882027+0000", 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28710772299999998 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882048+0000", 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1081e-05 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882056+0000", 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9049999999999997e-06 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882114+0000", 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7762e-05 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882132+0000", 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8762000000000001e-05 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.884525+0000", 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002392595 2024-09-19T05:44:03.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.884562+0000", 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6788000000000001e-05 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:161 3.2b 3:d691b431:::10000000002.0000009e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.351631202, 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.206406749, 2024-09-19T05:44:03.983 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 161 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599800+0000", 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.655e-06 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609524+0000", 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097240499999999997 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609544+0000", 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9772e-05 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609553+0000", 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3130000000000004e-06 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609604+0000", 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0933999999999997e-05 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609704+0000", 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9746000000000003e-05 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609799+0000", 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4823999999999999e-05 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.613360+0000", 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035613950000000002 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806134+0000", 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19277316 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806173+0000", 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9873000000000003e-05 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806204+0000", 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1027e-05 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:168 3.2b 3:d5eb8d7b:::10000000002.000000a5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.341852227, 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19673869999999999, 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.989 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 168 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609579+0000", 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6819999999999999e-06 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619452+0000", 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0098723679999999994 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619470+0000", 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8515000000000001e-05 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619479+0000", 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8650000000000007e-06 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619556+0000", 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6569000000000005e-05 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619691+0000", 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013584900000000001 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619804+0000", 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112989 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.626514+0000", 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067099350000000002 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806244+0000", 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.179729837 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806278+0000", 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4221999999999999e-05 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806315+0000", 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6869000000000002e-05 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:172 3.2b 3:d5480d6b:::10000000002.000000a9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.331923136, 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.186920695, 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 172 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.996 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619508+0000", 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:44:03.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630572+0000", 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011063454 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630589+0000", 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6860000000000001e-05 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630596+0000", 2024-09-19T05:44:03.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6420000000000007e-06 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630659+0000", 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3145999999999995e-05 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630785+0000", 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000125788 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:03.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630894+0000", 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001087 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.635728+0000", 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048346090000000001 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806354+0000", 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.170625477 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806396+0000", 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2252999999999997e-05 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806426+0000", 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0266000000000001e-05 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:179 3.2 3:42ce5726:::10000000002.000000b0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:04.001 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.320808057000001, 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17570660699999999, 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 179 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:04.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630624+0000", 2024-09-19T05:44:04.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7870000000000002e-06 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641403+0000", 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01077914 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641420+0000", 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7564e-05 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641442+0000", 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1355000000000001e-05 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641517+0000", 2024-09-19T05:44:04.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5712999999999998e-05 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641680+0000", 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016307800000000001 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641848+0000", 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016733500000000001 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647878+0000", 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060299159999999997 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806222+0000", 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.158344174 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806290+0000", 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8201999999999995e-05 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806327+0000", 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7342999999999998e-05 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:269 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3583112350000004, 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.44962057300000002, 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.008 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 269 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:04.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593119+0000", 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5099999999999999e-06 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614489+0000", 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.021369545 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614511+0000", 2024-09-19T05:44:04.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2353000000000001e-05 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614525+0000", 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4311000000000001e-05 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614691+0000", 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016586800000000001 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614811+0000", 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119552 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614956+0000", 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014476499999999999 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.619618+0000", 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046627020000000003 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042637+0000", 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.42301865999999999 2024-09-19T05:44:04.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042689+0000", 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1876000000000003e-05 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042738+0000", 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9431000000000002e-05 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.014 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:320 3.2 3:40abce4f:::10000000002.0000003d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2138872870000004, 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30531098200000001, 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 320 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.015 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737544+0000", 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6539999999999999e-06 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760554+0000", 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023009943000000001 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760577+0000", 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2538e-05 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760585+0000", 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9219999999999997e-06 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760691+0000", 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106606 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760791+0000", 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100239 2024-09-19T05:44:04.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760966+0000", 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017421299999999999 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.766414+0000", 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005448067 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042779+0000", 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27636495599999999 2024-09-19T05:44:04.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042822+0000", 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3359000000000003e-05 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042853+0000", 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0485e-05 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.020 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:338 3.2b 3:d50470f2:::10000000002.0000004f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.1739089859999998, 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30392799300000001, 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.021 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 338 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777522+0000", 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3539999999999998e-06 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.793967+0000", 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016445120000000001 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794010+0000", 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3143999999999999e-05 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794026+0000", 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5404999999999999e-05 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794124+0000", 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7806999999999994e-05 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794224+0000", 2024-09-19T05:44:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100318 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794397+0000", 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017272500000000001 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.800396+0000", 2024-09-19T05:44:04.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059990449999999997 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081329+0000", 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28093356000000003 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081398+0000", 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8786000000000002e-05 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081448+0000", 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9728999999999998e-05 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.027 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:388 3.2b 3:d52309a9:::10000000002.00000081:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.0601979579999998, 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19034685300000001, 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 388 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891233+0000", 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4059999999999999e-06 2024-09-19T05:44:04.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905206+0000", 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013972507 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905225+0000", 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9167000000000001e-05 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905259+0000", 2024-09-19T05:44:04.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3633999999999997e-05 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905360+0000", 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101512 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905437+0000", 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6964000000000004e-05 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905546+0000", 2024-09-19T05:44:04.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108933 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.909399+0000", 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003853249 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081492+0000", 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.172092667 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081535+0000", 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3287000000000001e-05 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081578+0000", 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2527000000000001e-05 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.033 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:417 3.2b 3:d691b431:::10000000002.0000009e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.0339116290000003, 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19858436900000001, 2024-09-19T05:44:04.034 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 417 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917519+0000", 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2330000000000001e-06 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936253+0000", 2024-09-19T05:44:04.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018733107999999998 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936269+0000", 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6441999999999999e-05 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936278+0000", 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8370000000000003e-06 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936338+0000", 2024-09-19T05:44:04.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0318000000000001e-05 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936414+0000", 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6249999999999997e-05 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936508+0000", 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3194000000000003e-05 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.941616+0000", 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051086789999999997 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116003+0000", 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17438693 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116057+0000", 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3829999999999998e-05 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116102+0000", 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:424 3.2b 3:d5eb8d7b:::10000000002.000000a5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.0151255160000003, 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17989760599999999, 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.040 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 424 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:04.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936305+0000", 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9910000000000001e-06 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950048+0000", 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013742437 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950078+0000", 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9904e-05 2024-09-19T05:44:04.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950092+0000", 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4543e-05 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950182+0000", 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0236999999999997e-05 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950294+0000", 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111993 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950451+0000", 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015673400000000001 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.954644+0000", 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041925249999999999 2024-09-19T05:44:04.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116134+0000", 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16149000999999999 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116179+0000", 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4919000000000001e-05 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116201+0000", 2024-09-19T05:44:04.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2313000000000001e-05 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:428 3.2b 3:d5480d6b:::10000000002.000000a9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.0007671040000004, 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17460092799999999, 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.047 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 428 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950665+0000", 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.931e-06 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967645+0000", 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016980439 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967663+0000", 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7657999999999999e-05 2024-09-19T05:44:04.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967670+0000", 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3529999999999997e-06 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967735+0000", 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4379000000000001e-05 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967844+0000", 2024-09-19T05:44:04.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109863 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.968022+0000", 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177987 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973507+0000", 2024-09-19T05:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054845980000000003 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125044+0000", 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15153746700000001 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125149+0000", 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104286 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125263+0000", 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113967 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:439 3.1b 3:db82751e:::10000000002.000000b4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.9716636029999997, 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19035659999999999, 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 439 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979767+0000", 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6819999999999999e-06 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993421+0000", 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013654314000000001 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993442+0000", 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0911e-05 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993451+0000", 2024-09-19T05:44:04.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.918e-06 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993525+0000", 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4259999999999997e-05 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993635+0000", 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109851 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993795+0000", 2024-09-19T05:44:04.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015986599999999999 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.000943+0000", 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071475469999999998 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170055+0000", 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16911190800000001 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170090+0000", 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5222999999999998e-05 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170122+0000", 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2119999999999997e-05 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:443 3.1b 3:da9dcad1:::10000000002.000000b8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:04.059 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.9579520280000002, 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.186683074, 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 443 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:04.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993478+0000", 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.398e-06 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004522+0000", 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011043536 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004541+0000", 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9301999999999999e-05 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004551+0000", 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7589999999999992e-06 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004657+0000", 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106069 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004805+0000", 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148197 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004912+0000", 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010655800000000001 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.007998+0000", 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030858679999999999 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180063+0000", 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17206524200000001 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180114+0000", 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1122999999999998e-05 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180160+0000", 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6022000000000001e-05 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:490 3.2 3:437e6408:::10000000002.000000e7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.9068705139999995, 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15379004299999999, 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.066 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 490 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044560+0000", 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.685e-06 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054251+0000", 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096909700000000001 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054267+0000", 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5755000000000001e-05 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054277+0000", 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0504e-05 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054333+0000", 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6001000000000001e-05 2024-09-19T05:44:04.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054396+0000", 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2852000000000001e-05 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054483+0000", 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7332999999999999e-05 2024-09-19T05:44:04.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.057123+0000", 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002639705 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198289+0000", 2024-09-19T05:44:04.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.14116562599999999 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198321+0000", 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2543000000000003e-05 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198348+0000", 2024-09-19T05:44:04.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7069000000000001e-05 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:502 3.2 3:4041dd16:::10000000002.000000f3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.8971308970000003, 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.14802890099999999, 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.073 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 502 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:04.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054300+0000", 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.012e-06 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063321+0000", 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090211760000000005 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063333+0000", 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2308e-05 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063340+0000", 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7819999999999996e-06 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063427+0000", 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6948999999999997e-05 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063536+0000", 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109066 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063706+0000", 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000169906 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068352+0000", 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046461590000000004 2024-09-19T05:44:04.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202270+0000", 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.133917486 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202302+0000", 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2237999999999999e-05 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202327+0000", 2024-09-19T05:44:04.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4819000000000001e-05 2024-09-19T05:44:04.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.079 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.079 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.079 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:04.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.002+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f34180c79d0 con 0x7f341815c530 2024-09-19T05:44:04.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:04.081 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:04.078+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:04.147 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:04.147 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 re-readers = 7252024.00 kB/sec 2024-09-19T05:44:04.147 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 re-readers = 7242505.90 kB/sec 2024-09-19T05:44:04.147 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 7252024.00 kB/sec 2024-09-19T05:44:04.147 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 7252024.00 kB/sec 2024-09-19T05:44:04.148 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 7252024.00 kB/sec 2024-09-19T05:44:04.148 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:44:04.181 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:04.179+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:04.224 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:04.224 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:04.224 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:04.224 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:04.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:04.281 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:04.280+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:04.336 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:04.336 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:04.336 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:04.336 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:04.336 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:04.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:04.382 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:04.386 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:04.384+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:04.483 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:04.481+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:04.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.482+0000 7f6bc7fbd640 1 Processor -- start 2024-09-19T05:44:04.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.482+0000 7f6bc7fbd640 1 -- start start 2024-09-19T05:44:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc7fbd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6bc0152ce0 0x7f6bc006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc7fbd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6bc00c62a0 0x7f6bc006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc7fbd640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc0159760 con 0x7f6bc00c3000 2024-09-19T05:44:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc7fbd640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc00c3d80 con 0x7f6bc00c62a0 2024-09-19T05:44:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc7fbd640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc0159220 con 0x7f6bc0152ce0 2024-09-19T05:44:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc655a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6bc00c62a0 0x7f6bc006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc5558640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6bc00c3000 0x7f6bc00c3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55562/0 (socket says 172.21.15.2:55562) 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc655a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6bc00c62a0 0x7f6bc006f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55120/0 (socket says 172.21.15.2:55120) 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc5558640 1 -- 172.21.15.2:0/2963542068 learned_addr learned my addr 172.21.15.2:0/2963542068 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/2963542068 >> v2:172.21.15.123:3300/0 conn(0x7f6bc0152ce0 0x7f6bc006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc655a640 1 -- 172.21.15.2:0/2963542068 >> v2:172.21.15.123:3300/0 conn(0x7f6bc0152ce0 msgr2=0x7f6bc006ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc655a640 1 --2- 172.21.15.2:0/2963542068 >> v2:172.21.15.123:3300/0 conn(0x7f6bc0152ce0 0x7f6bc006ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc655a640 1 -- 172.21.15.2:0/2963542068 >> v1:172.21.15.2:6789/0 conn(0x7f6bc00c3000 legacy=0x7f6bc00c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc4d57640 1 -- 172.21.15.2:0/2963542068 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204141024 0 0) 0x7f6bc0159760 con 0x7f6bc00c3000 2024-09-19T05:44:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.483+0000 7f6bc655a640 1 -- 172.21.15.2:0/2963542068 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc00c2c30 con 0x7f6bc00c62a0 2024-09-19T05:44:04.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.484+0000 7f6bc655a640 1 --2- 172.21.15.2:0/2963542068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c62a0 0x7f6bc006f4c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0073f90 tx=0x7f6bb802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b5df8acbf9dae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.484+0000 7f6bc4d57640 1 -- 172.21.15.2:0/2963542068 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8007740 con 0x7f6bc00c62a0 2024-09-19T05:44:04.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.484+0000 7f6bc4d57640 1 -- 172.21.15.2:0/2963542068 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb80078e0 con 0x7f6bc00c62a0 2024-09-19T05:44:04.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.484+0000 7f6bc4d57640 1 -- 172.21.15.2:0/2963542068 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8007cb0 con 0x7f6bc00c62a0 2024-09-19T05:44:04.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/2963542068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c62a0 msgr2=0x7f6bc006f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/2963542068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c62a0 0x7f6bc006f4c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0073f90 tx=0x7f6bb802a6a0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/2963542068 shutdown_connections 2024-09-19T05:44:04.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/2963542068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c62a0 0x7f6bc006f4c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/2963542068 >> v2:172.21.15.123:3300/0 conn(0x7f6bc0152ce0 0x7f6bc006ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/2963542068 >> 172.21.15.2:0/2963542068 conn(0x7f6bc00cc0d0 msgr2=0x7f6bc00cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/2963542068 shutdown_connections 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.485+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/2963542068 wait complete. 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.486+0000 7f6bc7fbd640 1 Processor -- start 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.486+0000 7f6bc7fbd640 1 -- start start 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc7fbd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c3000 0x7f6bc0154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc7fbd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc00c62a0 0x7f6bc01e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc7fbd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc0152ce0 0x7f6bc0158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc7fbd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc006ea60 con 0x7f6bc00c3000 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc7fbd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc006e780 con 0x7f6bc0152ce0 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc7fbd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc00c4690 con 0x7f6bc00c62a0 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c3000 0x7f6bc0154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c3000 0x7f6bc0154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55124/0 (socket says 172.21.15.2:55124) 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 -- 172.21.15.2:0/1852975820 learned_addr learned my addr 172.21.15.2:0/1852975820 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc00c62a0 0x7f6bc01e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc655a640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc0152ce0 0x7f6bc0158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc00c62a0 msgr2=0x7f6bc01e1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc00c62a0 0x7f6bc01e1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc0152ce0 msgr2=0x7f6bc0158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bc0152ce0 0x7f6bc0158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc01e2160 con 0x7f6bc00c3000 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6bc00c62a0 0x7f6bc01e1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c3000 0x7f6bc0154470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00e8a0 tx=0x7f6bbc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.487+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc017070 con 0x7f6bc00c3000 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.488+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc00c940 con 0x7f6bc00c3000 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.488+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc0155030 con 0x7f6bc00c3000 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.488+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00cd30 con 0x7f6bc00c3000 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.488+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc0154d50 con 0x7f6bc00c3000 2024-09-19T05:44:04.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.488+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f6bc00c67c0 con 0x7f6bc00c3000 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.489+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6bbc01a070 con 0x7f6bc00c3000 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.489+0000 7f6bb67fc640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2b0 0x7f6ba406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f6bbc08b340 con 0x7f6bc00c3000 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v56) ==== 54+0+16 (secure 0 0 0) 0x7f6bbc08d8e0 con 0x7f6bc00c3000 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2b0 0x7f6ba406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6bc00ca350 0x7f6bc01f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bc0159760 con 0x7f6bc00ca350 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bc655a640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6bc00ca350 0x7f6bc01f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2b0 0x7f6ba406e770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00c4390 tx=0x7f6bb0007c30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.490+0000 7f6bc655a640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6bc00ca350 0x7f6bc01f19f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.491+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc0159760 con 0x7f6bc00ca350 2024-09-19T05:44:04.504 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:04.504 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:32 3.31 3:8d4d39bf:::10000000002.0000001d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.121340676000001, 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48911290200000002, 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.505 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 32 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366127+0000", 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429999999999999e-06 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388967+0000", 2024-09-19T05:44:04.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022840284999999998 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388978+0000", 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0811e-05 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388983+0000", 2024-09-19T05:44:04.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7879999999999997e-06 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389069+0000", 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6670999999999997e-05 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389303+0000", 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023413399999999999 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389482+0000", 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017864399999999999 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394972+0000", 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054895530000000003 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855135+0000", 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46016378400000002 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855197+0000", 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1365999999999995e-05 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855237+0000", 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9823000000000001e-05 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.511 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:50 3.17 3:eaf1bb0b:::10000000002.0000002f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.098458344000001, 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15463449900000001, 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389008+0000", 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414228+0000", 2024-09-19T05:44:04.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025220256 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414244+0000", 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5821999999999999e-05 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414262+0000", 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7750000000000001e-05 2024-09-19T05:44:04.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414324+0000", 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2250999999999997e-05 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414475+0000", 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015051800000000001 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414621+0000", 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014692000000000001 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.419434+0000", 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048125429999999999 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543566+0000", 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12413210600000001 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543602+0000", 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5420000000000003e-05 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543640+0000", 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8976000000000001e-05 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:88 3.31 3:8c5098ad:::10000000002.00000055:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.029195099000001, 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41301092299999997, 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 88 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:04.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:04.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458271+0000", 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.804e-06 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481919+0000", 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.02364809 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481946+0000", 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7370000000000001e-05 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481957+0000", 2024-09-19T05:44:04.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.066e-05 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482050+0000", 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2991999999999993e-05 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482206+0000", 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000155611 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482359+0000", 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000153728 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.485588+0000", 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003228557 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871172+0000", 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.38558421500000001 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871237+0000", 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4562999999999998e-05 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871280+0000", 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3333e-05 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.527 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:142 3.31 3:8d07500b:::10000000002.0000008b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.875735003999999, 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26228585799999998, 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.528 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 142 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611732+0000", 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8870000000000001e-06 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623428+0000", 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011695743 2024-09-19T05:44:04.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623449+0000", 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1178e-05 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623463+0000", 2024-09-19T05:44:04.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3923e-05 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623546+0000", 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3090999999999994e-05 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623713+0000", 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000167176 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623874+0000", 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016018299999999999 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.628673+0000", 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047995529999999998 2024-09-19T05:44:04.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.873957+0000", 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24528372500000001 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.873991+0000", 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4663999999999999e-05 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.874015+0000", 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3734999999999999e-05 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:173 3.38 3:1ee175b9:::10000000002.000000aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.839693866999999, 2024-09-19T05:44:04.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24937071899999999, 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 173 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.537 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:04.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647773+0000", 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3870000000000002e-06 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659807+0000", 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012033909000000001 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659823+0000", 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6667000000000002e-05 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659850+0000", 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6798e-05 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659930+0000", 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0041000000000002e-05 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.660057+0000", 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127164 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.660179+0000", 2024-09-19T05:44:04.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012194 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.664305+0000", 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041255900000000002 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897061+0000", 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.23275601800000001 2024-09-19T05:44:04.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897101+0000", 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0156e-05 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897141+0000", 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0049000000000002e-05 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:288 3.31 3:8d4d39bf:::10000000002.0000001d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8392077970000003, 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54072771799999997, 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.544 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 288 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648258+0000", 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.686e-06 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673526+0000", 2024-09-19T05:44:04.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025268142 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673552+0000", 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5500999999999999e-05 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673562+0000", 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.825e-06 2024-09-19T05:44:04.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673673+0000", 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011167000000000001 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673778+0000", 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104987 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673894+0000", 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115639 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.677601+0000", 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037067900000000002 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188877+0000", 2024-09-19T05:44:04.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.51127632700000003 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188935+0000", 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7553000000000003e-05 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188984+0000", 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9598000000000002e-05 2024-09-19T05:44:04.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:306 3.17 3:eaf1bb0b:::10000000002.0000002f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8138680019999995, 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17070818600000001, 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.551 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 306 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673598+0000", 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1129999999999999e-06 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692350+0000", 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018751790000000001 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692373+0000", 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2629999999999998e-05 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692385+0000", 2024-09-19T05:44:04.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1905999999999999e-05 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692488+0000", 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010292499999999999 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692599+0000", 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011170100000000001 2024-09-19T05:44:04.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692750+0000", 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151099 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.697776+0000", 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050251590000000004 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844188+0000", 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.146412135 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844251+0000", 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3523999999999997e-05 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844305+0000", 2024-09-19T05:44:04.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3204000000000001e-05 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:319 3.17 3:ead2f1a2:::10000000002.0000003c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7950547669999999, 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22292820499999999, 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.558 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.559 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.559 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.559 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.559 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 319 2024-09-19T05:44:04.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.559 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692412+0000", 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6900000000000001e-06 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708570+0000", 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016157659000000001 2024-09-19T05:44:04.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708591+0000", 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1260999999999999e-05 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708602+0000", 2024-09-19T05:44:04.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708663+0000", 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1811999999999997e-05 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708741+0000", 2024-09-19T05:44:04.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7385999999999994e-05 2024-09-19T05:44:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708853+0000", 2024-09-19T05:44:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112642 2024-09-19T05:44:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715058+0000", 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062048729999999996 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915221+0000", 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.200162954 2024-09-19T05:44:04.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915283+0000", 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2059999999999999e-05 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915338+0000", 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4487999999999997e-05 2024-09-19T05:44:04.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:292 3.3b e54/20 3:ddac2f4e:::10000000002.00000021:head v 54'6, mlcod=53'4)", 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7891547589999997, 2024-09-19T05:44:04.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.40806384200000001, 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:04.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698311+0000", 2024-09-19T05:44:04.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3880000000000001e-06 2024-09-19T05:44:04.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103237+0000", 2024-09-19T05:44:04.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.40492591100000003 2024-09-19T05:44:04.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103250+0000", 2024-09-19T05:44:04.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3593e-05 2024-09-19T05:44:04.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103261+0000", 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0181e-05 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103332+0000", 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.114e-05 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103357+0000", 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5094000000000001e-05 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.106302+0000", 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029454260000000001 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.106373+0000", 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1109e-05 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:344 3.31 3:8c5098ad:::10000000002.00000055:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7608090519999999, 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.47092658999999998, 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.574 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 344 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726658+0000", 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.649e-06 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758353+0000", 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031695424999999999 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758372+0000", 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8723e-05 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758387+0000", 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4882000000000001e-05 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758510+0000", 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012282100000000001 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758628+0000", 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118544 2024-09-19T05:44:04.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758867+0000", 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0002385 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.765925+0000", 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070580670000000003 2024-09-19T05:44:04.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197522+0000", 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.43159685599999997 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197553+0000", 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1319999999999998e-05 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197582+0000", 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8802999999999998e-05 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:358 3.17 3:eaf05c3c:::10000000002.00000063:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7072741469999997, 2024-09-19T05:44:04.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30462805700000001, 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 358 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780193+0000", 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5660000000000001e-06 2024-09-19T05:44:04.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804477+0000", 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024284329 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804498+0000", 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1251e-05 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804511+0000", 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2234e-05 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804595+0000", 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4651e-05 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804686+0000", 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1062000000000003e-05 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804937+0000", 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025027200000000001 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.808988+0000", 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040513900000000002 2024-09-19T05:44:04.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084691+0000", 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.275703319 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084761+0000", 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9845999999999995e-05 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084818+0000", 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7136999999999999e-05 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:367 3.17 3:e9c25901:::10000000002.0000006c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.6829253409999998, 2024-09-19T05:44:04.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.35149704900000001, 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 367 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804541+0000", 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9360000000000002e-06 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832876+0000", 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.028334734 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832891+0000", 2024-09-19T05:44:04.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5404e-05 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832899+0000", 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1759999999999993e-06 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832970+0000", 2024-09-19T05:44:04.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000006e-05 2024-09-19T05:44:04.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:04.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8103000000000007e-05 2024-09-19T05:44:04.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833196+0000", 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138131 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837012+0000", 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038158770000000001 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.155960+0000", 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.318948334 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.155994+0000", 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3630000000000002e-05 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156036+0000", 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2224000000000001e-05 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.596 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:390 3.17 3:e85118c5:::10000000002.00000083:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.6164796040000002, 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28513696999999999, 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.597 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 390 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.871040+0000", 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5596000000000001e-05 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883834+0000", 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012794005000000001 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883858+0000", 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3380999999999999e-05 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883869+0000", 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1245e-05 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883964+0000", 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4749999999999999e-05 2024-09-19T05:44:04.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.884049+0000", 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5023000000000002e-05 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.884176+0000", 2024-09-19T05:44:04.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012745500000000001 2024-09-19T05:44:04.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.887612+0000", 2024-09-19T05:44:04.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034360850000000002 2024-09-19T05:44:04.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156071+0000", 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26845824099999999 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156100+0000", 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9368000000000001e-05 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156122+0000", 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1821e-05 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.605 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:392 3.1e 3:7b5aaf6d:::10000000002.00000085:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.6035673959999999, 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.25285440300000001, 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.606 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 392 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883899+0000", 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4399999999999999e-06 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896769+0000", 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012869307 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896783+0000", 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896793+0000", 2024-09-19T05:44:04.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0182e-05 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896887+0000", 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3704999999999998e-05 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.897039+0000", 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015165299999999999 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.897229+0000", 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000190552 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.901089+0000", 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038593680000000002 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136638+0000", 2024-09-19T05:44:04.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.235549007 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136703+0000", 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5030999999999995e-05 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136751+0000", 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8798000000000003e-05 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.612 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:398 3.31 3:8d07500b:::10000000002.0000008b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5906350620000005, 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.326241485, 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.613 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 398 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896832+0000", 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6410000000000002e-06 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908746+0000", 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011914118 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908772+0000", 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6149999999999999e-05 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908789+0000", 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6322000000000001e-05 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908897+0000", 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010891 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908999+0000", 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101216 2024-09-19T05:44:04.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.909146+0000", 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014757099999999999 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.913702+0000", 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004555875 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.222963+0000", 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30926109600000001 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223026+0000", 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3182999999999995e-05 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223071+0000", 2024-09-19T05:44:04.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4403000000000002e-05 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:429 3.38 3:1ee175b9:::10000000002.000000aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5445981250000003, 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.23704366099999999, 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 429 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942868+0000", 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5319999999999999e-06 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957357+0000", 2024-09-19T05:44:04.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014489149 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957373+0000", 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6575e-05 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957381+0000", 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6410000000000002e-06 2024-09-19T05:44:04.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957443+0000", 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2200999999999996e-05 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957560+0000", 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117044 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957686+0000", 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012538100000000001 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.961805+0000", 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041189649999999996 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179828+0000", 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21802357999999999 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179875+0000", 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6609e-05 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179910+0000", 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4984000000000003e-05 2024-09-19T05:44:04.626 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.627 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.627 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.627 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:434 3.1e 3:789394ea:::10000000002.000000af:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.627 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:04.627 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5161218080000003, 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24320008700000001, 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 434 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971344+0000", 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.598e-06 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985502+0000", 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014158082000000001 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985526+0000", 2024-09-19T05:44:04.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4085000000000001e-05 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985537+0000", 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1143e-05 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985630+0000", 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2417000000000006e-05 2024-09-19T05:44:04.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985741+0000", 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011084 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985917+0000", 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017616499999999999 2024-09-19T05:44:04.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.991540+0000", 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056230630000000002 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214462+0000", 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22292171499999999 2024-09-19T05:44:04.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214510+0000", 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8154e-05 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214543+0000", 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2824999999999998e-05 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:472 3.31 3:8c3036ad:::10000000002.000000d5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.4186355129999999, 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.154846594, 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.635 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 472 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068831+0000", 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0140000000000001e-06 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078582+0000", 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097511360000000005 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078597+0000", 2024-09-19T05:44:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5194000000000001e-05 2024-09-19T05:44:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078605+0000", 2024-09-19T05:44:04.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0860000000000002e-06 2024-09-19T05:44:04.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078689+0000", 2024-09-19T05:44:04.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4184999999999994e-05 2024-09-19T05:44:04.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078786+0000", 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6094000000000005e-05 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078912+0000", 2024-09-19T05:44:04.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126749 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.083288+0000", 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004375421 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223574+0000", 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.140286203 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223628+0000", 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4401000000000003e-05 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223675+0000", 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7110999999999997e-05 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.644 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:496 3.1e 3:7b5946a1:::10000000002.000000ed:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3936877489999997, 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15292847600000001, 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 496 2024-09-19T05:44:04.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093778+0000", 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.319e-06 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099527+0000", 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057487959999999996 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099543+0000", 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6767000000000001e-05 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099552+0000", 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0919999999999998e-06 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099633+0000", 2024-09-19T05:44:04.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1337999999999999e-05 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099734+0000", 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101005 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099877+0000", 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000143067 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103900+0000", 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040226250000000002 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246621+0000", 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.142720927 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246660+0000", 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9567000000000003e-05 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246705+0000", 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4972999999999998e-05 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:500 3.29 3:96fc3913:::10000000002.000000f1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:04.651 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3868554670000002, 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.153671691, 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 500 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.652 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100612+0000", 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.055e-06 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108795+0000", 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081828820000000007 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108807+0000", 2024-09-19T05:44:04.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2468e-05 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108815+0000", 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3690000000000001e-06 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108887+0000", 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2509000000000007e-05 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108964+0000", 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6787999999999997e-05 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.109095+0000", 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013103499999999999 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.112337+0000", 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032419950000000001 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254185+0000", 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.141847683 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254240+0000", 2024-09-19T05:44:04.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5702999999999998e-05 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254281+0000", 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0204000000000003e-05 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:04.658 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:04.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:04.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.505+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc006ffe0 con 0x7f6bc00ca350 2024-09-19T05:44:04.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.507+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc006ffe0 con 0x7f6bc00ca350 2024-09-19T05:44:04.661 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.507+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f6bc01f2570 0x7f6bc0153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.507+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6bc0151930 con 0x7f6bc01f2570 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.507+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f6bc01f2570 0x7f6bc0153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.507+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f6bc01f2570 0x7f6bc0153140 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.508+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc0151930 con 0x7f6bc01f2570 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.522+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc0153680 con 0x7f6bc01f2570 2024-09-19T05:44:04.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.524+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc0153680 con 0x7f6bc01f2570 2024-09-19T05:44:04.663 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.524+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6bc014caa0 0x7f6bc014cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.524+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f6bc014d3e0 con 0x7f6bc014caa0 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.524+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6bc014caa0 0x7f6bc014cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.524+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6bc014caa0 0x7f6bc014cea0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.525+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc014d3e0 con 0x7f6bc014caa0 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.539+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc014d860 con 0x7f6bc014caa0 2024-09-19T05:44:04.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.541+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc014d860 con 0x7f6bc014caa0 2024-09-19T05:44:04.665 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.541+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6bc0062f00 0x7f6bc01e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.541+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f6bc0063380 con 0x7f6bc0062f00 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.541+0000 7f6bc655a640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6bc0062f00 0x7f6bc01e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.541+0000 7f6bc655a640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6bc0062f00 0x7f6bc01e4f60 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.542+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc0063380 con 0x7f6bc0062f00 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.556+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc01e5770 con 0x7f6bc0062f00 2024-09-19T05:44:04.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.557+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc01e5770 con 0x7f6bc0062f00 2024-09-19T05:44:04.668 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:04.669 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:04.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.558+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f6bc002c640 0x7f6bc002caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.558+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f6bc0063590 con 0x7f6bc002c640 2024-09-19T05:44:04.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.558+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f6bc002c640 0x7f6bc002caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.558+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f6bc002c640 0x7f6bc002caa0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.559+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc0063590 con 0x7f6bc002c640 2024-09-19T05:44:04.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.573+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc0063530 con 0x7f6bc002c640 2024-09-19T05:44:04.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.575+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc0063530 con 0x7f6bc002c640 2024-09-19T05:44:04.670 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:04.671 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:04.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.575+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6bc002d4f0 0x7f6bc0204f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.575+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f6bc002d970 con 0x7f6bc002d4f0 2024-09-19T05:44:04.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.575+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6bc002d4f0 0x7f6bc0204f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.575+0000 7f6bc5558640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6bc002d4f0 0x7f6bc0204f00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.576+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc002d970 con 0x7f6bc002d4f0 2024-09-19T05:44:04.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.590+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc002d970 con 0x7f6bc002d4f0 2024-09-19T05:44:04.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.591+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc002d970 con 0x7f6bc002d4f0 2024-09-19T05:44:04.673 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:04.674 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:04.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.592+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6bc0205950 0x7f6bc0207d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.592+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f6bc020ab10 con 0x7f6bc0205950 2024-09-19T05:44:04.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.592+0000 7f6bc655a640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6bc0205950 0x7f6bc0207d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.592+0000 7f6bc655a640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6bc0205950 0x7f6bc0207d40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.593+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc020ab10 con 0x7f6bc0205950 2024-09-19T05:44:04.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.607+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc0208280 con 0x7f6bc0205950 2024-09-19T05:44:04.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.609+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc0208280 con 0x7f6bc0205950 2024-09-19T05:44:04.675 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:04.677 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:04.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.609+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6bc02089a0 0x7f6bc0208da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:04.677 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.609+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f6bc020df00 con 0x7f6bc02089a0 2024-09-19T05:44:04.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.609+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6bc02089a0 0x7f6bc0208da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:04.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.609+0000 7f6bc5d59640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6bc02089a0 0x7f6bc0208da0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:04.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.610+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc020df00 con 0x7f6bc02089a0 2024-09-19T05:44:04.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.623+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6bc0209350 con 0x7f6bc02089a0 2024-09-19T05:44:04.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bb67fc640 1 -- 172.21.15.2:0/1852975820 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6bc0209350 con 0x7f6bc02089a0 2024-09-19T05:44:04.678 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6bc00ca350 msgr2=0x7f6bc01f19f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6bc00ca350 0x7f6bc01f19f0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f6bc01f2570 msgr2=0x7f6bc0153140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f6bc01f2570 0x7f6bc0153140 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6bc014caa0 msgr2=0x7f6bc014cea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6bc014caa0 0x7f6bc014cea0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6bc0062f00 msgr2=0x7f6bc01e4f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6bc0062f00 0x7f6bc01e4f60 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f6bc002c640 msgr2=0x7f6bc002caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f6bc002c640 0x7f6bc002caa0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.625+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6bc002d4f0 msgr2=0x7f6bc0204f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f6bc002d4f0 0x7f6bc0204f00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc655a640 1 -- 172.21.15.2:0/1852975820 reap_dead start 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6bc0205950 msgr2=0x7f6bc0207d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6bc0205950 0x7f6bc0207d40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc655a640 1 -- 172.21.15.2:0/1852975820 reap_dead start 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6bc02089a0 msgr2=0x7f6bc0208da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc655a640 1 -- 172.21.15.2:0/1852975820 reap_dead start 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc655a640 1 -- 172.21.15.2:0/1852975820 reap_dead start 2024-09-19T05:44:04.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6bc02089a0 0x7f6bc0208da0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2b0 msgr2=0x7f6ba406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2b0 0x7f6ba406e770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00c4390 tx=0x7f6bb0007c30 comp rx=0 tx=0).stop 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c3000 msgr2=0x7f6bc0154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c3000 0x7f6bc0154470 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00e8a0 tx=0x7f6bbc00ed60 comp rx=0 tx=0).stop 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 shutdown_connections 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6bc02089a0 0x7f6bc0208da0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6ba406c2b0 0x7f6ba406e770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 --2- 172.21.15.2:0/1852975820 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc00c3000 0x7f6bc0154470 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 >> 172.21.15.2:0/1852975820 conn(0x7f6bc00cc0d0 msgr2=0x7f6bc00c8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 shutdown_connections 2024-09-19T05:44:04.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.626+0000 7f6bc7fbd640 1 -- 172.21.15.2:0/1852975820 wait complete. 2024-09-19T05:44:04.684 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:04.658+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:04.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.684+0000 7f340effd640 1 -- 172.21.15.2:0/214242037 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v57) ==== 97+0+0 (secure 0 0 0) 0x7f34080587a0 con 0x7f341815c530 2024-09-19T05:44:04.686 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33f806c2b0 msgr2=0x7f33f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33f806c2b0 0x7f33f806e770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3410034000 tx=0x7f341002f000 comp rx=0 tx=0).stop 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c530 msgr2=0x7f34181f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c530 0x7f34181f1850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f340800c480 tx=0x7f340800c940 comp rx=0 tx=0).stop 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 shutdown_connections 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f33f806c2b0 0x7f33f806e770 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f341815c530 0x7f34181f1850 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f341815c150 0x7f3418158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 --2- 172.21.15.2:0/214242037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3418152b70 0x7f3418157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:04.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 >> 172.21.15.2:0/214242037 conn(0x7f341806bc10 msgr2=0x7f341814fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:04.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 shutdown_connections 2024-09-19T05:44:04.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:04.686+0000 7f341cf8d640 1 -- 172.21.15.2:0/214242037 wait complete. 2024-09-19T05:44:04.762 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:04.760+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:04.765 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:04.765 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:04.775 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:04.863 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:04.861+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:04.879 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:04.879 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:04.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:04.963 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:04.962+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:04.991 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:04.992 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:05.064 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:05.063+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:05.104 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.104 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:05.166 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:05.164+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:05.256 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.257 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:05.266 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:05.265+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:05.367 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:05.365+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:05.368 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.368 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.376 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:05.469 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:05.468+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:05.490 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.490 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:05.571 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:05.569+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.610 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.618 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:05.670 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:05.669+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:05.732 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.733 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.741 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:05.771 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:05.769+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:05.849 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.850 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.859 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:05.872 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:05.870+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:05.966 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:05.966 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:05.972 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:05.971+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:05.975 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:06.073 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:06.071+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:06.083 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:06.083 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:06.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:06.138 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:44:06.138 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:44:06.138 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-19T05:44:06.138 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-09-19T05:44:06.138 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0 2024-09-19T05:44:06.173 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:06.172+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:06.200 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:06.200 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:06.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:06.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.242+0000 7f06e135b640 1 Processor -- start 2024-09-19T05:44:06.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.242+0000 7f06e135b640 1 -- start start 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06e135b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f06dc072ed0 0x7f06dc070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06e135b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f06dc0ca5b0 0x7f06dc152150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06e135b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06dc0c82f0 con 0x7f06dc0732b0 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06e135b640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f06dc0c9900 con 0x7f06dc0ca5b0 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06e135b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f06dc0c8590 con 0x7f06dc072ed0 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06db7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f06dc0ca5b0 0x7f06dc152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06da7fc640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f06dc0732b0 0x7f06dc0c9ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55578/0 (socket says 172.21.15.2:55578) 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06db7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f06dc0ca5b0 0x7f06dc152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55134/0 (socket says 172.21.15.2:55134) 2024-09-19T05:44:06.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06da7fc640 1 -- 172.21.15.2:0/2002437012 learned_addr learned my addr 172.21.15.2:0/2002437012 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:06.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.243+0000 7f06daffd640 1 --2- 172.21.15.2:0/2002437012 >> v2:172.21.15.123:3300/0 conn(0x7f06dc072ed0 0x7f06dc070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.244+0000 7f06db7fe640 1 -- 172.21.15.2:0/2002437012 >> v2:172.21.15.123:3300/0 conn(0x7f06dc072ed0 msgr2=0x7f06dc070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:06.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.244+0000 7f06db7fe640 1 --2- 172.21.15.2:0/2002437012 >> v2:172.21.15.123:3300/0 conn(0x7f06dc072ed0 0x7f06dc070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.244+0000 7f06db7fe640 1 -- 172.21.15.2:0/2002437012 >> v1:172.21.15.2:6789/0 conn(0x7f06dc0732b0 legacy=0x7f06dc0c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.244+0000 7f06db7fe640 1 -- 172.21.15.2:0/2002437012 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc0738c0 con 0x7f06dc0ca5b0 2024-09-19T05:44:06.247 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.244+0000 7f06d9ffb640 1 -- 172.21.15.2:0/2002437012 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3867622960 0 0) 0x7f06dc0c82f0 con 0x7f06dc0732b0 2024-09-19T05:44:06.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.244+0000 7f06db7fe640 1 --2- 172.21.15.2:0/2002437012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0ca5b0 0x7f06dc152150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f06dc072090 tx=0x7f06d402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbd292c10c33f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06d9ffb640 1 -- 172.21.15.2:0/2002437012 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4007740 con 0x7f06dc0ca5b0 2024-09-19T05:44:06.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06d9ffb640 1 -- 172.21.15.2:0/2002437012 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d40078e0 con 0x7f06dc0ca5b0 2024-09-19T05:44:06.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06d9ffb640 1 -- 172.21.15.2:0/2002437012 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4007cb0 con 0x7f06dc0ca5b0 2024-09-19T05:44:06.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 -- 172.21.15.2:0/2002437012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0ca5b0 msgr2=0x7f06dc152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 --2- 172.21.15.2:0/2002437012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0ca5b0 0x7f06dc152150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f06dc072090 tx=0x7f06d402a710 comp rx=0 tx=0).stop 2024-09-19T05:44:06.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 -- 172.21.15.2:0/2002437012 shutdown_connections 2024-09-19T05:44:06.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 --2- 172.21.15.2:0/2002437012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0ca5b0 0x7f06dc152150 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 --2- 172.21.15.2:0/2002437012 >> v2:172.21.15.123:3300/0 conn(0x7f06dc072ed0 0x7f06dc070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 -- 172.21.15.2:0/2002437012 >> 172.21.15.2:0/2002437012 conn(0x7f06dc06cd20 msgr2=0x7f06dc06d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:06.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 -- 172.21.15.2:0/2002437012 shutdown_connections 2024-09-19T05:44:06.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.245+0000 7f06e135b640 1 -- 172.21.15.2:0/2002437012 wait complete. 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 Processor -- start 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 -- start start 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06dc072ed0 0x7f06dc0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0732b0 0x7f06dc15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06dc0ca5b0 0x7f06dc1ee250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc074160 con 0x7f06dc0732b0 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc073e80 con 0x7f06dc072ed0 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.247+0000 7f06e135b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f06dc0705c0 con 0x7f06dc0ca5b0 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0732b0 0x7f06dc15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0732b0 0x7f06dc15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55148/0 (socket says 172.21.15.2:55148) 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 -- 172.21.15.2:0/726455849 learned_addr learned my addr 172.21.15.2:0/726455849 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06daffd640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06dc072ed0 0x7f06dc0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06db7fe640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06dc0ca5b0 0x7f06dc1ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 -- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06dc0ca5b0 msgr2=0x7f06dc1ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06dc0ca5b0 0x7f06dc1ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 -- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06dc072ed0 msgr2=0x7f06dc0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06dc072ed0 0x7f06dc0c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 -- 172.21.15.2:0/726455849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc1f2300 con 0x7f06dc0732b0 2024-09-19T05:44:06.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06da7fc640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0732b0 0x7f06dc15b7b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c470 tx=0x7f06c400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06d37fe640 1 -- 172.21.15.2:0/726455849 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4017070 con 0x7f06dc0732b0 2024-09-19T05:44:06.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06d37fe640 1 -- 172.21.15.2:0/726455849 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c4005ce0 con 0x7f06dc0732b0 2024-09-19T05:44:06.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc1f25e0 con 0x7f06dc0732b0 2024-09-19T05:44:06.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06d37fe640 1 -- 172.21.15.2:0/726455849 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c4007310 con 0x7f06dc0732b0 2024-09-19T05:44:06.253 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.248+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc1f2ba0 con 0x7f06dc0732b0 2024-09-19T05:44:06.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.249+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06dc0c7f40 con 0x7f06dc0732b0 2024-09-19T05:44:06.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.250+0000 7f06d37fe640 1 -- 172.21.15.2:0/726455849 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f06c401a070 con 0x7f06dc0732b0 2024-09-19T05:44:06.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.250+0000 7f06d37fe640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06bc06c2b0 0x7f06bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.251+0000 7f06d37fe640 1 -- 172.21.15.2:0/726455849 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f06c408a890 con 0x7f06dc0732b0 2024-09-19T05:44:06.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.251+0000 7f06daffd640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06bc06c2b0 0x7f06bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.254 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.251+0000 7f06daffd640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06bc06c2b0 0x7f06bc06e770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009b40 tx=0x7f06cc009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.257 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.255+0000 7f06d37fe640 1 -- 172.21.15.2:0/726455849 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f06c4058920 con 0x7f06dc0732b0 2024-09-19T05:44:06.274 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:06.272+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:06.321 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:06.321 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:06.330 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:06.375 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:06.373+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:06.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.449+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 0.0} v 0) -- 0x7f06dc0c7560 con 0x7f06dc0732b0 2024-09-19T05:44:06.476 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:06.474+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:06.478 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:06.478 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:06.487 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:06.577 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:06.575+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:06.594 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:06.594 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:06.603 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:06.677 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:06.675+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:06.684 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:06.692 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.690+0000 7f06d37fe640 1 -- 172.21.15.2:0/726455849 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]=0 set osd.1 primary-affinity to 0 (802) v58) ==== 129+0+0 (secure 0 0 0) 0x7f06c405cc20 con 0x7f06dc0732b0 2024-09-19T05:44:06.692 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.1 primary-affinity to 0 (802) 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06bc06c2b0 msgr2=0x7f06bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06bc06c2b0 0x7f06bc06e770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009b40 tx=0x7f06cc009190 comp rx=0 tx=0).stop 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0732b0 msgr2=0x7f06dc15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0732b0 0x7f06dc15b7b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f06c400c470 tx=0x7f06c400c930 comp rx=0 tx=0).stop 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 shutdown_connections 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06bc06c2b0 0x7f06bc06e770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f06dc0ca5b0 0x7f06dc1ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06dc0732b0 0x7f06dc15b7b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 --2- 172.21.15.2:0/726455849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06dc072ed0 0x7f06dc0c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.694+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 >> 172.21.15.2:0/726455849 conn(0x7f06dc06cd20 msgr2=0x7f06dc14cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.695+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 shutdown_connections 2024-09-19T05:44:06.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.695+0000 7f06e135b640 1 -- 172.21.15.2:0/726455849 wait complete. 2024-09-19T05:44:06.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:44:06.720 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:06.720 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:06.730 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:06.778 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:06.776+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:06.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.787+0000 7fb91f627640 1 Processor -- start 2024-09-19T05:44:06.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.788+0000 7fb91f627640 1 -- start start 2024-09-19T05:44:06.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.788+0000 7fb91f627640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb918152740 0x7fb9180c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.788+0000 7fb91f627640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152b20 0x7fb9180c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91f627640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb918071130 con 0x7fb9180ce260 2024-09-19T05:44:06.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91f627640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb918158880 con 0x7fb918152740 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91f627640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb9180c8ec0 con 0x7fb918152b20 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91dbc4640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb9180ce260 0x7fb918158d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55584/0 (socket says 172.21.15.2:55584) 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91dbc4640 1 -- 172.21.15.2:0/322171185 learned_addr learned my addr 172.21.15.2:0/322171185 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/322171185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152b20 0x7fb9180c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/322171185 >> v2:172.21.15.123:3300/0 conn(0x7fb918152740 0x7fb9180c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb90ffff640 1 -- 172.21.15.2:0/322171185 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931251735 0 0) 0x7fb918071130 con 0x7fb9180ce260 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb90ffff640 1 -- 172.21.15.2:0/322171185 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb904003520 con 0x7fb9180ce260 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91cbc2640 1 -- 172.21.15.2:0/322171185 >> v2:172.21.15.123:3300/0 conn(0x7fb918152740 msgr2=0x7fb9180c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/322171185 >> v2:172.21.15.123:3300/0 conn(0x7fb918152740 0x7fb9180c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91cbc2640 1 -- 172.21.15.2:0/322171185 >> v1:172.21.15.2:6789/0 conn(0x7fb9180ce260 legacy=0x7fb918158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91cbc2640 1 -- 172.21.15.2:0/322171185 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb918070560 con 0x7fb918152b20 2024-09-19T05:44:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.789+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/322171185 >> v2:172.21.15.123:3300/0 conn(0x7fb918152740 0x7fb9180c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.790+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/322171185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152b20 0x7fb9180c83c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9080097a0 tx=0x7fb90802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f240f1014223e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.790+0000 7fb90ffff640 1 -- 172.21.15.2:0/322171185 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908037070 con 0x7fb918152b20 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.790+0000 7fb90ffff640 1 -- 172.21.15.2:0/322171185 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb908032440 con 0x7fb918152b20 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.790+0000 7fb90ffff640 1 -- 172.21.15.2:0/322171185 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908032810 con 0x7fb918152b20 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 -- 172.21.15.2:0/322171185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152b20 msgr2=0x7fb9180c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 --2- 172.21.15.2:0/322171185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152b20 0x7fb9180c83c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9080097a0 tx=0x7fb90802a210 comp rx=0 tx=0).stop 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 -- 172.21.15.2:0/322171185 shutdown_connections 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 --2- 172.21.15.2:0/322171185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152b20 0x7fb9180c83c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 --2- 172.21.15.2:0/322171185 >> v2:172.21.15.123:3300/0 conn(0x7fb918152740 0x7fb9180c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 -- 172.21.15.2:0/322171185 >> 172.21.15.2:0/322171185 conn(0x7fb91806c240 msgr2=0x7fb918067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 -- 172.21.15.2:0/322171185 shutdown_connections 2024-09-19T05:44:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.791+0000 7fb91f627640 1 -- 172.21.15.2:0/322171185 wait complete. 2024-09-19T05:44:06.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.792+0000 7fb91f627640 1 Processor -- start 2024-09-19T05:44:06.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.792+0000 7fb91f627640 1 -- start start 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91f627640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9180ce260 0x7fb9180c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91f627640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152740 0x7fb91814db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91f627640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb918152b20 0x7fb91814e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91f627640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb918070200 con 0x7fb9180ce260 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91f627640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb918070e50 con 0x7fb918152b20 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91f627640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb918159420 con 0x7fb918152740 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91d3c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9180ce260 0x7fb9180c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91d3c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9180ce260 0x7fb9180c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55156/0 (socket says 172.21.15.2:55156) 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.793+0000 7fb91d3c3640 1 -- 172.21.15.2:0/3243234442 learned_addr learned my addr 172.21.15.2:0/3243234442 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91dbc4640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb918152b20 0x7fb91814e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152740 0x7fb91814db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91d3c3640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152740 msgr2=0x7fb91814db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb918152740 0x7fb91814db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91d3c3640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb918152b20 msgr2=0x7fb91814e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb918152b20 0x7fb91814e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91d3c3640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9181f36b0 con 0x7fb9180ce260 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9180ce260 0x7fb9180c69d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb91400ec10 tx=0x7fb91400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914017070 con 0x7fb9180ce260 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91400cc30 con 0x7fb9180ce260 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914013470 con 0x7fb9180ce260 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91814d0a0 con 0x7fb9180ce260 2024-09-19T05:44:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.794+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91814d660 con 0x7fb9180ce260 2024-09-19T05:44:06.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.795+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb9180c51b0 con 0x7fb9180ce260 2024-09-19T05:44:06.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.797+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb91401a070 con 0x7fb9180ce260 2024-09-19T05:44:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.797+0000 7fb90dffb640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb8fc06c2b0 0x7fb8fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.797+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5741+0+0 (secure 0 0 0) 0x7fb91408b6c0 con 0x7fb9180ce260 2024-09-19T05:44:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.797+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v58) ==== 54+0+16 (secure 0 0 0) 0x7fb914091170 con 0x7fb9180ce260 2024-09-19T05:44:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.797+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb8fc06c2b0 0x7fb8fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.797+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb91814aba0 0x7fb91814af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.797+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb91814b4c0 con 0x7fb91814aba0 2024-09-19T05:44:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.798+0000 7fb91dbc4640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb91814aba0 0x7fb91814af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.798+0000 7fb91dbc4640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb91814aba0 0x7fb91814af80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.798+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb8fc06c2b0 0x7fb8fc06e770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb908034000 tx=0x7fb90802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.799+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb91814b4c0 con 0x7fb91814aba0 2024-09-19T05:44:06.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.807+0000 7fbd5461e640 1 Processor -- start 2024-09-19T05:44:06.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.807+0000 7fbd5461e640 1 -- start start 2024-09-19T05:44:06.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd5461e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbd4c152b90 0x7fbd4c0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd5461e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c06bcb0 con 0x7fbd4c152f70 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd5461e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c06bf50 con 0x7fbd4c158f70 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd5461e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fbd4c06c230 con 0x7fbd4c152b90 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd523ba640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbd4c152b90 0x7fbd4c0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd523ba640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbd4c152b90 0x7fbd4c0c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55162/0 (socket says 172.21.15.2:55162) 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd51bb9640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fbd4c152f70 0x7fbd4c0c6b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55598/0 (socket says 172.21.15.2:55598) 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.808+0000 7fbd523ba640 1 -- 172.21.15.2:0/2132161786 learned_addr learned my addr 172.21.15.2:0/2132161786 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.809+0000 7fbd523ba640 1 -- 172.21.15.2:0/2132161786 >> v1:172.21.15.123:6789/0 conn(0x7fbd4c158f70 legacy=0x7fbd4c1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.809+0000 7fbd523ba640 1 -- 172.21.15.2:0/2132161786 >> v1:172.21.15.2:6789/0 conn(0x7fbd4c152f70 legacy=0x7fbd4c0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.809+0000 7fbd523ba640 1 -- 172.21.15.2:0/2132161786 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c06c510 con 0x7fbd4c152b90 2024-09-19T05:44:06.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.809+0000 7fbd523ba640 1 --2- 172.21.15.2:0/2132161786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c152b90 0x7fbd4c0c7e60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009870 tx=0x7fbd4402a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad6d3ac906ccf52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd513b8640 1 -- 172.21.15.2:0/2132161786 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44037070 con 0x7fbd4c152b90 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd513b8640 1 -- 172.21.15.2:0/2132161786 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd44032640 con 0x7fbd4c152b90 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd513b8640 1 -- 172.21.15.2:0/2132161786 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44032a10 con 0x7fbd4c152b90 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd5461e640 1 -- 172.21.15.2:0/2132161786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c152b90 msgr2=0x7fbd4c0c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd5461e640 1 --2- 172.21.15.2:0/2132161786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c152b90 0x7fbd4c0c7e60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009870 tx=0x7fbd4402a3c0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd5461e640 1 -- 172.21.15.2:0/2132161786 shutdown_connections 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd5461e640 1 --2- 172.21.15.2:0/2132161786 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c152b90 0x7fbd4c0c7e60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd5461e640 1 -- 172.21.15.2:0/2132161786 >> 172.21.15.2:0/2132161786 conn(0x7fbd4c06f3d0 msgr2=0x7fbd4c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd5461e640 1 -- 172.21.15.2:0/2132161786 shutdown_connections 2024-09-19T05:44:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.810+0000 7fbd5461e640 1 -- 172.21.15.2:0/2132161786 wait complete. 2024-09-19T05:44:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.812+0000 7fbd5461e640 1 Processor -- start 2024-09-19T05:44:06.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.812+0000 7fbd5461e640 1 -- start start 2024-09-19T05:44:06.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd5461e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd4c152b90 0x7fbd4c1f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd5461e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c152f70 0x7fbd4c14f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd5461e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c158f70 0x7fbd4c1ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd5461e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c0c88b0 con 0x7fbd4c158f70 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd5461e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1589e0 con 0x7fbd4c152f70 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd5461e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c0c8a30 con 0x7fbd4c152b90 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c158f70 0x7fbd4c1ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c158f70 0x7fbd4c1ef6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55172/0 (socket says 172.21.15.2:55172) 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 -- 172.21.15.2:0/810637009 learned_addr learned my addr 172.21.15.2:0/810637009 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd51bb9640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c152f70 0x7fbd4c14f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd523ba640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd4c152b90 0x7fbd4c1f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 -- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd4c152b90 msgr2=0x7fbd4c1f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd4c152b90 0x7fbd4c1f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 -- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c152f70 msgr2=0x7fbd4c14f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c152f70 0x7fbd4c14f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:06.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 -- 172.21.15.2:0/810637009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c15cd80 con 0x7fbd4c158f70 2024-09-19T05:44:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.813+0000 7fbd52bbb640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c158f70 0x7fbd4c1ef6e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c3f0 tx=0x7fbd3c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.814+0000 7fbd4affd640 1 -- 172.21.15.2:0/810637009 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c017070 con 0x7fbd4c158f70 2024-09-19T05:44:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.814+0000 7fbd4affd640 1 -- 172.21.15.2:0/810637009 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd3c0052d0 con 0x7fbd4c158f70 2024-09-19T05:44:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.814+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c1f2380 con 0x7fbd4c158f70 2024-09-19T05:44:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.814+0000 7fbd4affd640 1 -- 172.21.15.2:0/810637009 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c003850 con 0x7fbd4c158f70 2024-09-19T05:44:06.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.814+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c1454d0 con 0x7fbd4c158f70 2024-09-19T05:44:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.815+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c0c9480 con 0x7fbd4c158f70 2024-09-19T05:44:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.815+0000 7fbd4affd640 1 -- 172.21.15.2:0/810637009 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbd3c01a070 con 0x7fbd4c158f70 2024-09-19T05:44:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.816+0000 7fbd4affd640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbd3806c2b0 0x7fbd3806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.816+0000 7fbd4affd640 1 -- 172.21.15.2:0/810637009 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5741+0+0 (secure 0 0 0) 0x7fbd3c08a6d0 con 0x7fbd4c158f70 2024-09-19T05:44:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.816+0000 7fbd523ba640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbd3806c2b0 0x7fbd3806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.817+0000 7fbd523ba640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbd3806c2b0 0x7fbd3806e770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd44034000 tx=0x7fbd4402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.820+0000 7fbd4affd640 1 -- 172.21.15.2:0/810637009 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbd3c058eb0 con 0x7fbd4c158f70 2024-09-19T05:44:06.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.830+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb918152520 con 0x7fb91814aba0 2024-09-19T05:44:06.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.833+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb918152520 con 0x7fb91814aba0 2024-09-19T05:44:06.835 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:06.836 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:06.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.833+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb91814c420 0x7fb9181effd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.833+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb91815a910 con 0x7fb91814c420 2024-09-19T05:44:06.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.833+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb91814c420 0x7fb9181effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.834+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb91814c420 0x7fb9181effd0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.834+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb91815a910 con 0x7fb91814c420 2024-09-19T05:44:06.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.849+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb918151c60 con 0x7fb91814c420 2024-09-19T05:44:06.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.851+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb918151c60 con 0x7fb91814c420 2024-09-19T05:44:06.853 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:06.854 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:06.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.851+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb9181f0a20 0x7fb918062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.851+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb9181f0e20 con 0x7fb9181f0a20 2024-09-19T05:44:06.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.851+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb9181f0a20 0x7fb918062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.852+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb9181f0a20 0x7fb918062630 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.852+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9181f0e20 con 0x7fb9181f0a20 2024-09-19T05:44:06.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.867+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb918062e40 con 0x7fb9181f0a20 2024-09-19T05:44:06.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.868+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb918062e40 con 0x7fb9181f0a20 2024-09-19T05:44:06.871 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:06.871 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:06.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.869+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb91815bfd0 0x7fb91815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.869+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb91815c950 con 0x7fb91815bfd0 2024-09-19T05:44:06.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.869+0000 7fb91dbc4640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb91815bfd0 0x7fb91815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.869+0000 7fb91dbc4640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb91815bfd0 0x7fb91815c410 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.870+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb91815c950 con 0x7fb91815bfd0 2024-09-19T05:44:06.878 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:06.876+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:06.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.889+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb91815c950 con 0x7fb91815bfd0 2024-09-19T05:44:06.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.891+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb91815c950 con 0x7fb91815bfd0 2024-09-19T05:44:06.894 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:06.894 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:06.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.892+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb91802cb50 0x7fb91802cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:06.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.892+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb91815c950 con 0x7fb91802cb50 2024-09-19T05:44:06.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.892+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb91802cb50 0x7fb91802cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:06.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.893+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb91802cb50 0x7fb91802cfb0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:06.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.893+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb91815c950 con 0x7fb91802cb50 2024-09-19T05:44:06.901 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:06.901 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:06.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:06.901 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:11 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.610595212, 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53368345399999995, 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11 2024-09-19T05:44:06.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287043+0000", 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294617+0000", 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075741280000000003 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294641+0000", 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3424000000000001e-05 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294656+0000", 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4679e-05 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294841+0000", 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185176 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295155+0000", 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031388199999999997 2024-09-19T05:44:06.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295811+0000", 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065616700000000003 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.308016+0000", 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012205472 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820590+0000", 2024-09-19T05:44:06.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51257333400000005 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820658+0000", 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8836000000000003e-05 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820724+0000", 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5426999999999995e-05 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:18 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.559884209, 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48181817700000001, 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 18 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.908 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337754+0000", 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346234+0000", 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084795919999999993 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346309+0000", 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4950999999999994e-05 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346324+0000", 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5199e-05 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346456+0000", 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013213799999999999 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346703+0000", 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024689500000000001 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346893+0000", 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019002400000000001 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382206+0000", 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035313485999999998 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819443+0000", 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43723664899999998 2024-09-19T05:44:06.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819516+0000", 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3021000000000004e-05 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819570+0000", 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3696999999999998e-05 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:37 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.480332807, 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.412251706, 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417306+0000", 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3790000000000002e-06 2024-09-19T05:44:06.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421246+0000", 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039399869999999998 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421273+0000", 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6412999999999999e-05 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421284+0000", 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1856e-05 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421374+0000", 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9411000000000002e-05 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429518+0000", 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008143908 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429741+0000", 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022306100000000001 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.460095+0000", 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030354269999999999 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829462+0000", 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36936722900000002 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829510+0000", 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7698999999999999e-05 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829555+0000", 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4493000000000002e-05 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:53 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.467165416, 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45610759200000001, 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 53 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430472+0000", 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1320000000000001e-06 2024-09-19T05:44:06.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437581+0000", 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071082300000000001 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444289+0000", 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067080070000000002 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444322+0000", 2024-09-19T05:44:06.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3034000000000003e-05 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445189+0000", 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086775100000000005 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445402+0000", 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021237099999999999 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445599+0000", 2024-09-19T05:44:06.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019734099999999999 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501230+0000", 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055630863000000003 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886462+0000", 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38523158200000002 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886525+0000", 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3138000000000004e-05 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886578+0000", 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3143000000000003e-05 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:61 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:06.925 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.440231083, 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.424715008, 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 61 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:06.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457409+0000", 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461813+0000", 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044040060000000002 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461834+0000", 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0789e-05 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461876+0000", 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2268000000000003e-05 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461987+0000", 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110593 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467170+0000", 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051828849999999999 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467417+0000", 2024-09-19T05:44:06.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024757199999999999 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490516+0000", 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023098802000000002 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.881965+0000", 2024-09-19T05:44:06.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39144917200000001 2024-09-19T05:44:06.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882043+0000", 2024-09-19T05:44:06.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7830000000000005e-05 2024-09-19T05:44:06.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882120+0000", 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6360999999999996e-05 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:267 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.3010845539999991, 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52119119999999997, 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 267 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596555+0000", 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.596e-06 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597947+0000", 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001392517 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597975+0000", 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7993000000000001e-05 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597989+0000", 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3927999999999999e-05 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598090+0000", 2024-09-19T05:44:06.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010057500000000001 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598219+0000", 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012952999999999999 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598408+0000", 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018851600000000001 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604394+0000", 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059861159999999997 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117572+0000", 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51317804499999997 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117624+0000", 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2126000000000002e-05 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117742+0000", 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011825800000000001 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:274 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.2854425020000004, 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63191469899999997, 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 274 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612198+0000", 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.476e-06 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619545+0000", 2024-09-19T05:44:06.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073473399999999999 2024-09-19T05:44:06.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619574+0000", 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8994000000000001e-05 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619585+0000", 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1103e-05 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619846+0000", 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026047199999999998 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619979+0000", 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013353099999999999 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.620203+0000", 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000223945 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629556+0000", 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093532560000000008 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244028+0000", 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61447147300000005 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244065+0000", 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6968999999999998e-05 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244108+0000", 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3139999999999997e-05 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:293 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.2439438159999998, 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59051000200000003, 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 293 2024-09-19T05:44:06.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653696+0000", 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.33e-06 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658892+0000", 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051954360000000003 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658913+0000", 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1705999999999999e-05 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658928+0000", 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4361e-05 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659326+0000", 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000398253 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659446+0000", 2024-09-19T05:44:06.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012050200000000001 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659655+0000", 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020868500000000001 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.734308+0000", 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074653061000000007 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244147+0000", 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.50983838800000003 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244175+0000", 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8901999999999999e-05 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244202+0000", 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6378000000000001e-05 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:309 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.2282098850000001, 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.505676974, 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 309 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.955 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669428+0000", 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.593e-06 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.676971+0000", 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075425839999999997 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677009+0000", 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8154000000000001e-05 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677039+0000", 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9788999999999999e-05 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677223+0000", 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000184372 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677347+0000", 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123979 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:06.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677516+0000", 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016852499999999999 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732124+0000", 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054607912000000002 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.174998+0000", 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44287454799999998 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175054+0000", 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5476999999999998e-05 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175103+0000", 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9041e-05 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:313 3.2f 3:f7c72662:::10000000002.00000036:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.960 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.2205091869999993, 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47726204500000002, 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 313 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677129+0000", 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3479999999999998e-06 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685213+0000", 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080837730000000007 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685237+0000", 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4311000000000002e-05 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685254+0000", 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7428999999999999e-05 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863622+0000", 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.178367422 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863724+0000", 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101938 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863861+0000", 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013715 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.152292+0000", 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.288431614 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154270+0000", 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019771250000000001 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154330+0000", 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-19T05:44:06.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154388+0000", 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8717e-05 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:317 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.2122868189999991, 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56821626300000005, 2024-09-19T05:44:06.966 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 317 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685351+0000", 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3549999999999999e-06 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692638+0000", 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072869909999999996 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692666+0000", 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8010000000000001e-05 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692677+0000", 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0776e-05 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692800+0000", 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012327200000000001 2024-09-19T05:44:06.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692936+0000", 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013544100000000001 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.693426+0000", 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00049003600000000001 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732131+0000", 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038705212000000003 2024-09-19T05:44:06.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253472+0000", 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52134096299999999 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253518+0000", 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5731e-05 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253565+0000", 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7475999999999997e-05 2024-09-19T05:44:06.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:331 3.30 3:0d2dbd17:::10000000002.00000048:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.1486225230000002, 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.495291814, 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 331 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749016+0000", 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.754846+0000", 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058302459999999999 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756282+0000", 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001436295 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756297+0000", 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4921e-05 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756370+0000", 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2856999999999997e-05 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756495+0000", 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012511999999999999 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756711+0000", 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216233 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828851+0000", 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072139743000000006 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244223+0000", 2024-09-19T05:44:06.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41537228599999998 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244270+0000", 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244305+0000", 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4527999999999999e-05 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:346 3.11 3:8815798b:::10000000002.00000057:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.1413083739999994, 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49739494099999998, 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 346 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756330+0000", 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759712+0000", 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033821889999999999 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759737+0000", 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5069e-05 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759752+0000", 2024-09-19T05:44:06.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4372999999999999e-05 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759847+0000", 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5291999999999995e-05 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760098+0000", 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025112099999999998 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760422+0000", 2024-09-19T05:44:06.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032433199999999998 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855328+0000", 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094905274999999997 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253622+0000", 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39829475600000003 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253679+0000", 2024-09-19T05:44:06.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6047000000000001e-05 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253722+0000", 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3702999999999998e-05 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:361 3.30 3:0eedefab:::10000000002.00000066:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.1251728550000006, 2024-09-19T05:44:06.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47196730199999998, 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 361 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772466+0000", 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8279999999999999e-06 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777394+0000", 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049281790000000004 2024-09-19T05:44:06.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777413+0000", 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8913999999999999e-05 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777424+0000", 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1192999999999999e-05 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799507+0000", 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022083230999999998 2024-09-19T05:44:06.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799656+0000", 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014921300000000001 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799858+0000", 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020186999999999999 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830773+0000", 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030915148999999999 2024-09-19T05:44:06.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244343+0000", 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41356947900000002 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244392+0000", 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9489e-05 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244430+0000", 2024-09-19T05:44:06.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7756999999999998e-05 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:366 3.11 3:8b4c71c3:::10000000002.0000006b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.120183033, 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47662371999999997, 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 366 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777455+0000", 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7640000000000001e-06 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783373+0000", 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005917533 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783420+0000", 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6709999999999998e-05 2024-09-19T05:44:06.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783443+0000", 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3836999999999999e-05 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808013+0000", 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024569607 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808128+0000", 2024-09-19T05:44:06.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011490299999999999 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808375+0000", 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024700299999999998 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937889+0000", 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951412500000001 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253987+0000", 2024-09-19T05:44:06.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31609763600000002 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254037+0000", 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0383999999999997e-05 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254076+0000", 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9218000000000003e-05 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:06.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:368 3.14 3:280f29a3:::10000000002.0000006d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.1141376560000005, 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45471794900000001, 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 368 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:06.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783504+0000", 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7749999999999998e-06 2024-09-19T05:44:06.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785589+0000", 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020852409999999998 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785606+0000", 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6609999999999999e-05 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785614+0000", 2024-09-19T05:44:06.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4940000000000003e-06 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789369+0000", 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037552250000000001 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789485+0000", 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116019 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789654+0000", 2024-09-19T05:44:06.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016819299999999999 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828885+0000", 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039231901999999999 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238106+0000", 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40922085699999999 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238164+0000", 2024-09-19T05:44:06.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7373000000000001e-05 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238216+0000", 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2259999999999998e-05 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:382 3.11 3:8818af48:::10000000002.0000007b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.1087297350000007, 2024-09-19T05:44:07.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46533816500000003, 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 382 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788911+0000", 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6609999999999999e-06 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796253+0000", 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073421060000000002 2024-09-19T05:44:07.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796284+0000", 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1383000000000001e-05 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796299+0000", 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4793e-05 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832493+0000", 2024-09-19T05:44:07.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036194021 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832612+0000", 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011929 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832778+0000", 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:07.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.066630+0000", 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.233852226 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254144+0000", 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18751385100000001 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254200+0000", 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6028e-05 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:07.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254244+0000", 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4373999999999999e-05 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:403 3.9 3:91a8729a:::10000000002.00000090:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.0810646879999997, 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41822062799999998, 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 403 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816573+0000", 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.345e-06 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.820472+0000", 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038989060000000002 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:07.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827487+0000", 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070152189999999996 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827507+0000", 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9434000000000001e-05 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858059+0000", 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030552598 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858252+0000", 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019281999999999999 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871029+0000", 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012776648999999999 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175256+0000", 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30422741199999997 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234683+0000", 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059426432000000001 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234738+0000", 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4923999999999998e-05 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234792+0000", 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3888999999999999e-05 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:405 3.11 3:8834f03f:::10000000002.00000092:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.0701013699999997, 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42686835699999998, 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 405 2024-09-19T05:44:07.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827537+0000", 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6620000000000001e-06 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830490+0000", 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029526209999999999 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830506+0000", 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6891000000000001e-05 2024-09-19T05:44:07.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830535+0000", 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8377e-05 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855366+0000", 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024831576000000001 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855488+0000", 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012119499999999999 2024-09-19T05:44:07.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855676+0000", 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018832899999999999 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233600+0000", 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37792366900000002 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254308+0000", 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020708684000000002 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254362+0000", 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3430000000000002e-05 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254403+0000", 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0923000000000001e-05 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:07.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:410 3.11 3:8988b152:::10000000002.00000097:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.0670791990000001, 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42399440900000002, 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 410 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.018 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830559+0000", 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-19T05:44:07.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835124+0000", 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045654520000000002 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835145+0000", 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0750999999999998e-05 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835179+0000", 2024-09-19T05:44:07.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4013000000000001e-05 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937947+0000", 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102767884 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938050+0000", 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010270199999999999 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938225+0000", 2024-09-19T05:44:07.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000175494 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234859+0000", 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.296633759 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254459+0000", 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019599979999999999 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:07.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254510+0000", 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1431000000000003e-05 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254551+0000", 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0391e-05 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:07.023 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:07.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.908+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb91815c950 con 0x7fb91802cb50 2024-09-19T05:44:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.910+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb91815c950 con 0x7fb91802cb50 2024-09-19T05:44:07.026 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.910+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fb918204960 0x7fb918206d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.910+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb91802da00 con 0x7fb918204960 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.911+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fb918204960 0x7fb918206d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.911+0000 7fb91cbc2640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fb918204960 0x7fb918206d50 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.912+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb91802da00 con 0x7fb918204960 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.925+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb91802da00 con 0x7fb918204960 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.928+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb91802da00 con 0x7fb918204960 2024-09-19T05:44:07.027 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.928+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb9182077a0 0x7fb918209b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.928+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb91820c960 con 0x7fb9182077a0 2024-09-19T05:44:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.928+0000 7fb91dbc4640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb9182077a0 0x7fb918209b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.928+0000 7fb91dbc4640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb9182077a0 0x7fb918209b90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.929+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb91820c960 con 0x7fb9182077a0 2024-09-19T05:44:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.942+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb91820a0d0 con 0x7fb9182077a0 2024-09-19T05:44:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.945+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb91820a0d0 con 0x7fb9182077a0 2024-09-19T05:44:07.029 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.945+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb91820a750 0x7fb91820cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.945+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb91820fb10 con 0x7fb91820a750 2024-09-19T05:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.945+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb91820a750 0x7fb91820cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.945+0000 7fb91d3c3640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb91820a750 0x7fb91820cb40 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.946+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb91820fb10 con 0x7fb91820a750 2024-09-19T05:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.959+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb918063490 con 0x7fb91820a750 2024-09-19T05:44:07.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb90dffb640 1 -- 172.21.15.2:0/3243234442 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb918063490 con 0x7fb91820a750 2024-09-19T05:44:07.031 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:07.032 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:07.023+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:07.033 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb91814aba0 msgr2=0x7fb91814af80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb91814aba0 0x7fb91814af80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb91814c420 msgr2=0x7fb9181effd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb91814c420 0x7fb9181effd0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb9181f0a20 msgr2=0x7fb918062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb9181f0a20 0x7fb918062630 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb91815bfd0 msgr2=0x7fb91815c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb91815bfd0 0x7fb91815c410 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb91802cb50 msgr2=0x7fb91802cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb91802cb50 0x7fb91802cfb0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fb918204960 msgr2=0x7fb918206d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fb918204960 0x7fb918206d50 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91dbc4640 1 -- 172.21.15.2:0/3243234442 reap_dead start 2024-09-19T05:44:07.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb9182077a0 msgr2=0x7fb918209b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb9182077a0 0x7fb918209b90 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91dbc4640 1 -- 172.21.15.2:0/3243234442 reap_dead start 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb91820a750 msgr2=0x7fb91820cb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb91820a750 0x7fb91820cb40 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91dbc4640 1 -- 172.21.15.2:0/3243234442 reap_dead start 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91dbc4640 1 -- 172.21.15.2:0/3243234442 reap_dead start 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb8fc06c2b0 msgr2=0x7fb8fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb8fc06c2b0 0x7fb8fc06e770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb908034000 tx=0x7fb90802e000 comp rx=0 tx=0).stop 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9180ce260 msgr2=0x7fb9180c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9180ce260 0x7fb9180c69d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb91400ec10 tx=0x7fb91400c520 comp rx=0 tx=0).stop 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 shutdown_connections 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb8fc06c2b0 0x7fb8fc06e770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 --2- 172.21.15.2:0/3243234442 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9180ce260 0x7fb9180c69d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 >> 172.21.15.2:0/3243234442 conn(0x7fb91806c240 msgr2=0x7fb9180cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:07.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 shutdown_connections 2024-09-19T05:44:07.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:06.962+0000 7fb91f627640 1 -- 172.21.15.2:0/3243234442 wait complete. 2024-09-19T05:44:07.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.016+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fbd4c158660 con 0x7fbd4c158f70 2024-09-19T05:44:07.125 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:07.124+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:07.145 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:07.145 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:07.145 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:07.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:07.226 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:07.224+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:07.258 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:07.259 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:07.259 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:07.259 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:07.259 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:07.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:07.353 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:07.352+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:07.413 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:13 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:07.414 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.097126546, 2024-09-19T05:44:07.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41922241599999999, 2024-09-19T05:44:07.415 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.415 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.415 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 13 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:07.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313315+0000", 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.978e-06 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329516+0000", 2024-09-19T05:44:07.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016201601999999999 2024-09-19T05:44:07.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329535+0000", 2024-09-19T05:44:07.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9157e-05 2024-09-19T05:44:07.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329545+0000", 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9909999999999997e-06 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329609+0000", 2024-09-19T05:44:07.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3931999999999997e-05 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329764+0000", 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015456700000000001 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329895+0000", 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013116700000000001 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.336041+0000", 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061460580000000003 2024-09-19T05:44:07.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732421+0000", 2024-09-19T05:44:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.396379484 2024-09-19T05:44:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732484+0000", 2024-09-19T05:44:07.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3100000000000002e-05 2024-09-19T05:44:07.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732535+0000", 2024-09-19T05:44:07.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1379999999999999e-05 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:64 3.2 3:40abce4f:::10000000002.0000003d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.948497059999999, 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29680736000000002, 2024-09-19T05:44:07.425 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 64 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461945+0000", 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6860000000000002e-06 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478410+0000", 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016465418999999999 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478422+0000", 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1732e-05 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478431+0000", 2024-09-19T05:44:07.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8890000000000008e-06 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478503+0000", 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.25e-05 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478662+0000", 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000158696 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478774+0000", 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011198500000000001 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481734+0000", 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002960018 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758602+0000", 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27686762799999998 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758681+0000", 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9566000000000004e-05 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758750+0000", 2024-09-19T05:44:07.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8240999999999999e-05 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:82 3.2b 3:d50470f2:::10000000002.0000004f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.918702608, 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29488188799999998, 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.432 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 82 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491754+0000", 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7186000000000001e-05 2024-09-19T05:44:07.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.506963+0000", 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015209675000000001 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.506991+0000", 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7872e-05 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507003+0000", 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507080+0000", 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7327e-05 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507223+0000", 2024-09-19T05:44:07.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014263500000000001 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507439+0000", 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000215873 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.511608+0000", 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041688610000000003 2024-09-19T05:44:07.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786537+0000", 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27492934899999999 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786576+0000", 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8575000000000003e-05 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786619+0000", 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2815999999999998e-05 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:132 3.2b 3:d52309a9:::10000000002.00000081:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.837828507999999, 2024-09-19T05:44:07.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21416674099999999, 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 132 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.440 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:07.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572614+0000", 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585573+0000", 2024-09-19T05:44:07.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012959205999999999 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585590+0000", 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7326999999999999e-05 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585605+0000", 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5061e-05 2024-09-19T05:44:07.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585665+0000", 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9991999999999998e-05 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585800+0000", 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013432599999999999 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585932+0000", 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013247199999999999 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.588624+0000", 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026916520000000001 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786688+0000", 2024-09-19T05:44:07.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.198064771 2024-09-19T05:44:07.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786738+0000", 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9951999999999999e-05 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786777+0000", 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9053e-05 2024-09-19T05:44:07.447 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.448 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.448 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.448 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:53 3.9 e53/20 3:9387ba5e:::10000000002.00000032:head v 53'1, mlcod=0'0)", 2024-09-19T05:44:07.448 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:07.448 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.815520884, 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28964351700000002, 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:07.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:07.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594919+0000", 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882027+0000", 2024-09-19T05:44:07.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28710772299999998 2024-09-19T05:44:07.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882048+0000", 2024-09-19T05:44:07.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1081e-05 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882056+0000", 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9049999999999997e-06 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882114+0000", 2024-09-19T05:44:07.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7762e-05 2024-09-19T05:44:07.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882132+0000", 2024-09-19T05:44:07.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8762000000000001e-05 2024-09-19T05:44:07.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.884525+0000", 2024-09-19T05:44:07.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002392595 2024-09-19T05:44:07.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.884562+0000", 2024-09-19T05:44:07.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6788000000000001e-05 2024-09-19T05:44:07.457 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.457 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.457 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.457 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:161 3.2b 3:d691b431:::10000000002.0000009e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.810641573, 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.206406749, 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.458 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 161 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:07.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599800+0000", 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.655e-06 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609524+0000", 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097240499999999997 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609544+0000", 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9772e-05 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609553+0000", 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3130000000000004e-06 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609604+0000", 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0933999999999997e-05 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609704+0000", 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9746000000000003e-05 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609799+0000", 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4823999999999999e-05 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.613360+0000", 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035613950000000002 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806134+0000", 2024-09-19T05:44:07.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19277316 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806173+0000", 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9873000000000003e-05 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806204+0000", 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1027e-05 2024-09-19T05:44:07.464 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:168 3.2b 3:d5eb8d7b:::10000000002.000000a5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.800862598, 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19673869999999999, 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.465 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 168 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609579+0000", 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6819999999999999e-06 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619452+0000", 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0098723679999999994 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619470+0000", 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8515000000000001e-05 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619479+0000", 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8650000000000007e-06 2024-09-19T05:44:07.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619556+0000", 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6569000000000005e-05 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619691+0000", 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013584900000000001 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619804+0000", 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112989 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.626514+0000", 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067099350000000002 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806244+0000", 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.179729837 2024-09-19T05:44:07.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806278+0000", 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4221999999999999e-05 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806315+0000", 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6869000000000002e-05 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.471 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:172 3.2b 3:d5480d6b:::10000000002.000000a9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.790933507, 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.186920695, 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.472 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 172 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619508+0000", 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630572+0000", 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011063454 2024-09-19T05:44:07.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630589+0000", 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6860000000000001e-05 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630596+0000", 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6420000000000007e-06 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630659+0000", 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3145999999999995e-05 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630785+0000", 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000125788 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630894+0000", 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001087 2024-09-19T05:44:07.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.635728+0000", 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048346090000000001 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806354+0000", 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.170625477 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806396+0000", 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2252999999999997e-05 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806426+0000", 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0266000000000001e-05 2024-09-19T05:44:07.478 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:179 3.2 3:42ce5726:::10000000002.000000b0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.779818428, 2024-09-19T05:44:07.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17570660699999999, 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 179 2024-09-19T05:44:07.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.481 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:07.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630624+0000", 2024-09-19T05:44:07.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7870000000000002e-06 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641403+0000", 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01077914 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641420+0000", 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7564e-05 2024-09-19T05:44:07.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641442+0000", 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1355000000000001e-05 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641517+0000", 2024-09-19T05:44:07.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5712999999999998e-05 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641680+0000", 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016307800000000001 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641848+0000", 2024-09-19T05:44:07.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016733500000000001 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647878+0000", 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060299159999999997 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806222+0000", 2024-09-19T05:44:07.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.158344174 2024-09-19T05:44:07.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806290+0000", 2024-09-19T05:44:07.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8201999999999995e-05 2024-09-19T05:44:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806327+0000", 2024-09-19T05:44:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7342999999999998e-05 2024-09-19T05:44:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.488 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:269 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:07.489 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.8173216060000001, 2024-09-19T05:44:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.44962057300000002, 2024-09-19T05:44:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.490 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 269 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:07.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593119+0000", 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5099999999999999e-06 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614489+0000", 2024-09-19T05:44:07.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.021369545 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614511+0000", 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2353000000000001e-05 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614525+0000", 2024-09-19T05:44:07.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4311000000000001e-05 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614691+0000", 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016586800000000001 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614811+0000", 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119552 2024-09-19T05:44:07.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614956+0000", 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014476499999999999 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.619618+0000", 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046627020000000003 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042637+0000", 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.42301865999999999 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042689+0000", 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1876000000000003e-05 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042738+0000", 2024-09-19T05:44:07.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9431000000000002e-05 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:320 3.2 3:40abce4f:::10000000002.0000003d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.6728976580000001, 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30531098200000001, 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.498 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 320 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737544+0000", 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6539999999999999e-06 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760554+0000", 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023009943000000001 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760577+0000", 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2538e-05 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760585+0000", 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9219999999999997e-06 2024-09-19T05:44:07.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760691+0000", 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106606 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760791+0000", 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100239 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760966+0000", 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017421299999999999 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.766414+0000", 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005448067 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042779+0000", 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27636495599999999 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042822+0000", 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3359000000000003e-05 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042853+0000", 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0485e-05 2024-09-19T05:44:07.504 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:338 3.2b 3:d50470f2:::10000000002.0000004f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.6329193570000005, 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30392799300000001, 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.505 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 338 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777522+0000", 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3539999999999998e-06 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.793967+0000", 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016445120000000001 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794010+0000", 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3143999999999999e-05 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794026+0000", 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5404999999999999e-05 2024-09-19T05:44:07.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794124+0000", 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7806999999999994e-05 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794224+0000", 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100318 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794397+0000", 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017272500000000001 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.800396+0000", 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059990449999999997 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081329+0000", 2024-09-19T05:44:07.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28093356000000003 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081398+0000", 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8786000000000002e-05 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081448+0000", 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9728999999999998e-05 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:388 3.2b 3:d52309a9:::10000000002.00000081:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.5192083289999996, 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19034685300000001, 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.512 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 388 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891233+0000", 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4059999999999999e-06 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905206+0000", 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013972507 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905225+0000", 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9167000000000001e-05 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905259+0000", 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3633999999999997e-05 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905360+0000", 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101512 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905437+0000", 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6964000000000004e-05 2024-09-19T05:44:07.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905546+0000", 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108933 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.909399+0000", 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003853249 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081492+0000", 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.172092667 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081535+0000", 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3287000000000001e-05 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081578+0000", 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2527000000000001e-05 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:417 3.2b 3:d691b431:::10000000002.0000009e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.519 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.4929220000000001, 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19858436900000001, 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 417 2024-09-19T05:44:07.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917519+0000", 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2330000000000001e-06 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936253+0000", 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018733107999999998 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936269+0000", 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6441999999999999e-05 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936278+0000", 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8370000000000003e-06 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936338+0000", 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0318000000000001e-05 2024-09-19T05:44:07.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936414+0000", 2024-09-19T05:44:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6249999999999997e-05 2024-09-19T05:44:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936508+0000", 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3194000000000003e-05 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.941616+0000", 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051086789999999997 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116003+0000", 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17438693 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116057+0000", 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3829999999999998e-05 2024-09-19T05:44:07.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116102+0000", 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:424 3.2b 3:d5eb8d7b:::10000000002.000000a5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.4741358869999992, 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17989760599999999, 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.529 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 424 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936305+0000", 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9910000000000001e-06 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950048+0000", 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013742437 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.531 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950078+0000", 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9904e-05 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950092+0000", 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4543e-05 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950182+0000", 2024-09-19T05:44:07.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0236999999999997e-05 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950294+0000", 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111993 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950451+0000", 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015673400000000001 2024-09-19T05:44:07.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.954644+0000", 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041925249999999999 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116134+0000", 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16149000999999999 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116179+0000", 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4919000000000001e-05 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116201+0000", 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2313000000000001e-05 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.535 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:428 3.2b 3:d5480d6b:::10000000002.000000a9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.4597774749999992, 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17460092799999999, 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 428 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.536 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950665+0000", 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.931e-06 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967645+0000", 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016980439 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967663+0000", 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7657999999999999e-05 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967670+0000", 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3529999999999997e-06 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967735+0000", 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4379000000000001e-05 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967844+0000", 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109863 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.968022+0000", 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177987 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973507+0000", 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054845980000000003 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125044+0000", 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15153746700000001 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125149+0000", 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104286 2024-09-19T05:44:07.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125263+0000", 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113967 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:439 3.1b 3:db82751e:::10000000002.000000b4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:07.542 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.4306739739999994, 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19035659999999999, 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 439 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979767+0000", 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6819999999999999e-06 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993421+0000", 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013654314000000001 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993442+0000", 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0911e-05 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993451+0000", 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.918e-06 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993525+0000", 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4259999999999997e-05 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993635+0000", 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109851 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993795+0000", 2024-09-19T05:44:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015986599999999999 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.000943+0000", 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071475469999999998 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170055+0000", 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16911190800000001 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170090+0000", 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5222999999999998e-05 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170122+0000", 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2119999999999997e-05 2024-09-19T05:44:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:443 3.1b 3:da9dcad1:::10000000002.000000b8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.4169623989999991, 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.186683074, 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.550 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 443 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:07.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993478+0000", 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.398e-06 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004522+0000", 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011043536 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004541+0000", 2024-09-19T05:44:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9301999999999999e-05 2024-09-19T05:44:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004551+0000", 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7589999999999992e-06 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004657+0000", 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106069 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004805+0000", 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148197 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004912+0000", 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010655800000000001 2024-09-19T05:44:07.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.007998+0000", 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030858679999999999 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180063+0000", 2024-09-19T05:44:07.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17206524200000001 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180114+0000", 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1122999999999998e-05 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180160+0000", 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6022000000000001e-05 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.561 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:490 3.2 3:437e6408:::10000000002.000000e7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.3658808849999993, 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15379004299999999, 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.562 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 490 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044560+0000", 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.685e-06 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054251+0000", 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096909700000000001 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054267+0000", 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5755000000000001e-05 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054277+0000", 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0504e-05 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054333+0000", 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6001000000000001e-05 2024-09-19T05:44:07.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054396+0000", 2024-09-19T05:44:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2852000000000001e-05 2024-09-19T05:44:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054483+0000", 2024-09-19T05:44:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7332999999999999e-05 2024-09-19T05:44:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.057123+0000", 2024-09-19T05:44:07.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002639705 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198289+0000", 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.14116562599999999 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198321+0000", 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2543000000000003e-05 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198348+0000", 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7069000000000001e-05 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:502 3.2 3:4041dd16:::10000000002.000000f3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.356141268, 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.14802890099999999, 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 502 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:07.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054300+0000", 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.012e-06 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063321+0000", 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090211760000000005 2024-09-19T05:44:07.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063333+0000", 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2308e-05 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063340+0000", 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7819999999999996e-06 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063427+0000", 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6948999999999997e-05 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063536+0000", 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109066 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063706+0000", 2024-09-19T05:44:07.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000169906 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068352+0000", 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046461590000000004 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202270+0000", 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.133917486 2024-09-19T05:44:07.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202302+0000", 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2237999999999999e-05 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202327+0000", 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4819000000000001e-05 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.577 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.578 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.578 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:07.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:07.580 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:07.578+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:07.680 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:07.678+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:07.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.695+0000 7fbd4affd640 1 -- 172.21.15.2:0/810637009 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v59) ==== 91+0+0 (secure 0 0 0) 0x7fbd3c0587b0 con 0x7fbd4c158f70 2024-09-19T05:44:07.697 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:44:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbd3806c2b0 msgr2=0x7fbd3806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbd3806c2b0 0x7fbd3806e770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd44034000 tx=0x7fbd4402e000 comp rx=0 tx=0).stop 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c158f70 msgr2=0x7fbd4c1ef6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c158f70 0x7fbd4c1ef6e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c3f0 tx=0x7fbd3c00c8b0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 shutdown_connections 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbd3806c2b0 0x7fbd3806e770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd4c152f70 0x7fbd4c14f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbd4c158f70 0x7fbd4c1ef6e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 --2- 172.21.15.2:0/810637009 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd4c152b90 0x7fbd4c1f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 >> 172.21.15.2:0/810637009 conn(0x7fbd4c06f3d0 msgr2=0x7fbd4c1576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 shutdown_connections 2024-09-19T05:44:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:07.699+0000 7fbd5461e640 1 -- 172.21.15.2:0/810637009 wait complete. 2024-09-19T05:44:07.727 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:07.727 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:07.727 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:07.727 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:07.736 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:07.781 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:07.779+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:07.839 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:07.839 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:07.839 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:07.839 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:07.840 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:07.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:07.891 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:07.890+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:07.954 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:32 3.31 3:8d4d39bf:::10000000002.0000001d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.585423201999999, 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48911290200000002, 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 32 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366127+0000", 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429999999999999e-06 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388967+0000", 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022840284999999998 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388978+0000", 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0811e-05 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388983+0000", 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7879999999999997e-06 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389069+0000", 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6670999999999997e-05 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389303+0000", 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023413399999999999 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389482+0000", 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017864399999999999 2024-09-19T05:44:07.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394972+0000", 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054895530000000003 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855135+0000", 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46016378400000002 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855197+0000", 2024-09-19T05:44:07.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1365999999999995e-05 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855237+0000", 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9823000000000001e-05 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:50 3.17 3:eaf1bb0b:::10000000002.0000002f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.562540869999999, 2024-09-19T05:44:07.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15463449900000001, 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389008+0000", 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414228+0000", 2024-09-19T05:44:07.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025220256 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414244+0000", 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5821999999999999e-05 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414262+0000", 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7750000000000001e-05 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414324+0000", 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2250999999999997e-05 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414475+0000", 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015051800000000001 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414621+0000", 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014692000000000001 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.419434+0000", 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048125429999999999 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543566+0000", 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12413210600000001 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543602+0000", 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5420000000000003e-05 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543640+0000", 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8976000000000001e-05 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:88 3.31 3:8c5098ad:::10000000002.00000055:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.493277624999999, 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41301092299999997, 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.967 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 88 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458271+0000", 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.804e-06 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481919+0000", 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.02364809 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481946+0000", 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7370000000000001e-05 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481957+0000", 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.066e-05 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482050+0000", 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2991999999999993e-05 2024-09-19T05:44:07.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482206+0000", 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000155611 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482359+0000", 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000153728 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.485588+0000", 2024-09-19T05:44:07.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003228557 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871172+0000", 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.38558421500000001 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871237+0000", 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4562999999999998e-05 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871280+0000", 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3333e-05 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:142 3.31 3:8d07500b:::10000000002.0000008b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.339817529999999, 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26228585799999998, 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.973 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 142 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611732+0000", 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8870000000000001e-06 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623428+0000", 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011695743 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623449+0000", 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1178e-05 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623463+0000", 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3923e-05 2024-09-19T05:44:07.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623546+0000", 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3090999999999994e-05 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623713+0000", 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000167176 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623874+0000", 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016018299999999999 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.628673+0000", 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047995529999999998 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.873957+0000", 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24528372500000001 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.873991+0000", 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4663999999999999e-05 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.874015+0000", 2024-09-19T05:44:07.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3734999999999999e-05 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:173 3.38 3:1ee175b9:::10000000002.000000aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.303776393, 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24937071899999999, 2024-09-19T05:44:07.980 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 173 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647773+0000", 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3870000000000002e-06 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659807+0000", 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012033909000000001 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659823+0000", 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6667000000000002e-05 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659850+0000", 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6798e-05 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659930+0000", 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0041000000000002e-05 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.660057+0000", 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127164 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.660179+0000", 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012194 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.664305+0000", 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041255900000000002 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897061+0000", 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.23275601800000001 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897101+0000", 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0156e-05 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897141+0000", 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0049000000000002e-05 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:288 3.31 3:8d4d39bf:::10000000002.0000001d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.3032903230000006, 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54072771799999997, 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 288 2024-09-19T05:44:07.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648258+0000", 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.686e-06 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673526+0000", 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025268142 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673552+0000", 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5500999999999999e-05 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673562+0000", 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.825e-06 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673673+0000", 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011167000000000001 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673778+0000", 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104987 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673894+0000", 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115639 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.677601+0000", 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037067900000000002 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188877+0000", 2024-09-19T05:44:07.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.51127632700000003 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188935+0000", 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7553000000000003e-05 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188984+0000", 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9598000000000002e-05 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:306 3.17 3:eaf1bb0b:::10000000002.0000002f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.2779505279999999, 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17070818600000001, 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 306 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.994 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:07.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673598+0000", 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1129999999999999e-06 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692350+0000", 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018751790000000001 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692373+0000", 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2629999999999998e-05 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692385+0000", 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1905999999999999e-05 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692488+0000", 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010292499999999999 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692599+0000", 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011170100000000001 2024-09-19T05:44:07.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692750+0000", 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151099 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.697776+0000", 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050251590000000004 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844188+0000", 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.146412135 2024-09-19T05:44:07.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844251+0000", 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3523999999999997e-05 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844305+0000", 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3204000000000001e-05 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:07.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:319 3.17 3:ead2f1a2:::10000000002.0000003c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.2591372930000002, 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22292820499999999, 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 319 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692412+0000", 2024-09-19T05:44:08.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6900000000000001e-06 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708570+0000", 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016157659000000001 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708591+0000", 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1260999999999999e-05 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708602+0000", 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708663+0000", 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1811999999999997e-05 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708741+0000", 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7385999999999994e-05 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708853+0000", 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112642 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715058+0000", 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062048729999999996 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915221+0000", 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.200162954 2024-09-19T05:44:08.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915283+0000", 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2059999999999999e-05 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915338+0000", 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4487999999999997e-05 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:292 3.3b e54/20 3:ddac2f4e:::10000000002.00000021:head v 54'6, mlcod=53'4)", 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.2532372850000009, 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.40806384200000001, 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698311+0000", 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3880000000000001e-06 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103237+0000", 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.40492591100000003 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103250+0000", 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3593e-05 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103261+0000", 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0181e-05 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103332+0000", 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.114e-05 2024-09-19T05:44:08.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103357+0000", 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5094000000000001e-05 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.106302+0000", 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029454260000000001 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.106373+0000", 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1109e-05 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:344 3.31 3:8c5098ad:::10000000002.00000055:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.2248915779999994, 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.47092658999999998, 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.010 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 344 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726658+0000", 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.649e-06 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758353+0000", 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031695424999999999 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758372+0000", 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8723e-05 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758387+0000", 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4882000000000001e-05 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758510+0000", 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012282100000000001 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758628+0000", 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118544 2024-09-19T05:44:08.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758867+0000", 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0002385 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.765925+0000", 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070580670000000003 2024-09-19T05:44:08.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197522+0000", 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.43159685599999997 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197553+0000", 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1319999999999998e-05 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197582+0000", 2024-09-19T05:44:08.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8802999999999998e-05 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:358 3.17 3:eaf05c3c:::10000000002.00000063:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.171356673, 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30462805700000001, 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.017 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 358 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780193+0000", 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5660000000000001e-06 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804477+0000", 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024284329 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804498+0000", 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1251e-05 2024-09-19T05:44:08.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804511+0000", 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2234e-05 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804595+0000", 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4651e-05 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804686+0000", 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1062000000000003e-05 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804937+0000", 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025027200000000001 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.808988+0000", 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040513900000000002 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084691+0000", 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.275703319 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084761+0000", 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9845999999999995e-05 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084818+0000", 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7136999999999999e-05 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:367 3.17 3:e9c25901:::10000000002.0000006c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.1470078669999992, 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.35149704900000001, 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 367 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804541+0000", 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9360000000000002e-06 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832876+0000", 2024-09-19T05:44:08.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.028334734 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832891+0000", 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5404e-05 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832899+0000", 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1759999999999993e-06 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832970+0000", 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000006e-05 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8103000000000007e-05 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833196+0000", 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138131 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837012+0000", 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038158770000000001 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.155960+0000", 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.318948334 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.155994+0000", 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3630000000000002e-05 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156036+0000", 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2224000000000001e-05 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:390 3.17 3:e85118c5:::10000000002.00000083:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.0805621300000006, 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28513696999999999, 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.030 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 390 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.871040+0000", 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5596000000000001e-05 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883834+0000", 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012794005000000001 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883858+0000", 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3380999999999999e-05 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883869+0000", 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1245e-05 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883964+0000", 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4749999999999999e-05 2024-09-19T05:44:08.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.884049+0000", 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5023000000000002e-05 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.884176+0000", 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012745500000000001 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.887612+0000", 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034360850000000002 2024-09-19T05:44:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156071+0000", 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26845824099999999 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156100+0000", 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9368000000000001e-05 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156122+0000", 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1821e-05 2024-09-19T05:44:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:392 3.1e 3:7b5aaf6d:::10000000002.00000085:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.0676499219999993, 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.25285440300000001, 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.036 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 392 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883899+0000", 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4399999999999999e-06 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896769+0000", 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012869307 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896783+0000", 2024-09-19T05:44:08.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896793+0000", 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0182e-05 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896887+0000", 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3704999999999998e-05 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.897039+0000", 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015165299999999999 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.897229+0000", 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000190552 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.901089+0000", 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038593680000000002 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136638+0000", 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.235549007 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136703+0000", 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5030999999999995e-05 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136751+0000", 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8798000000000003e-05 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.041 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:398 3.31 3:8d07500b:::10000000002.0000008b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.0547175880000008, 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.326241485, 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 398 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.042 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896832+0000", 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6410000000000002e-06 2024-09-19T05:44:08.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908746+0000", 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011914118 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908772+0000", 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6149999999999999e-05 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908789+0000", 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6322000000000001e-05 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908897+0000", 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010891 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908999+0000", 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101216 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.909146+0000", 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014757099999999999 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.913702+0000", 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004555875 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.222963+0000", 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30926109600000001 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223026+0000", 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3182999999999995e-05 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223071+0000", 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4403000000000002e-05 2024-09-19T05:44:08.047 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:429 3.38 3:1ee175b9:::10000000002.000000aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.0086806510000006, 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.23704366099999999, 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.048 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 429 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942868+0000", 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5319999999999999e-06 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957357+0000", 2024-09-19T05:44:08.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014489149 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957373+0000", 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6575e-05 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957381+0000", 2024-09-19T05:44:08.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6410000000000002e-06 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957443+0000", 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2200999999999996e-05 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957560+0000", 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117044 2024-09-19T05:44:08.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957686+0000", 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012538100000000001 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.961805+0000", 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041189649999999996 2024-09-19T05:44:08.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179828+0000", 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21802357999999999 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179875+0000", 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6609e-05 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179910+0000", 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4984000000000003e-05 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:434 3.1e 3:789394ea:::10000000002.000000af:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.9802043339999997, 2024-09-19T05:44:08.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24320008700000001, 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 434 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971344+0000", 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.598e-06 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985502+0000", 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014158082000000001 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985526+0000", 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4085000000000001e-05 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985537+0000", 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1143e-05 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985630+0000", 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2417000000000006e-05 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985741+0000", 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011084 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985917+0000", 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017616499999999999 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.991540+0000", 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056230630000000002 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214462+0000", 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22292171499999999 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214510+0000", 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8154e-05 2024-09-19T05:44:08.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214543+0000", 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2824999999999998e-05 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:472 3.31 3:8c3036ad:::10000000002.000000d5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.8827180390000002, 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.154846594, 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.062 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 472 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068831+0000", 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0140000000000001e-06 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078582+0000", 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097511360000000005 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078597+0000", 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5194000000000001e-05 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078605+0000", 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0860000000000002e-06 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078689+0000", 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4184999999999994e-05 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078786+0000", 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6094000000000005e-05 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078912+0000", 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126749 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.083288+0000", 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004375421 2024-09-19T05:44:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223574+0000", 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.140286203 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223628+0000", 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4401000000000003e-05 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223675+0000", 2024-09-19T05:44:08.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7110999999999997e-05 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:496 3.1e 3:7b5946a1:::10000000002.000000ed:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.857770275, 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15292847600000001, 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.068 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 496 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:08.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093778+0000", 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.319e-06 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099527+0000", 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057487959999999996 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099543+0000", 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6767000000000001e-05 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099552+0000", 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0919999999999998e-06 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099633+0000", 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1337999999999999e-05 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099734+0000", 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101005 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099877+0000", 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000143067 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103900+0000", 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040226250000000002 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246621+0000", 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.142720927 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246660+0000", 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9567000000000003e-05 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246705+0000", 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4972999999999998e-05 2024-09-19T05:44:08.073 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:500 3.29 3:96fc3913:::10000000002.000000f1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 8.8509379930000005, 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.153671691, 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.074 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 500 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100612+0000", 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.055e-06 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108795+0000", 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081828820000000007 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108807+0000", 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2468e-05 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108815+0000", 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3690000000000001e-06 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108887+0000", 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2509000000000007e-05 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108964+0000", 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6787999999999997e-05 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.109095+0000", 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013103499999999999 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.112337+0000", 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032419950000000001 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254185+0000", 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.141847683 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254240+0000", 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5702999999999998e-05 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254281+0000", 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0204000000000003e-05 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.079 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.080 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.080 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.080 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:08.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:08.082 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:08.080+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:08.182 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:08.180+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:08.226 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:08.226 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:08.227 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:08.227 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:08.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:08.321 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:08.319+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:08.381 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:08.382 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:08.382 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:08.382 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:08.382 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:08.383 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:08.381+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:08.391 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:08.484 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:08.482+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:08.537 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:08.538 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3 3.12 3:4bec3886:::10000000002.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.253623232000001, 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57240116200000002, 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.539 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280487+0000", 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.751e-06 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287574+0000", 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070870530000000003 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287592+0000", 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8241e-05 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287601+0000", 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2800000000000003e-06 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287664+0000", 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3474999999999998e-05 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287758+0000", 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3577000000000002e-05 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287882+0000", 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012487199999999999 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.292606+0000", 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047238860000000001 2024-09-19T05:44:08.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852756+0000", 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56015009400000004 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852823+0000", 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6138999999999998e-05 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852886+0000", 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3794000000000006e-05 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:27 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.231089234000001, 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55004442399999998, 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.545 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 27 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303023+0000", 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4340000000000001e-06 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316366+0000", 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013343634 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316392+0000", 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6120999999999999e-05 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316402+0000", 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0066e-05 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316514+0000", 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111859 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316686+0000", 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017135699999999999 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316868+0000", 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018244100000000001 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.324019+0000", 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071506060000000003 2024-09-19T05:44:08.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852956+0000", 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52893702899999995 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853018+0000", 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2178e-05 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853064+0000", 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5698999999999997e-05 2024-09-19T05:44:08.550 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:46 3.12 3:489b4446:::10000000002.0000002b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.202513368, 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54589824600000003, 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.551 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 46 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331598+0000", 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5689999999999999e-06 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347173+0000", 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015575235999999999 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347192+0000", 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9113000000000001e-05 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347202+0000", 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0151e-05 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347265+0000", 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3163000000000005e-05 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347408+0000", 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014307299999999999 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347532+0000", 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012328400000000001 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.351950+0000", 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044183620000000003 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877426+0000", 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52547596500000004 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877461+0000", 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877493+0000", 2024-09-19T05:44:08.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2020000000000002e-05 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:51 3.12 3:499536aa:::10000000002.00000030:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.170764252, 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53664442800000001, 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.557 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 51 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363346+0000", 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9709999999999998e-06 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383174+0000", 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019827860999999999 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383192+0000", 2024-09-19T05:44:08.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383200+0000", 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9589999999999995e-06 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383263+0000", 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2502999999999994e-05 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383391+0000", 2024-09-19T05:44:08.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012766399999999999 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383510+0000", 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118998 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.386613+0000", 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031035260000000001 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899913+0000", 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.51330011600000003 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899958+0000", 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4487000000000003e-05 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899989+0000", 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0886999999999998e-05 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.562 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:157 3.12 3:4b703321:::10000000002.0000009a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.971470764999999, 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34817550400000002, 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 157 2024-09-19T05:44:08.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562641+0000", 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576231+0000", 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013590286 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576263+0000", 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1875999999999998e-05 2024-09-19T05:44:08.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576270+0000", 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4840000000000001e-06 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576349+0000", 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.975e-05 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576547+0000", 2024-09-19T05:44:08.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000198154 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576738+0000", 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019091400000000001 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.581161+0000", 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044224930000000004 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910746+0000", 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32958495700000001 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910782+0000", 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6636e-05 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910813+0000", 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0675e-05 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.568 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:158 3.1 3:82785032:::10000000002.0000009b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.955573632, 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.11650379299999999, 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 158 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.569 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578538+0000", 2024-09-19T05:44:08.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4939999999999999e-06 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597197+0000", 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018658534000000001 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597218+0000", 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1296000000000001e-05 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597224+0000", 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4020000000000001e-06 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597274+0000", 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0556000000000002e-05 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597387+0000", 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113119 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:08.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597484+0000", 2024-09-19T05:44:08.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6346000000000002e-05 2024-09-19T05:44:08.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.600132+0000", 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026487059999999998 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.694929+0000", 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.094797131000000007 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.694990+0000", 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0287000000000001e-05 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.695039+0000", 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8921999999999997e-05 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:184 3.12 3:4bdf515e:::10000000002.000000b5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:08.578 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.912980288, 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29608825300000002, 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 184 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:08.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621130+0000", 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0789999999999999e-06 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632344+0000", 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01121422 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632369+0000", 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4712999999999999e-05 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632380+0000", 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1037e-05 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632436+0000", 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5887999999999998e-05 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632544+0000", 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107601 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632641+0000", 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7460000000000005e-05 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.636592+0000", 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039505950000000003 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917116+0000", 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.280524098 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917174+0000", 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7720999999999998e-05 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917216+0000", 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2840999999999999e-05 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:200 3.24 3:2525a577:::10000000002.000000c5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.882514208, 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.176819009, 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.584 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 200 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651597+0000", 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661419+0000", 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009822239 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661445+0000", 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5922999999999999e-05 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661463+0000", 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8501000000000001e-05 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661572+0000", 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108471 2024-09-19T05:44:08.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661690+0000", 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118387 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661822+0000", 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013183800000000001 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.666069+0000", 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042470839999999999 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828327+0000", 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16225789600000001 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828368+0000", 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1402000000000001e-05 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828413+0000", 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4712999999999997e-05 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:252 3.24 3:25920220:::10000000002.000000f9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.828918969, 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12927656200000001, 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.590 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 252 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705192+0000", 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5849999999999998e-06 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720126+0000", 2024-09-19T05:44:08.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01493366 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720143+0000", 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7385e-05 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720154+0000", 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0601000000000001e-05 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720232+0000", 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8584999999999994e-05 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720420+0000", 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018729799999999999 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720631+0000", 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021127400000000001 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.728280+0000", 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076494989999999997 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834381+0000", 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.106101029 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834423+0000", 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1634999999999997e-05 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834466+0000", 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3010999999999999e-05 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:283 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.9201455470000006, 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57880034499999999, 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.596 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 283 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613965+0000", 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8759999999999999e-06 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633465+0000", 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019500483999999998 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633485+0000", 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.984e-05 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633494+0000", 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5520000000000005e-06 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633589+0000", 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5144999999999998e-05 2024-09-19T05:44:08.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633713+0000", 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012382300000000001 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633835+0000", 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122705 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.640960+0000", 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071243599999999997 2024-09-19T05:44:08.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192691+0000", 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55173159599999999 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192737+0000", 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5487e-05 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192763+0000", 2024-09-19T05:44:08.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6477000000000001e-05 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:294 3.24 3:2742a876:::10000000002.00000023:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.9005753940000005, 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12659410500000001, 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.602 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 294 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633536+0000", 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.649e-06 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651603+0000", 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018067155000000001 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651628+0000", 2024-09-19T05:44:08.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4950999999999999e-05 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651639+0000", 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-05 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651756+0000", 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117209 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651847+0000", 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1372000000000005e-05 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.652064+0000", 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021635099999999999 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.657902+0000", 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058383100000000002 2024-09-19T05:44:08.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.759968+0000", 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.102066002 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760054+0000", 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5728000000000003e-05 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760127+0000", 2024-09-19T05:44:08.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3248000000000001e-05 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:302 3.12 3:489b4446:::10000000002.0000002b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.8824205880000004, 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54116458000000001, 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.608 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 302 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651691+0000", 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1810000000000001e-06 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664738+0000", 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013046540000000001 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664754+0000", 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6019999999999999e-05 2024-09-19T05:44:08.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664767+0000", 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3315999999999999e-05 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664845+0000", 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8472000000000004e-05 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664945+0000", 2024-09-19T05:44:08.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9804999999999997e-05 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.665089+0000", 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014373700000000001 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.669251+0000", 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041617379999999999 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192796+0000", 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52354539899999997 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192829+0000", 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2668e-05 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192852+0000", 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3703999999999999e-05 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:307 3.12 3:499536aa:::10000000002.00000030:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.8483240869999999, 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.50714811299999996, 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 307 2024-09-19T05:44:08.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685788+0000", 2024-09-19T05:44:08.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2480000000000001e-06 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698429+0000", 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012641172000000001 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698451+0000", 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2172e-05 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698462+0000", 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1141000000000001e-05 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698591+0000", 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012867999999999999 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698704+0000", 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113409 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698834+0000", 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012981599999999999 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.703334+0000", 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045004600000000004 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192881+0000", 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48954666899999999 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192908+0000", 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6988e-05 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192932+0000", 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4358e-05 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.619 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:397 3.24 3:273fbfa7:::10000000002.0000008a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.7240033389999994, 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24366095600000001, 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 397 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810107+0000", 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-09-19T05:44:08.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824232+0000", 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014124503 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824247+0000", 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4834000000000001e-05 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824259+0000", 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1769e-05 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824368+0000", 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109722 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824500+0000", 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013183699999999999 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824766+0000", 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000266016 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.830112+0000", 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053457620000000004 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053694+0000", 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22358192099999999 2024-09-19T05:44:08.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053736+0000", 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2534999999999998e-05 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053766+0000", 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9643999999999999e-05 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.625 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:413 3.12 3:4b703321:::10000000002.0000009a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.6963656930000006, 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.36390152399999998, 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 413 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837745+0000", 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.04e-06 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851397+0000", 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013652687 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851412+0000", 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4776999999999999e-05 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851420+0000", 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.43e-06 2024-09-19T05:44:08.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851493+0000", 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3583000000000003e-05 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851600+0000", 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010678 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851717+0000", 2024-09-19T05:44:08.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117237 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.855412+0000", 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036952439999999999 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201584+0000", 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34617155799999999 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201618+0000", 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3494999999999997e-05 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201644+0000", 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6693e-05 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:422 3.1 3:8350567c:::10000000002.000000a3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:08.631 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.6680255499999994, 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15887042900000001, 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 422 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:08.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866086+0000", 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880367+0000", 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014281046 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880392+0000", 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5054000000000001e-05 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880406+0000", 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4401e-05 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880532+0000", 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012579600000000001 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880667+0000", 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000135146 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880827+0000", 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015946299999999999 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.886956+0000", 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061295730000000001 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024837+0000", 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.13788043999999999 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024890+0000", 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3483000000000003e-05 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024953+0000", 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3097999999999998e-05 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:440 3.12 3:4bdf515e:::10000000002.000000b5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.6419427710000001, 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.31533669800000003, 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.637 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 440 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892170+0000", 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6179999999999997e-06 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900748+0000", 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085778480000000008 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900763+0000", 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5231999999999999e-05 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900773+0000", 2024-09-19T05:44:08.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7240000000000004e-06 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900829+0000", 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5562000000000001e-05 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900902+0000", 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3313999999999996e-05 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:08.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.901041+0000", 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000139419 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.903913+0000", 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028715619999999998 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207434+0000", 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30352098799999999 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207468+0000", 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3861e-05 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207502+0000", 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000003e-05 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:456 3.24 3:2525a577:::10000000002.000000c5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:08.643 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.608906546, 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26312323300000001, 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 456 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925204+0000", 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1110000000000002e-06 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938061+0000", 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012856615 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938083+0000", 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2543000000000001e-05 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938093+0000", 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0084e-05 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938178+0000", 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4827000000000006e-05 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938280+0000", 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010234900000000001 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938437+0000", 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015626099999999999 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942534+0000", 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004097458 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188244+0000", 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24570961099999999 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188281+0000", 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7134000000000001e-05 2024-09-19T05:44:08.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188325+0000", 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4240000000000003e-05 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:487 3.33 3:ce8011cb:::10000000002.000000e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.5861512490000003, 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22306556599999999, 2024-09-19T05:44:08.649 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 487 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947959+0000", 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1519999999999999e-06 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962107+0000", 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014147649 2024-09-19T05:44:08.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962127+0000", 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9868999999999999e-05 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962137+0000", 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0407e-05 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962206+0000", 2024-09-19T05:44:08.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9122000000000007e-05 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962312+0000", 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105982 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962445+0000", 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000133108 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966463+0000", 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004017779 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170931+0000", 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.20446781999999999 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170977+0000", 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6245000000000002e-05 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.171023+0000", 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5432999999999997e-05 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:508 3.24 3:25920220:::10000000002.000000f9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 9.5609674420000008, 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21526778799999999, 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:08.655 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 508 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973144+0000", 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.542e-06 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983914+0000", 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010770711000000001 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983945+0000", 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0648999999999997e-05 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983955+0000", 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8269999999999993e-06 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984017+0000", 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2652999999999998e-05 2024-09-19T05:44:08.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984100+0000", 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3094000000000001e-05 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984208+0000", 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107768 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.988635+0000", 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044265620000000002 2024-09-19T05:44:08.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188359+0000", 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19972409799999999 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188389+0000", 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9882999999999998e-05 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188409+0000", 2024-09-19T05:44:08.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0001e-05 2024-09-19T05:44:08.661 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.661 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.661 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.661 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:08.661 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:08.661 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:08.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:08.663 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:08.660+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:08.763 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:08.762+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:08.807 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:08.807 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:08.807 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:08.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:08.900 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:08.898+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:08.961 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:08.961 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:08.961 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:08.961 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:08.961 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:08.965 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:08.963+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:08.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:09.037 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:09.081 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:09.079+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:17 3.21 3:86f42aac:::10000000002.0000000e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.853026142000001, 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.072757916000000006, 2024-09-19T05:44:09.151 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 17 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293556+0000", 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1830000000000001e-06 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320078+0000", 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.026521807000000001 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320090+0000", 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1873e-05 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320096+0000", 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.852e-06 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320193+0000", 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7145999999999995e-05 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320321+0000", 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012788799999999999 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320445+0000", 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000124206 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.325801+0000", 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053561700000000004 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366202+0000", 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.040400806999999997 2024-09-19T05:44:09.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366258+0000", 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5971999999999999e-05 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366312+0000", 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4011999999999997e-05 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:20 3.21 3:85380590:::10000000002.00000011:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.826456819000001, 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.094367290000000006, 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 20 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.158 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320125+0000", 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5319999999999999e-06 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345237+0000", 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025112457000000001 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345260+0000", 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3184999999999999e-05 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345274+0000", 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3168e-05 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345351+0000", 2024-09-19T05:44:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7600000000000002e-05 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345507+0000", 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015602300000000001 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345659+0000", 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015159599999999999 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.351961+0000", 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063018880000000003 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414334+0000", 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.062373799000000001 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414436+0000", 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101795 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414491+0000", 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4246999999999997e-05 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:23 3.21 3:87fa6b7a:::10000000002.00000014:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.801273071000001, 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.106935525, 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.164 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345309+0000", 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369545+0000", 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024236034 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369565+0000", 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9928999999999998e-05 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369576+0000", 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0373e-05 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369677+0000", 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101765 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369863+0000", 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000185692 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370039+0000", 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017557 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.374766+0000", 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047269210000000002 2024-09-19T05:44:09.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452093+0000", 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.077327075999999995 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452160+0000", 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7805999999999994e-05 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452243+0000", 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2078999999999998e-05 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:39 3.2e 3:75eea580:::10000000002.00000024:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.776557515, 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41578167199999999, 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.170 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 39 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370025+0000", 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393960+0000", 2024-09-19T05:44:09.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023934536999999999 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393976+0000", 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6441000000000001e-05 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393989+0000", 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2679e-05 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394066+0000", 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6919e-05 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394208+0000", 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014247500000000001 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394304+0000", 2024-09-19T05:44:09.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5369000000000001e-05 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.399842+0000", 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055380000000000004 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785727+0000", 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.38588529199999999 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785763+0000", 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6494e-05 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785804+0000", 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0729000000000003e-05 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:62 3.c e53/20 3:31d8a57c:::10000000002.0000003b:head v 53'1, mlcod=0'0)", 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.768896311000001, 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.027856545, 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0600000000000001e-07 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400389+0000", 2024-09-19T05:44:09.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022704832000000001 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400402+0000", 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2893e-05 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400411+0000", 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.037e-06 2024-09-19T05:44:09.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400473+0000", 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400489+0000", 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6651e-05 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.405444+0000", 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049551279999999996 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.405540+0000", 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5969000000000002e-05 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:124 3.21 3:844ac05a:::10000000002.00000079:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:09.181 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.697138275, 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.060327543999999997, 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 124 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449444+0000", 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2239999999999998e-06 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464881+0000", 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015437222 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464904+0000", 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2274999999999999e-05 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464914+0000", 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0657e-05 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464999+0000", 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5031000000000006e-05 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.465166+0000", 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016689400000000001 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.465311+0000", 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145208 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.470491+0000", 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051794229999999998 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509673+0000", 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.039182002 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509724+0000", 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1205999999999997e-05 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509769+0000", 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5401999999999997e-05 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:138 3.2e 3:7768043d:::10000000002.00000087:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.681628355000001, 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34341082099999998, 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.188 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 138 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464954+0000", 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482737+0000", 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017782893000000001 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482757+0000", 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0352000000000001e-05 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482766+0000", 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0119999999999994e-06 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482841+0000", 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4386000000000002e-05 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482989+0000", 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148405 2024-09-19T05:44:09.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.483128+0000", 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013918200000000001 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.486268+0000", 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031401229999999999 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808279+0000", 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32201085299999999 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808321+0000", 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1962000000000002e-05 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808363+0000", 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1434999999999999e-05 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.193 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:257 3.21 3:86aaefcc:::10000000002.000000fe:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.647640661000001, 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.11301415099999999, 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 257 2024-09-19T05:44:09.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498941+0000", 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9350000000000001e-06 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521369+0000", 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022427589000000001 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521386+0000", 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6986999999999998e-05 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521393+0000", 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8569999999999997e-06 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521454+0000", 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1571999999999999e-05 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521577+0000", 2024-09-19T05:44:09.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122998 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521744+0000", 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016694100000000001 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.527047+0000", 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053028559999999999 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611743+0000", 2024-09-19T05:44:09.198 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.084696294000000005 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611895+0000", 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015154900000000001 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611954+0000", 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8573000000000003e-05 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.199 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:273 3.21 3:86f42aac:::10000000002.0000000e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.543245412999999, 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.13998105, 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.200 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 273 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:09.201 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603338+0000", 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624529+0000", 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.021190186 2024-09-19T05:44:09.202 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.203 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.203 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.203 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624551+0000", 2024-09-19T05:44:09.203 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.213e-05 2024-09-19T05:44:09.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624562+0000", 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1452e-05 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624630+0000", 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7738999999999996e-05 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.204 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624721+0000", 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1168999999999994e-05 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624854+0000", 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013313600000000001 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.630092+0000", 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052376819999999996 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.205 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743176+0000", 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.113084107 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743260+0000", 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4309999999999997e-05 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743316+0000", 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5448999999999997e-05 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.206 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:276 3.21 3:85380590:::10000000002.00000011:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.521988148, 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15545139099999999, 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.207 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 276 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.208 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624594+0000", 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0959999999999999e-06 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650750+0000", 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.026156266000000001 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.209 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650772+0000", 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2027000000000001e-05 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650780+0000", 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650866+0000", 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5521999999999999e-05 2024-09-19T05:44:09.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650959+0000", 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3524999999999996e-05 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651140+0000", 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018071799999999999 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.658731+0000", 2024-09-19T05:44:09.211 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075914989999999998 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.779803+0000", 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12107206399999999 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.779912+0000", 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108564 2024-09-19T05:44:09.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780043+0000", 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013128900000000001 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:279 3.21 3:87fa6b7a:::10000000002.00000014:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.495779667000001, 2024-09-19T05:44:09.213 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17630584099999999, 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 279 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650803+0000", 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.232e-06 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684038+0000", 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.033235812000000003 2024-09-19T05:44:09.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684068+0000", 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9118000000000001e-05 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684080+0000", 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2704999999999999e-05 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.216 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684157+0000", 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6879999999999996e-05 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684261+0000", 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010367 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684423+0000", 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016253799999999999 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.217 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.689113+0000", 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046892729999999999 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.826966+0000", 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.137853054 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.827028+0000", 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2030000000000001e-05 2024-09-19T05:44:09.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.827106+0000", 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8529000000000007e-05 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:295 3.2e 3:75eea580:::10000000002.00000024:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.462469779999999, 2024-09-19T05:44:09.219 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52131509399999998, 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 295 2024-09-19T05:44:09.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.221 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684121+0000", 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0455000000000001e-05 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715121+0000", 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031000119 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715142+0000", 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0871999999999999e-05 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.222 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715152+0000", 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.064e-05 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715284+0000", 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013133 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715363+0000", 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9676000000000001e-05 2024-09-19T05:44:09.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715467+0000", 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000103816 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.722648+0000", 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071811460000000002 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205361+0000", 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48271265600000002 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205396+0000", 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4638000000000003e-05 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205425+0000", 2024-09-19T05:44:09.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9745999999999999e-05 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:318 3.c e54/20 3:31d8a57c:::10000000002.0000003b:head v 54'5, mlcod=53'3)", 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.434522040999999, 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035734850999999998, 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.226 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712059+0000", 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.471e-06 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742630+0000", 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.030570631000000001 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742640+0000", 2024-09-19T05:44:09.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0264e-05 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742648+0000", 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1890000000000001e-06 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742738+0000", 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0172000000000003e-05 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742760+0000", 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2005e-05 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747734+0000", 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049739600000000004 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747793+0000", 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9159000000000001e-05 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.230 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:321 3.b 3:d381a89f:::10000000002.0000003e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.431402800000001, 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.038149887, 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.231 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 321 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715179+0000", 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.08e-06 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746761+0000", 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031581559000000002 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746774+0000", 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2994e-05 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746798+0000", 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4443999999999998e-05 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746852+0000", 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3492000000000002e-05 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746956+0000", 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104455 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747084+0000", 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012780800000000001 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.752122+0000", 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050380809999999998 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753229+0000", 2024-09-19T05:44:09.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011063399999999999 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753281+0000", 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2618999999999999e-05 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753327+0000", 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6014999999999999e-05 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:324 3.b 3:d0493c82:::10000000002.00000041:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.399752017999999, 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035384118999999999, 2024-09-19T05:44:09.237 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 324 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746830+0000", 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0839999999999999e-06 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774044+0000", 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.027214075000000001 2024-09-19T05:44:09.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774064+0000", 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0020000000000001e-05 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774074+0000", 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3789999999999995e-06 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774161+0000", 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7663000000000004e-05 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774271+0000", 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110229 2024-09-19T05:44:09.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774421+0000", 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014956099999999999 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780042+0000", 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056212520000000002 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782100+0000", 2024-09-19T05:44:09.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002057513 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782160+0000", 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0513999999999997e-05 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782212+0000", 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1829000000000001e-05 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:339 3.b 3:d24a1766:::10000000002.00000050:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.372481884999999, 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.032212151000000001, 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 339 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.244 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774100+0000", 2024-09-19T05:44:09.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.968e-06 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796613+0000", 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022512880999999998 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796670+0000", 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7011e-05 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796685+0000", 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5208000000000001e-05 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796884+0000", 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000198641 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.797116+0000", 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232368 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.797291+0000", 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017451899999999999 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.802626+0000", 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005335528 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806209+0000", 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035828069999999999 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806260+0000", 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0961999999999998e-05 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806310+0000", 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0257999999999999e-05 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:380 3.21 3:844ac05a:::10000000002.00000079:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.349830978, 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.063018490999999996, 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.250 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 380 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796752+0000", 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.078e-06 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816450+0000", 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019697531000000001 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816475+0000", 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5420000000000001e-05 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816505+0000", 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0199e-05 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816597+0000", 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1739999999999999e-05 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816725+0000", 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000128116 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816874+0000", 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148913 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.821849+0000", 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004974493 2024-09-19T05:44:09.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859654+0000", 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.037805041999999997 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859717+0000", 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859768+0000", 2024-09-19T05:44:09.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0772000000000001e-05 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:394 3.2e 3:7768043d:::10000000002.00000087:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.330021768, 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.400247509, 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.256 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 394 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816560+0000", 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.745e-06 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833002+0000", 2024-09-19T05:44:09.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016442149999999999 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833022+0000", 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9446999999999999e-05 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833032+0000", 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0287e-05 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833104+0000", 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2365000000000003e-05 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833189+0000", 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4470999999999998e-05 2024-09-19T05:44:09.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833328+0000", 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138913 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837647+0000", 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043193600000000004 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216711+0000", 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.379063539 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216765+0000", 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4302999999999998e-05 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216806+0000", 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0929000000000001e-05 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:402 3.b 3:d1738051:::10000000002.0000008f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.313521847000001, 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035654618999999999, 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.263 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 402 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833060+0000", 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7209999999999999e-06 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861464+0000", 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.028403690999999998 2024-09-19T05:44:09.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861483+0000", 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8947999999999999e-05 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861494+0000", 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1168e-05 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861660+0000", 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016661000000000001 2024-09-19T05:44:09.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861769+0000", 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108205 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861892+0000", 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012325699999999999 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868420+0000", 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065282420000000001 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868597+0000", 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177195 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868658+0000", 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0844000000000002e-05 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868713+0000", 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4737999999999997e-05 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:513 3.21 3:86aaefcc:::10000000002.000000fe:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 10.285054016, 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.104537124, 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:09.269 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 513 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:09.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861528+0000", 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.875e-06 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877329+0000", 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015801329999999999 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877342+0000", 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2311e-05 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877347+0000", 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8220000000000003e-06 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:09.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877397+0000", 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9227999999999997e-05 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877488+0000", 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1828000000000002e-05 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:09.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877601+0000", 2024-09-19T05:44:09.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011255399999999999 2024-09-19T05:44:09.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:09.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.882452+0000", 2024-09-19T05:44:09.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048512929999999996 2024-09-19T05:44:09.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.275 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:09.275 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.965958+0000", 2024-09-19T05:44:09.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.083506094000000003 2024-09-19T05:44:09.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:09.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966010+0000", 2024-09-19T05:44:09.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1544e-05 2024-09-19T05:44:09.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966063+0000", 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3245000000000003e-05 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.277 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:09.278 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:09.278 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:09.280 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:09.277+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:09.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.207+0000 7fd34245c640 1 Processor -- start 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.207+0000 7fd34245c640 1 -- start start 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd34245c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd33c0cd470 0x7fd33c0c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd34245c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd33c0cb3c0 con 0x7fd33c0cd850 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd34245c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd33c0cb6a0 con 0x7fd33c15e9b0 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd34245c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd33c0cc4d0 con 0x7fd33c0cd470 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd33b7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd33c0cd850 0x7fd33c1548a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55602/0 (socket says 172.21.15.2:55602) 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd33b7fe640 1 -- 172.21.15.2:0/1290169857 learned_addr learned my addr 172.21.15.2:0/1290169857 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd33bfff640 1 --2- 172.21.15.2:0/1290169857 >> v2:172.21.15.123:3300/0 conn(0x7fd33c0cd470 0x7fd33c0c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.209+0000 7fd33bfff640 1 -- 172.21.15.2:0/1290169857 >> v1:172.21.15.123:6789/0 conn(0x7fd33c15e9b0 legacy=0x7fd33c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.210+0000 7fd33bfff640 1 -- 172.21.15.2:0/1290169857 >> v1:172.21.15.2:6789/0 conn(0x7fd33c0cd850 legacy=0x7fd33c1548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.210+0000 7fd33bfff640 1 -- 172.21.15.2:0/1290169857 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c0cc720 con 0x7fd33c0cd470 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.211+0000 7fd33bfff640 1 --2- 172.21.15.2:0/1290169857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 0x7fd33c0c52d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0097a0 tx=0x7fd32c02a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ed5bc19a9453b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.211+0000 7fd33affd640 1 -- 172.21.15.2:0/1290169857 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c037070 con 0x7fd33c0cd470 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.211+0000 7fd33affd640 1 -- 172.21.15.2:0/1290169857 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd32c02ad60 con 0x7fd33c0cd470 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.211+0000 7fd33affd640 1 -- 172.21.15.2:0/1290169857 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32c032660 con 0x7fd33c0cd470 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.212+0000 7fd34245c640 1 -- 172.21.15.2:0/1290169857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 msgr2=0x7fd33c0c52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.212+0000 7fd34245c640 1 --2- 172.21.15.2:0/1290169857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 0x7fd33c0c52d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0097a0 tx=0x7fd32c02a0d0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.212+0000 7fd34245c640 1 -- 172.21.15.2:0/1290169857 shutdown_connections 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.212+0000 7fd34245c640 1 --2- 172.21.15.2:0/1290169857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 0x7fd33c0c52d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.212+0000 7fd34245c640 1 -- 172.21.15.2:0/1290169857 >> 172.21.15.2:0/1290169857 conn(0x7fd33c067b30 msgr2=0x7fd33c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.212+0000 7fd34245c640 1 -- 172.21.15.2:0/1290169857 shutdown_connections 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.212+0000 7fd34245c640 1 -- 172.21.15.2:0/1290169857 wait complete. 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.214+0000 7fd34245c640 1 Processor -- start 2024-09-19T05:44:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.214+0000 7fd34245c640 1 -- start start 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd34245c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 0x7fd33c1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd34245c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd33c0cd850 0x7fd33c157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd34245c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd33c15e9b0 0x7fd33c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd34245c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c155130 con 0x7fd33c0cd850 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd34245c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c154fb0 con 0x7fd33c0cd470 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd34245c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd33c15b1f0 con 0x7fd33c15e9b0 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd33b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd33c0cd850 0x7fd33c157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd33b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd33c0cd850 0x7fd33c157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55176/0 (socket says 172.21.15.2:55176) 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd33b7fe640 1 -- 172.21.15.2:0/371829070 learned_addr learned my addr 172.21.15.2:0/371829070 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.215+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 0x7fd33c1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd3409f9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd33c15e9b0 0x7fd33c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd33b7fe640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd33c15e9b0 msgr2=0x7fd33c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd33b7fe640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd33c15e9b0 0x7fd33c0c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd33b7fe640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 msgr2=0x7fd33c1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd33b7fe640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd33c0cd470 0x7fd33c1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd33b7fe640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c15e230 con 0x7fd33c0cd850 2024-09-19T05:44:09.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd33b7fe640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd33c0cd850 0x7fd33c157e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd33000c7b0 tx=0x7fd33000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330018070 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3300072d0 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3300047e0 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.216+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c158360 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.217+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c159c40 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.218+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd33c1535a0 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.218+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd330004980 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.219+0000 7fd338ff9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd32806c2b0 0x7fd32806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.219+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5741+0+0 (secure 0 0 0) 0x7fd33008a8b0 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.219+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v59) ==== 54+0+16 (secure 0 0 0) 0x7fd33c1535a0 con 0x7fd33c0cd850 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.219+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd32806c2b0 0x7fd32806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.219+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd33c0c8220 0x7fd33c159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.219+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd33c0cbdf0 con 0x7fd33c0c8220 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.220+0000 7fd3409f9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd33c0c8220 0x7fd33c159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.220+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd32806c2b0 0x7fd32806e770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd32c034000 tx=0x7fd32c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.220+0000 7fd3409f9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd33c0c8220 0x7fd33c159e40 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.220+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c0cbdf0 con 0x7fd33c0c8220 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.237+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c15b520 con 0x7fd33c0c8220 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.238+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c15b520 con 0x7fd33c0c8220 2024-09-19T05:44:09.290 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:09.291 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:09.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.239+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd33c0c87e0 0x7fd33c0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.239+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd33c06d8d0 con 0x7fd33c0c87e0 2024-09-19T05:44:09.291 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.239+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd33c0c87e0 0x7fd33c0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.239+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd33c0c87e0 0x7fd33c0c8bc0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.240+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c06d8d0 con 0x7fd33c0c87e0 2024-09-19T05:44:09.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.254+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c15a750 con 0x7fd33c0c87e0 2024-09-19T05:44:09.292 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.256+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c15a750 con 0x7fd33c0c87e0 2024-09-19T05:44:09.292 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:09.293 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:09.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.256+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd33c1f1220 0x7fd33c1f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.256+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd33c1524a0 con 0x7fd33c1f1220 2024-09-19T05:44:09.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.256+0000 7fd33b7fe640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd33c1f1220 0x7fd33c1f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.256+0000 7fd33b7fe640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd33c1f1220 0x7fd33c1f1600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.257+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c1524a0 con 0x7fd33c1f1220 2024-09-19T05:44:09.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.271+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c062630 con 0x7fd33c1f1220 2024-09-19T05:44:09.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.272+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c062630 con 0x7fd33c1f1220 2024-09-19T05:44:09.294 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:09.295 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:09.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.272+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd33c0630b0 0x7fd33c158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.272+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd33c0634b0 con 0x7fd33c0630b0 2024-09-19T05:44:09.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.273+0000 7fd3409f9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd33c0630b0 0x7fd33c158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.273+0000 7fd3409f9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd33c0630b0 0x7fd33c158530 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.273+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c0634b0 con 0x7fd33c0630b0 2024-09-19T05:44:09.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.287+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c0634b0 con 0x7fd33c0630b0 2024-09-19T05:44:09.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.288+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c0634b0 con 0x7fd33c0630b0 2024-09-19T05:44:09.296 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:09.297 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:09.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.288+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd33c02c640 0x7fd33c02caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.288+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd33c0634b0 con 0x7fd33c02c640 2024-09-19T05:44:09.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.289+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd33c02c640 0x7fd33c02caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.289+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd33c02c640 0x7fd33c02caa0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.290+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c0634b0 con 0x7fd33c02c640 2024-09-19T05:44:09.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.303+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c0634b0 con 0x7fd33c02c640 2024-09-19T05:44:09.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.305+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c0634b0 con 0x7fd33c02c640 2024-09-19T05:44:09.308 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:09.309 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:09.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.305+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd33c02d4f0 0x7fd33c2076d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.305+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fd33c158d10 con 0x7fd33c02d4f0 2024-09-19T05:44:09.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.306+0000 7fd33b7fe640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd33c02d4f0 0x7fd33c2076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.306+0000 7fd33b7fe640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd33c02d4f0 0x7fd33c2076d0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.307+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c158d10 con 0x7fd33c02d4f0 2024-09-19T05:44:09.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.321+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c158d10 con 0x7fd33c02d4f0 2024-09-19T05:44:09.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.322+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c158d10 con 0x7fd33c02d4f0 2024-09-19T05:44:09.325 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:09.325 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:09.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.323+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd33c02d950 0x7fd33c20a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.323+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fd33c20cf00 con 0x7fd33c02d950 2024-09-19T05:44:09.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.323+0000 7fd3409f9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd33c02d950 0x7fd33c20a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.324+0000 7fd3409f9640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd33c02d950 0x7fd33c20a130 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.324+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c20cf00 con 0x7fd33c02d950 2024-09-19T05:44:09.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.338+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c20a670 con 0x7fd33c02d950 2024-09-19T05:44:09.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.340+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c20a670 con 0x7fd33c02d950 2024-09-19T05:44:09.342 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:09.343 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:09.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.340+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd33c20ab80 0x7fd33c20cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.340+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fd33c20ff30 con 0x7fd33c20ab80 2024-09-19T05:44:09.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.340+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd33c20ab80 0x7fd33c20cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.341+0000 7fd33bfff640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd33c20ab80 0x7fd33c20cf70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.341+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33c20ff30 con 0x7fd33c20ab80 2024-09-19T05:44:09.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.355+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd33c075640 con 0x7fd33c20ab80 2024-09-19T05:44:09.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd338ff9640 1 -- 172.21.15.2:0/371829070 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd33c075640 con 0x7fd33c20ab80 2024-09-19T05:44:09.359 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:09.360 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd33c0c8220 msgr2=0x7fd33c159e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fd33c0c8220 0x7fd33c159e40 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd33c0c87e0 msgr2=0x7fd33c0c8bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd33c0c87e0 0x7fd33c0c8bc0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd33c1f1220 msgr2=0x7fd33c1f1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fd33c1f1220 0x7fd33c1f1600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd33c0630b0 msgr2=0x7fd33c158530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fd33c0630b0 0x7fd33c158530 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd33c02c640 msgr2=0x7fd33c02caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fd33c02c640 0x7fd33c02caa0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd33c02d4f0 msgr2=0x7fd33c2076d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd3409f9640 1 -- 172.21.15.2:0/371829070 reap_dead start 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fd33c02d4f0 0x7fd33c2076d0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd3409f9640 1 -- 172.21.15.2:0/371829070 reap_dead start 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd33c02d950 msgr2=0x7fd33c20a130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fd33c02d950 0x7fd33c20a130 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd33c20ab80 msgr2=0x7fd33c20cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fd33c20ab80 0x7fd33c20cf70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd3409f9640 1 -- 172.21.15.2:0/371829070 reap_dead start 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd32806c2b0 msgr2=0x7fd32806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd32806c2b0 0x7fd32806e770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd32c034000 tx=0x7fd32c02e000 comp rx=0 tx=0).stop 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd33c0cd850 msgr2=0x7fd33c157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd33c0cd850 0x7fd33c157e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd33000c7b0 tx=0x7fd33000cc70 comp rx=0 tx=0).stop 2024-09-19T05:44:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 shutdown_connections 2024-09-19T05:44:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd32806c2b0 0x7fd32806e770 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 --2- 172.21.15.2:0/371829070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd33c0cd850 0x7fd33c157e20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 >> 172.21.15.2:0/371829070 conn(0x7fd33c067b30 msgr2=0x7fd33c150810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 shutdown_connections 2024-09-19T05:44:09.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.357+0000 7fd34245c640 1 -- 172.21.15.2:0/371829070 wait complete. 2024-09-19T05:44:09.379 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:09.378+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:09.403 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:09.403 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:09.412 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:09.496 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:09.494+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:09.559 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:09.559 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:09.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:09.609 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:09.608+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:09.683 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:09.681+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:09.715 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:09.719 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:44:09.724 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:09.784 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:09.782+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:09.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.821+0000 7f1ca1edd640 1 Processor -- start 2024-09-19T05:44:09.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.821+0000 7f1ca1edd640 1 -- start start 2024-09-19T05:44:09.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.821+0000 7f1ca1edd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c06ea80 0x7f1c9c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.821+0000 7f1ca1edd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c9c071110 0x7f1c9c0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.821+0000 7f1ca1edd640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9c0706c0 con 0x7f1c9c0b0ee0 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1ca1edd640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9c06e6d0 con 0x7f1c9c071110 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1ca1edd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c15b7d0 con 0x7f1c9c06ea80 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9b7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f1c9c0b0ee0 0x7f1c9c0ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55610/0 (socket says 172.21.15.2:55610) 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9b7fe640 1 -- 172.21.15.2:0/3094867676 learned_addr learned my addr 172.21.15.2:0/3094867676 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9bfff640 1 --2- 172.21.15.2:0/3094867676 >> v2:172.21.15.123:3300/0 conn(0x7f1c9c071110 0x7f1c9c0c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9affd640 1 --2- 172.21.15.2:0/3094867676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c06ea80 0x7f1c9c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9a7fc640 1 -- 172.21.15.2:0/3094867676 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4071866863 0 0) 0x7f1c9c0706c0 con 0x7f1c9c0b0ee0 2024-09-19T05:44:09.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9a7fc640 1 -- 172.21.15.2:0/3094867676 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c84003520 con 0x7f1c9c0b0ee0 2024-09-19T05:44:09.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9a7fc640 1 -- 172.21.15.2:0/3094867676 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4177861878 0 0) 0x7f1c84003520 con 0x7f1c9c0b0ee0 2024-09-19T05:44:09.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9affd640 1 -- 172.21.15.2:0/3094867676 >> v2:172.21.15.123:3300/0 conn(0x7f1c9c071110 msgr2=0x7f1c9c0c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:09.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9affd640 1 --2- 172.21.15.2:0/3094867676 >> v2:172.21.15.123:3300/0 conn(0x7f1c9c071110 0x7f1c9c0c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9affd640 1 -- 172.21.15.2:0/3094867676 >> v1:172.21.15.2:6789/0 conn(0x7f1c9c0b0ee0 legacy=0x7f1c9c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9affd640 1 -- 172.21.15.2:0/3094867676 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c070960 con 0x7f1c9c06ea80 2024-09-19T05:44:09.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.822+0000 7f1c9bfff640 1 --2- 172.21.15.2:0/3094867676 >> v2:172.21.15.123:3300/0 conn(0x7f1c9c071110 0x7f1c9c0c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:09.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.823+0000 7f1c9affd640 1 --2- 172.21.15.2:0/3094867676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c06ea80 0x7f1c9c06cd20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c070890 tx=0x7f1c9002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da5976c3e5a6111e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.823+0000 7f1c9a7fc640 1 -- 172.21.15.2:0/3094867676 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90037070 con 0x7f1c9c06ea80 2024-09-19T05:44:09.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.823+0000 7f1c9a7fc640 1 -- 172.21.15.2:0/3094867676 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9002ac30 con 0x7f1c9c06ea80 2024-09-19T05:44:09.828 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.823+0000 7f1c9a7fc640 1 -- 172.21.15.2:0/3094867676 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c001670 con 0x7f1c9c06ea80 2024-09-19T05:44:09.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 -- 172.21.15.2:0/3094867676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c06ea80 msgr2=0x7f1c9c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/3094867676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c06ea80 0x7f1c9c06cd20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c070890 tx=0x7f1c9002a1a0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 -- 172.21.15.2:0/3094867676 shutdown_connections 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/3094867676 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c06ea80 0x7f1c9c06cd20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/3094867676 >> v2:172.21.15.123:3300/0 conn(0x7f1c9c071110 0x7f1c9c0c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 -- 172.21.15.2:0/3094867676 >> 172.21.15.2:0/3094867676 conn(0x7f1c9c06c240 msgr2=0x7f1c9c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 -- 172.21.15.2:0/3094867676 shutdown_connections 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.824+0000 7f1ca1edd640 1 -- 172.21.15.2:0/3094867676 wait complete. 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.825+0000 7f1ca1edd640 1 Processor -- start 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.825+0000 7f1ca1edd640 1 -- start start 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1ca1edd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9c06ea80 0x7f1c9c1f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1ca1edd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c9c071110 0x7f1c9c0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1ca1edd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c0b0ee0 0x7f1c9c0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1ca1edd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c15bab0 con 0x7f1c9c071110 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1ca1edd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c0703b0 con 0x7f1c9c06ea80 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1ca1edd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9c0b12c0 con 0x7f1c9c0b0ee0 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1c9affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c9c071110 0x7f1c9c0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1c9affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c9c071110 0x7f1c9c0b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55184/0 (socket says 172.21.15.2:55184) 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1c9affd640 1 -- 172.21.15.2:0/2792670851 learned_addr learned my addr 172.21.15.2:0/2792670851 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1c9b7fe640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9c06ea80 0x7f1c9c1f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1c9bfff640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c0b0ee0 0x7f1c9c0c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.826+0000 7f1c9affd640 1 -- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c0b0ee0 msgr2=0x7f1c9c0c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c9affd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c0b0ee0 0x7f1c9c0c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c9affd640 1 -- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9c06ea80 msgr2=0x7f1c9c1f16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c9affd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9c06ea80 0x7f1c9c1f16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c9affd640 1 -- 172.21.15.2:0/2792670851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c0c5890 con 0x7f1c9c071110 2024-09-19T05:44:09.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c9affd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c9c071110 0x7f1c9c0b2b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009890 tx=0x7f1c900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c83fff640 1 -- 172.21.15.2:0/2792670851 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90041020 con 0x7f1c9c071110 2024-09-19T05:44:09.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c83fff640 1 -- 172.21.15.2:0/2792670851 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c9003d640 con 0x7f1c9c071110 2024-09-19T05:44:09.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1c83fff640 1 -- 172.21.15.2:0/2792670851 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90037040 con 0x7f1c9c071110 2024-09-19T05:44:09.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c1e5590 con 0x7f1c9c071110 2024-09-19T05:44:09.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.827+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c1e5850 con 0x7f1c9c071110 2024-09-19T05:44:09.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.828+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c0c98c0 con 0x7f1c9c071110 2024-09-19T05:44:09.833 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.829+0000 7f1c83fff640 1 -- 172.21.15.2:0/2792670851 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1c90044020 con 0x7f1c9c071110 2024-09-19T05:44:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.829+0000 7f1c83fff640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1c8806c2e0 0x7f1c8806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.829+0000 7f1c83fff640 1 -- 172.21.15.2:0/2792670851 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5741+0+0 (secure 0 0 0) 0x7f1c900ab5d0 con 0x7f1c9c071110 2024-09-19T05:44:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.829+0000 7f1c9b7fe640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1c8806c2e0 0x7f1c8806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:09.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.830+0000 7f1c9b7fe640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1c8806c2e0 0x7f1c8806e7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005880 tx=0x7f1c8c0057f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:09.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:09.834+0000 7f1c83fff640 1 -- 172.21.15.2:0/2792670851 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1c9004b070 con 0x7f1c9c071110 2024-09-19T05:44:09.874 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:09.874 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:09.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:09.884 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:09.882+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:09.986 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:09.984+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:09.999 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:10.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.021+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f1c9c0679f0 con 0x7f1c9c071110 2024-09-19T05:44:10.087 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:10.085+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:10.106 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:10.107 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:10.115 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:10.200 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:10.198+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:10.264 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:10.264 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:10.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:10.288 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:10.286+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:10.389 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:10.387+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:10.421 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:10.421 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:10.430 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:10.490 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:10.489+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:10.539 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:10.539 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:10.548 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:10.591 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:10.589+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:10.656 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:10.656 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:10.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:10.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.702+0000 7f1c83fff640 1 -- 172.21.15.2:0/2792670851 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v60) ==== 101+0+0 (secure 0 0 0) 0x7f1c90079d20 con 0x7f1c9c071110 2024-09-19T05:44:10.704 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:44:10.704 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:10.702+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:10.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1c8806c2e0 msgr2=0x7f1c8806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:10.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1c8806c2e0 0x7f1c8806e7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c005880 tx=0x7f1c8c0057f0 comp rx=0 tx=0).stop 2024-09-19T05:44:10.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c9c071110 msgr2=0x7f1c9c0b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:10.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c9c071110 0x7f1c9c0b2b00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009890 tx=0x7f1c900046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:10.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 shutdown_connections 2024-09-19T05:44:10.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1c8806c2e0 0x7f1c8806e7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:10.707 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c9c0b0ee0 0x7f1c9c0c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:10.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c9c071110 0x7f1c9c0b2b00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:10.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 --2- 172.21.15.2:0/2792670851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c9c06ea80 0x7f1c9c1f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:10.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 >> 172.21.15.2:0/2792670851 conn(0x7f1c9c06c240 msgr2=0x7f1c9c1e6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:10.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 shutdown_connections 2024-09-19T05:44:10.708 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:10.705+0000 7f1ca1edd640 1 -- 172.21.15.2:0/2792670851 wait complete. 2024-09-19T05:44:10.746 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:10.746 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:10.746 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:10.746 INFO:tasks.workunit.client.0.smithi002.stdout:iozone test complete. 2024-09-19T05:44:10.747 INFO:tasks.workunit.client.0.smithi002.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-19T05:44:10.776 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:10.776 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:10.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:10.792 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:10.791+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Iozone: Performance Test of File I/O 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Version $Revision: 3.494 $ 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Compiled for 64 bit mode. 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Build: linux-AMD64 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T05:44:10.813 INFO:tasks.workunit.client.0.smithi002.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Run began: Thu Sep 19 05:44:10 2024 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Include close in write timing 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Include fsync in write timing 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: File size set to 1048576 kB 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Record Size 1024 kB 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Output is in kBytes/sec 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T05:44:10.814 INFO:tasks.workunit.client.0.smithi002.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T05:44:10.815 INFO:tasks.workunit.client.0.smithi002.stdout: Processor cache line size set to 32 bytes. 2024-09-19T05:44:10.815 INFO:tasks.workunit.client.0.smithi002.stdout: File stride size set to 17 * record size. 2024-09-19T05:44:10.815 INFO:tasks.workunit.client.0.smithi002.stdout: Throughput test with 1 process 2024-09-19T05:44:10.815 INFO:tasks.workunit.client.0.smithi002.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-19T05:44:10.893 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:10.891+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:10.927 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:10.927 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:10.937 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:10.994 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:10.992+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:11.053 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:11.053 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:11.062 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:11.106 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:11.105+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:11.194 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:11.195 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:11.195 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:11.193+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:11.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:11.297 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:11.296+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:11.371 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:11.431 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:11.432 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:11.441 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:11.423+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:11.451 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:11.500 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:11.497+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:11.598 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:11.596+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:11.627 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:11.628 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:11.646 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.644+0000 7f8d28c23640 1 Processor -- start 2024-09-19T05:44:11.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.644+0000 7f8d28c23640 1 -- start start 2024-09-19T05:44:11.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d28c23640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d24158fe0 0x7f8d2406c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d28c23640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d24070cb0 0x7f8d2406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d28c23640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d2406fe00 con 0x7f8d240c8770 2024-09-19T05:44:11.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d28c23640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8d240700e0 con 0x7f8d24158fe0 2024-09-19T05:44:11.649 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d28c23640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8d240703c0 con 0x7f8d24070cb0 2024-09-19T05:44:11.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d2259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d24158fe0 0x7f8d2406c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d2259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d24158fe0 0x7f8d2406c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33090/0 (socket says 172.21.15.2:33090) 2024-09-19T05:44:11.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.647+0000 7f8d2259c640 1 -- 172.21.15.2:0/3504787170 learned_addr learned my addr 172.21.15.2:0/3504787170 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:11.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.648+0000 7f8d2259c640 1 -- 172.21.15.2:0/3504787170 >> v2:172.21.15.123:3300/0 conn(0x7f8d24070cb0 msgr2=0x7f8d2406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:11.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.648+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/3504787170 >> v2:172.21.15.123:3300/0 conn(0x7f8d24070cb0 0x7f8d2406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.648+0000 7f8d2259c640 1 --2- 172.21.15.2:0/3504787170 >> v2:172.21.15.123:3300/0 conn(0x7f8d24070cb0 0x7f8d2406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.648+0000 7f8d2259c640 1 -- 172.21.15.2:0/3504787170 >> v1:172.21.15.123:6789/0 conn(0x7f8d240c8770 legacy=0x7f8d240c8ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:44:11.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.648+0000 7f8d2259c640 1 -- 172.21.15.2:0/3504787170 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d240706a0 con 0x7f8d24158fe0 2024-09-19T05:44:11.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.648+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/3504787170 >> v2:172.21.15.123:3300/0 conn(0x7f8d24070cb0 0x7f8d2406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:11.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.649+0000 7f8d2259c640 1 --2- 172.21.15.2:0/3504787170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d24158fe0 0x7f8d2406c130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8d2415c9c0 tx=0x7f8d0c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dfbfbaf3ca7211 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.650+0000 7f8d2159a640 1 -- 172.21.15.2:0/3504787170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c037070 con 0x7f8d24158fe0 2024-09-19T05:44:11.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.650+0000 7f8d2159a640 1 -- 172.21.15.2:0/3504787170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d0c032440 con 0x7f8d24158fe0 2024-09-19T05:44:11.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.650+0000 7f8d2159a640 1 -- 172.21.15.2:0/3504787170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c032810 con 0x7f8d24158fe0 2024-09-19T05:44:11.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 -- 172.21.15.2:0/3504787170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d24158fe0 msgr2=0x7f8d2406c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 --2- 172.21.15.2:0/3504787170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d24158fe0 0x7f8d2406c130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8d2415c9c0 tx=0x7f8d0c02a230 comp rx=0 tx=0).stop 2024-09-19T05:44:11.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 -- 172.21.15.2:0/3504787170 shutdown_connections 2024-09-19T05:44:11.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 --2- 172.21.15.2:0/3504787170 >> v2:172.21.15.123:3300/0 conn(0x7f8d24070cb0 0x7f8d2406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 --2- 172.21.15.2:0/3504787170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d24158fe0 0x7f8d2406c130 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 -- 172.21.15.2:0/3504787170 >> 172.21.15.2:0/3504787170 conn(0x7f8d24072ad0 msgr2=0x7f8d24072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:11.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 -- 172.21.15.2:0/3504787170 shutdown_connections 2024-09-19T05:44:11.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.651+0000 7f8d28c23640 1 -- 172.21.15.2:0/3504787170 wait complete. 2024-09-19T05:44:11.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.654+0000 7f8d28c23640 1 Processor -- start 2024-09-19T05:44:11.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.654+0000 7f8d28c23640 1 -- start start 2024-09-19T05:44:11.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.656+0000 7f8d28c23640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d24070cb0 0x7f8d241eba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d28c23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d240c8770 0x7f8d2415b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d28c23640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d24158fe0 0x7f8d24151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d28c23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8d241593c0 con 0x7f8d240c8770 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d21d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d240c8770 0x7f8d2415b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d21d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d240c8770 0x7f8d2415b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33094/0 (socket says 172.21.15.2:33094) 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d21d9b640 1 -- 172.21.15.2:0/2763824614 learned_addr learned my addr 172.21.15.2:0/2763824614 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d240c93b0 con 0x7f8d24070cb0 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8d24159540 con 0x7f8d24158fe0 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d24070cb0 0x7f8d241eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.657+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d24158fe0 0x7f8d24151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d21d9b640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d24158fe0 msgr2=0x7f8d24151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d24158fe0 0x7f8d24151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d21d9b640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d24070cb0 msgr2=0x7f8d241eba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d24070cb0 0x7f8d241eba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d21d9b640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2415c180 con 0x7f8d240c8770 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d24158fe0 0x7f8d24151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d24070cb0 0x7f8d241eba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d240c8770 0x7f8d2415b540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400c820 tx=0x7f8d1400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14015070 con 0x7f8d240c8770 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d14007a30 con 0x7f8d240c8770 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14007de0 con 0x7f8d240c8770 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d241eac30 con 0x7f8d240c8770 2024-09-19T05:44:11.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.659+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2415c420 con 0x7f8d240c8770 2024-09-19T05:44:11.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.662+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8d14018070 con 0x7f8d240c8770 2024-09-19T05:44:11.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.662+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8d240c8ba0 con 0x7f8d240c8770 2024-09-19T05:44:11.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.663+0000 7f8d1affd640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d0406c2e0 0x7f8d0406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.663+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f8d1408b430 con 0x7f8d240c8770 2024-09-19T05:44:11.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.663+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v60) ==== 54+0+16 (secure 0 0 0) 0x7f8d1408d950 con 0x7f8d240c8770 2024-09-19T05:44:11.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.663+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d2415ba80 0x7f8d24154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.663+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d24157730 con 0x7f8d2415ba80 2024-09-19T05:44:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.664+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d0406c2e0 0x7f8d0406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.664+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d2415ba80 0x7f8d24154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.664+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d2415ba80 0x7f8d24154b60 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.665+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d24157730 con 0x7f8d2415ba80 2024-09-19T05:44:11.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.667+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d0406c2e0 0x7f8d0406e7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c034000 tx=0x7f8d0c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.699 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:11.698+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.704+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d24157450 con 0x7f8d2415ba80 2024-09-19T05:44:11.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.708+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d24157450 con 0x7f8d2415ba80 2024-09-19T05:44:11.709 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.708+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d241ea5e0 0x7f8d24159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.708+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8d24157170 con 0x7f8d241ea5e0 2024-09-19T05:44:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.709+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d241ea5e0 0x7f8d24159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.709+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d241ea5e0 0x7f8d24159e70 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.711 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.710+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d24157170 con 0x7f8d241ea5e0 2024-09-19T05:44:11.714 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:44:11.714 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:44:11.714 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-19T05:44:11.714 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-19T05:44:11.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.740+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d241f55f0 con 0x7f8d241ea5e0 2024-09-19T05:44:11.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.743+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d241f55f0 con 0x7f8d241ea5e0 2024-09-19T05:44:11.744 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:11.745 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:11.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.743+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d241f8380 0x7f8d241f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.743+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8d240cb1f0 con 0x7f8d241f8380 2024-09-19T05:44:11.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.743+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d241f8380 0x7f8d241f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.744+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d241f8380 0x7f8d241f87a0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.745+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d240cb1f0 con 0x7f8d241f8380 2024-09-19T05:44:11.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.761+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d240cb6d0 con 0x7f8d241f8380 2024-09-19T05:44:11.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.763+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d240cb6d0 con 0x7f8d241f8380 2024-09-19T05:44:11.765 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:11.765 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:11.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.764+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d24062630 0x7f8d24062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.764+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8d240cc1b0 con 0x7f8d24062630 2024-09-19T05:44:11.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.764+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d24062630 0x7f8d24062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.764+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d24062630 0x7f8d24062a50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.766+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d1c01a880 con 0x7f8d24062630 2024-09-19T05:44:11.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.782+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d240631e0 con 0x7f8d24062630 2024-09-19T05:44:11.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.784+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d240631e0 con 0x7f8d24062630 2024-09-19T05:44:11.785 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:11.786 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:11.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.784+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d24063350 0x7f8d2402cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.784+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8d2402d090 con 0x7f8d24063350 2024-09-19T05:44:11.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.785+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d24063350 0x7f8d2402cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.786+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d24063350 0x7f8d2402cb50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.788+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d2402d090 con 0x7f8d24063350 2024-09-19T05:44:11.800 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:11.799+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:11.802 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:11.802 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 initial writers = 1098571.88 kB/sec 2024-09-19T05:44:11.802 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 initial writers = 1098332.62 kB/sec 2024-09-19T05:44:11.802 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 1098571.88 kB/sec 2024-09-19T05:44:11.802 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 1098571.88 kB/sec 2024-09-19T05:44:11.802 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 1098571.88 kB/sec 2024-09-19T05:44:11.802 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:44:11.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.805+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d2402d090 con 0x7f8d24063350 2024-09-19T05:44:11.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.807+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d2402d090 con 0x7f8d24063350 2024-09-19T05:44:11.808 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:11.808 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:11.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.807+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d2402d7e0 0x7f8d2420b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.807+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8d2420de80 con 0x7f8d2402d7e0 2024-09-19T05:44:11.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.807+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d2402d7e0 0x7f8d2420b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.807+0000 7f8d21d9b640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d2402d7e0 0x7f8d2420b0b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.808+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d2420de80 con 0x7f8d2402d7e0 2024-09-19T05:44:11.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.822+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d2420b5f0 con 0x7f8d2402d7e0 2024-09-19T05:44:11.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.824+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d2420b5f0 con 0x7f8d2402d7e0 2024-09-19T05:44:11.825 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:11.825 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:11.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.824+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d2420bc70 0x7f8d2420e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.824+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8d24210f40 con 0x7f8d2420bc70 2024-09-19T05:44:11.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.824+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d2420bc70 0x7f8d2420e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.825+0000 7f8d22d9d640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d2420bc70 0x7f8d2420e060 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.826 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.825+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d24210f40 con 0x7f8d2420bc70 2024-09-19T05:44:11.830 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:11.831 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:11.831 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:11.831 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:11.831 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:11.841 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:11.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.839+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d2420e5a0 con 0x7f8d2420bc70 2024-09-19T05:44:11.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.841+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d2420e5a0 con 0x7f8d2420bc70 2024-09-19T05:44:11.842 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:11.843 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:11.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.841+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d2420ecc0 0x7f8d2420f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.841+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8d24214310 con 0x7f8d2420ecc0 2024-09-19T05:44:11.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.841+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d2420ecc0 0x7f8d2420f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.842+0000 7f8d2259c640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d2420ecc0 0x7f8d2420f120 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.842+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d24214310 con 0x7f8d2420ecc0 2024-09-19T05:44:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.848+0000 7f3c64fd8640 1 Processor -- start 2024-09-19T05:44:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.848+0000 7f3c64fd8640 1 -- start start 2024-09-19T05:44:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c64fd8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60152b80 0x7f3c601591d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c64fd8640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3c600c9000 0x7f3c600ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c64fd8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3c60071110 0x7f3c600ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c64fd8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f3c600c94f0 con 0x7f3c60071110 2024-09-19T05:44:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c64fd8640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3c600ca3a0 con 0x7f3c600c9000 2024-09-19T05:44:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c64fd8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3c600c3750 con 0x7f3c60152b80 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c5ed9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3c60071110 0x7f3c600ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c5e59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60152b80 0x7f3c601591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c5ed9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3c60071110 0x7f3c600ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33108/0 (socket says 172.21.15.2:33108) 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c5ed9d640 1 -- 172.21.15.2:0/1520800179 learned_addr learned my addr 172.21.15.2:0/1520800179 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.849+0000 7f3c5dd9b640 1 --2- 172.21.15.2:0/1520800179 >> v2:172.21.15.123:3300/0 conn(0x7f3c600c9000 0x7f3c600ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.850+0000 7f3c5ed9d640 1 -- 172.21.15.2:0/1520800179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60152b80 msgr2=0x7f3c601591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.850+0000 7f3c5ed9d640 1 --2- 172.21.15.2:0/1520800179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60152b80 0x7f3c601591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.850+0000 7f3c5ed9d640 1 -- 172.21.15.2:0/1520800179 >> v2:172.21.15.123:3300/0 conn(0x7f3c600c9000 msgr2=0x7f3c600ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.850+0000 7f3c5ed9d640 1 --2- 172.21.15.2:0/1520800179 >> v2:172.21.15.123:3300/0 conn(0x7f3c600c9000 0x7f3c600ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.850+0000 7f3c5ed9d640 1 -- 172.21.15.2:0/1520800179 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c600c3a30 con 0x7f3c60071110 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.850+0000 7f3c5ed9d640 1 --2- 172.21.15.2:0/1520800179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c60071110 0x7f3c600ceab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3c600687d0 tx=0x7f3c5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17cf8c60170a29b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c5d59a640 1 -- 172.21.15.2:0/1520800179 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58036070 con 0x7f3c60071110 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c5d59a640 1 -- 172.21.15.2:0/1520800179 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c58031600 con 0x7f3c60071110 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c5d59a640 1 -- 172.21.15.2:0/1520800179 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c580319d0 con 0x7f3c60071110 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1520800179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c60071110 msgr2=0x7f3c600ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 --2- 172.21.15.2:0/1520800179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c60071110 0x7f3c600ceab0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3c600687d0 tx=0x7f3c5802a6a0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1520800179 shutdown_connections 2024-09-19T05:44:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 --2- 172.21.15.2:0/1520800179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c60071110 0x7f3c600ceab0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 --2- 172.21.15.2:0/1520800179 >> v2:172.21.15.123:3300/0 conn(0x7f3c600c9000 0x7f3c600ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 --2- 172.21.15.2:0/1520800179 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60152b80 0x7f3c601591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1520800179 >> 172.21.15.2:0/1520800179 conn(0x7f3c6006c240 msgr2=0x7f3c60067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1520800179 shutdown_connections 2024-09-19T05:44:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.851+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1520800179 wait complete. 2024-09-19T05:44:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.853+0000 7f3c64fd8640 1 Processor -- start 2024-09-19T05:44:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.853+0000 7f3c64fd8640 1 -- start start 2024-09-19T05:44:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.853+0000 7f3c64fd8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60071110 0x7f3c60159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.853+0000 7f3c64fd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c600c9000 0x7f3c600c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.853+0000 7f3c64fd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c60152b80 0x7f3c601553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c64fd8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60158cb0 con 0x7f3c600c9000 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c64fd8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601589d0 con 0x7f3c60152b80 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c64fd8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3c60152f60 con 0x7f3c60071110 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c600c9000 0x7f3c600c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c600c9000 0x7f3c600c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33112/0 (socket says 172.21.15.2:33112) 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 -- 172.21.15.2:0/1484997800 learned_addr learned my addr 172.21.15.2:0/1484997800 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5e59c640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60071110 0x7f3c60159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5ed9d640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c60152b80 0x7f3c601553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 -- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60071110 msgr2=0x7f3c60159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c60071110 0x7f3c60159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 -- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c60152b80 msgr2=0x7f3c601553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c60152b80 0x7f3c601553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 -- 172.21.15.2:0/1484997800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c601531b0 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c5dd9b640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c600c9000 0x7f3c600c81f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400e870 tx=0x7f3c5400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54017070 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c5400ee60 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5400cb60 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.854+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c60154020 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.855+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c601545e0 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.855+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c6014d530 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.856+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3c5401a070 con 0x7f3c600c9000 2024-09-19T05:44:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c4effd640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3c4006c2e0 0x7f3c4006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f3c5408b270 con 0x7f3c600c9000 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c5e59c640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3c4006c2e0 0x7f3c4006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c4effd640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3c40074540 0x7f3c40076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c540068d0 con 0x7f3c40074540 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3c54092050 con 0x7f3c600c9000 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c5ed9d640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3c40074540 0x7f3c40076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c5e59c640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3c4006c2e0 0x7f3c4006e7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009b40 tx=0x7f3c50009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.857+0000 7f3c5ed9d640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3c40074540 0x7f3c40076960 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.858+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c540068d0 con 0x7f3c40074540 2024-09-19T05:44:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.859+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8d24213190 con 0x7f8d2420ecc0 2024-09-19T05:44:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.860+0000 7f8d1affd640 1 -- 172.21.15.2:0/2763824614 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d24213190 con 0x7f8d2420ecc0 2024-09-19T05:44:11.863 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:11.864 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:11.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d2415ba80 msgr2=0x7f8d24154b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8d2415ba80 0x7f8d24154b60 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d241ea5e0 msgr2=0x7f8d24159e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8d241ea5e0 0x7f8d24159e70 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d241f8380 msgr2=0x7f8d241f87a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8d241f8380 0x7f8d241f87a0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d24062630 msgr2=0x7f8d24062a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d24062630 0x7f8d24062a50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d24063350 msgr2=0x7f8d2402cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8d24063350 0x7f8d2402cb50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d2402d7e0 msgr2=0x7f8d2420b0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f8d2402d7e0 0x7f8d2420b0b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d22d9d640 1 -- 172.21.15.2:0/2763824614 reap_dead start 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d2420bc70 msgr2=0x7f8d2420e060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8d2420bc70 0x7f8d2420e060 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d22d9d640 1 -- 172.21.15.2:0/2763824614 reap_dead start 2024-09-19T05:44:11.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d2420ecc0 msgr2=0x7f8d2420f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8d2420ecc0 0x7f8d2420f120 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d22d9d640 1 -- 172.21.15.2:0/2763824614 reap_dead start 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d22d9d640 1 -- 172.21.15.2:0/2763824614 reap_dead start 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d0406c2e0 msgr2=0x7f8d0406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d0406c2e0 0x7f8d0406e7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c034000 tx=0x7f8d0c02e000 comp rx=0 tx=0).stop 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d240c8770 msgr2=0x7f8d2415b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d240c8770 0x7f8d2415b540 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400c820 tx=0x7f8d1400cc20 comp rx=0 tx=0).stop 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 shutdown_connections 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d0406c2e0 0x7f8d0406e7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 --2- 172.21.15.2:0/2763824614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d240c8770 0x7f8d2415b540 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 >> 172.21.15.2:0/2763824614 conn(0x7f8d24072ad0 msgr2=0x7f8d241625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:11.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 shutdown_connections 2024-09-19T05:44:11.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.861+0000 7f8d28c23640 1 -- 172.21.15.2:0/2763824614 wait complete. 2024-09-19T05:44:11.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.875+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f3c600ca5c0 con 0x7f3c40074540 2024-09-19T05:44:11.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.875+0000 7f3c4effd640 1 -- 172.21.15.2:0/1484997800 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f3c600ca5c0 con 0x7f3c40074540 2024-09-19T05:44:11.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3c40074540 msgr2=0x7f3c40076960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3c40074540 0x7f3c40076960 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3c4006c2e0 msgr2=0x7f3c4006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3c4006c2e0 0x7f3c4006e7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009b40 tx=0x7f3c50009190 comp rx=0 tx=0).stop 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c600c9000 msgr2=0x7f3c600c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 --2- 172.21.15.2:0/1484997800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3c600c9000 0x7f3c600c81f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400e870 tx=0x7f3c5400ed30 comp rx=0 tx=0).stop 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c5ed9d640 1 -- 172.21.15.2:0/1484997800 reap_dead start 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 shutdown_connections 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 >> 172.21.15.2:0/1484997800 conn(0x7f3c6006c240 msgr2=0x7f3c6014ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 shutdown_connections 2024-09-19T05:44:11.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:11.876+0000 7f3c64fd8640 1 -- 172.21.15.2:0/1484997800 wait complete. 2024-09-19T05:44:11.887 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-19T05:44:11.900 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:11.899+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:11.946 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:32 3.31 3:8d4d39bf:::10000000002.0000001d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.577455512, 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48911290200000002, 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.947 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 32 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366124+0000", 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366127+0000", 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0429999999999999e-06 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388967+0000", 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022840284999999998 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388978+0000", 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0811e-05 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.388983+0000", 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7879999999999997e-06 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389069+0000", 2024-09-19T05:44:11.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6670999999999997e-05 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389303+0000", 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023413399999999999 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389482+0000", 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017864399999999999 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394972+0000", 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054895530000000003 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855135+0000", 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46016378400000002 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855197+0000", 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1365999999999995e-05 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.855237+0000", 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9823000000000001e-05 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:50 3.17 3:eaf1bb0b:::10000000002.0000002f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.554573179999998, 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15463449900000001, 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 50 2024-09-19T05:44:11.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389006+0000", 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.389008+0000", 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9369999999999998e-06 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414228+0000", 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025220256 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414244+0000", 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5821999999999999e-05 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414262+0000", 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7750000000000001e-05 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414324+0000", 2024-09-19T05:44:11.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2250999999999997e-05 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414475+0000", 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015051800000000001 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414621+0000", 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014692000000000001 2024-09-19T05:44:11.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.419434+0000", 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048125429999999999 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543566+0000", 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12413210600000001 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543602+0000", 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5420000000000003e-05 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.543640+0000", 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8976000000000001e-05 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:88 3.31 3:8c5098ad:::10000000002.00000055:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.485309935, 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41301092299999997, 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.962 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 88 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458269+0000", 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.458271+0000", 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.804e-06 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481919+0000", 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.02364809 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481946+0000", 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7370000000000001e-05 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481957+0000", 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.066e-05 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:11.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482050+0000", 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2991999999999993e-05 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482206+0000", 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000155611 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482359+0000", 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000153728 2024-09-19T05:44:11.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.485588+0000", 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003228557 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871172+0000", 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.38558421500000001 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871237+0000", 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4562999999999998e-05 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.871280+0000", 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3333e-05 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:142 3.31 3:8d07500b:::10000000002.0000008b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.33184984, 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26228585799999998, 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 142 2024-09-19T05:44:11.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611729+0000", 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611732+0000", 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8870000000000001e-06 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623428+0000", 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011695743 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:11.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623449+0000", 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1178e-05 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623463+0000", 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3923e-05 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623546+0000", 2024-09-19T05:44:11.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3090999999999994e-05 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623713+0000", 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000167176 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.623874+0000", 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016018299999999999 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.628673+0000", 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047995529999999998 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.873957+0000", 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24528372500000001 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:11.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.873991+0000", 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4663999999999999e-05 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.874015+0000", 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3734999999999999e-05 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.975 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:173 3.38 3:1ee175b9:::10000000002.000000aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.295808702999999, 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24937071899999999, 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 173 2024-09-19T05:44:11.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647770+0000", 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647773+0000", 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3870000000000002e-06 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659807+0000", 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012033909000000001 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:11.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659823+0000", 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6667000000000002e-05 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659850+0000", 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6798e-05 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.659930+0000", 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0041000000000002e-05 2024-09-19T05:44:11.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.660057+0000", 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127164 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.660179+0000", 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012194 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.664305+0000", 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041255900000000002 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897061+0000", 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.23275601800000001 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:11.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897101+0000", 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0156e-05 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.897141+0000", 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0049000000000002e-05 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.982 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:288 3.31 3:8d4d39bf:::10000000002.0000001d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.295322633, 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54072771799999997, 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 288 2024-09-19T05:44:11.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648257+0000", 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.648258+0000", 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.686e-06 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673526+0000", 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025268142 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673552+0000", 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5500999999999999e-05 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673562+0000", 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.825e-06 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673673+0000", 2024-09-19T05:44:11.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011167000000000001 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673778+0000", 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104987 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673894+0000", 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115639 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.677601+0000", 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0037067900000000002 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188877+0000", 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.51127632700000003 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188935+0000", 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7553000000000003e-05 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188984+0000", 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9598000000000002e-05 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:306 3.17 3:eaf1bb0b:::10000000002.0000002f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.269982838000001, 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17070818600000001, 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 306 2024-09-19T05:44:11.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673596+0000", 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.673598+0000", 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1129999999999999e-06 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692350+0000", 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018751790000000001 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692373+0000", 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2629999999999998e-05 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692385+0000", 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1905999999999999e-05 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:11.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692488+0000", 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010292499999999999 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692599+0000", 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011170100000000001 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692750+0000", 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151099 2024-09-19T05:44:11.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.697776+0000", 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050251590000000004 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844188+0000", 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.146412135 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844251+0000", 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3523999999999997e-05 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.844305+0000", 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3204000000000001e-05 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:11.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:319 3.17 3:ead2f1a2:::10000000002.0000003c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.251169602999999, 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22292820499999999, 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:11.997 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 319 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692410+0000", 2024-09-19T05:44:11.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.692412+0000", 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6900000000000001e-06 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708570+0000", 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016157659000000001 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:11.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708591+0000", 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1260999999999999e-05 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708602+0000", 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0380000000000001e-05 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708663+0000", 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1811999999999997e-05 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708741+0000", 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7385999999999994e-05 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.708853+0000", 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112642 2024-09-19T05:44:12.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715058+0000", 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062048729999999996 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915221+0000", 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.200162954 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915283+0000", 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2059999999999999e-05 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.915338+0000", 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4487999999999997e-05 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:292 3.3b e54/20 3:ddac2f4e:::10000000002.00000021:head v 54'6, mlcod=53'4)", 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.245269595, 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.40806384200000001, 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698310+0000", 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698311+0000", 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3880000000000001e-06 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103237+0000", 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.40492591100000003 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103250+0000", 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3593e-05 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103261+0000", 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0181e-05 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103332+0000", 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.114e-05 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103357+0000", 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5094000000000001e-05 2024-09-19T05:44:12.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.106302+0000", 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0029454260000000001 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.106373+0000", 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1109e-05 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.008 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:344 3.31 3:8c5098ad:::10000000002.00000055:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.216923888, 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.47092658999999998, 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.009 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 344 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726655+0000", 2024-09-19T05:44:12.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.726658+0000", 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.649e-06 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758353+0000", 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031695424999999999 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758372+0000", 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8723e-05 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758387+0000", 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4882000000000001e-05 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758510+0000", 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012282100000000001 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758628+0000", 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118544 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.758867+0000", 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0002385 2024-09-19T05:44:12.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.765925+0000", 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070580670000000003 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197522+0000", 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.43159685599999997 2024-09-19T05:44:12.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197553+0000", 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1319999999999998e-05 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.197582+0000", 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8802999999999998e-05 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.015 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:358 3.17 3:eaf05c3c:::10000000002.00000063:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.163388983000001, 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30462805700000001, 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.016 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 358 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780190+0000", 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780193+0000", 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5660000000000001e-06 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804477+0000", 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024284329 2024-09-19T05:44:12.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804498+0000", 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1251e-05 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804511+0000", 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2234e-05 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804595+0000", 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4651e-05 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804686+0000", 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1062000000000003e-05 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804937+0000", 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00025027200000000001 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.808988+0000", 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040513900000000002 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084691+0000", 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.275703319 2024-09-19T05:44:12.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084761+0000", 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9845999999999995e-05 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.084818+0000", 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7136999999999999e-05 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.022 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:367 3.17 3:e9c25901:::10000000002.0000006c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.139040177, 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.35149704900000001, 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.023 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 367 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804539+0000", 2024-09-19T05:44:12.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.804541+0000", 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9360000000000002e-06 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832876+0000", 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.028334734 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832891+0000", 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5404e-05 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832899+0000", 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1759999999999993e-06 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.832970+0000", 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0500000000000006e-05 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8103000000000007e-05 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833196+0000", 2024-09-19T05:44:12.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138131 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837012+0000", 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038158770000000001 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.155960+0000", 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.318948334 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.155994+0000", 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3630000000000002e-05 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156036+0000", 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2224000000000001e-05 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:390 3.17 3:e85118c5:::10000000002.00000083:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.07259444, 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28513696999999999, 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.030 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 390 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.870985+0000", 2024-09-19T05:44:12.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.871040+0000", 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5596000000000001e-05 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883834+0000", 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012794005000000001 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883858+0000", 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3380999999999999e-05 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883869+0000", 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1245e-05 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883964+0000", 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4749999999999999e-05 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.884049+0000", 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5023000000000002e-05 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.884176+0000", 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012745500000000001 2024-09-19T05:44:12.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.887612+0000", 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034360850000000002 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156071+0000", 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26845824099999999 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156100+0000", 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9368000000000001e-05 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.156122+0000", 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1821e-05 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:392 3.1e 3:7b5aaf6d:::10000000002.00000085:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.059682232, 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.25285440300000001, 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 392 2024-09-19T05:44:12.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883897+0000", 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.883899+0000", 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4399999999999999e-06 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896769+0000", 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012869307 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896783+0000", 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.436e-05 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896793+0000", 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0182e-05 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896887+0000", 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3704999999999998e-05 2024-09-19T05:44:12.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.897039+0000", 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015165299999999999 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.897229+0000", 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000190552 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.901089+0000", 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038593680000000002 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136638+0000", 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.235549007 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136703+0000", 2024-09-19T05:44:12.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5030999999999995e-05 2024-09-19T05:44:12.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.136751+0000", 2024-09-19T05:44:12.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8798000000000003e-05 2024-09-19T05:44:12.044 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:398 3.31 3:8d07500b:::10000000002.0000008b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.046749898, 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.326241485, 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.045 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 398 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896829+0000", 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.896832+0000", 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6410000000000002e-06 2024-09-19T05:44:12.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908746+0000", 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011914118 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908772+0000", 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6149999999999999e-05 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908789+0000", 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6322000000000001e-05 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908897+0000", 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010891 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.908999+0000", 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101216 2024-09-19T05:44:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.909146+0000", 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014757099999999999 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.913702+0000", 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004555875 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.222963+0000", 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30926109600000001 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223026+0000", 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3182999999999995e-05 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223071+0000", 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4403000000000002e-05 2024-09-19T05:44:12.051 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:429 3.38 3:1ee175b9:::10000000002.000000aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.000712961, 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.23704366099999999, 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.052 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 429 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:12.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942866+0000", 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942868+0000", 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5319999999999999e-06 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957357+0000", 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014489149 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957373+0000", 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6575e-05 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957381+0000", 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6410000000000002e-06 2024-09-19T05:44:12.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957443+0000", 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2200999999999996e-05 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957560+0000", 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117044 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.957686+0000", 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012538100000000001 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.961805+0000", 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041189649999999996 2024-09-19T05:44:12.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179828+0000", 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21802357999999999 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179875+0000", 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6609e-05 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.179910+0000", 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4984000000000003e-05 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:434 3.1e 3:789394ea:::10000000002.000000af:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.972236644000001, 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24320008700000001, 2024-09-19T05:44:12.059 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 434 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971343+0000", 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.971344+0000", 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.598e-06 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985502+0000", 2024-09-19T05:44:12.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014158082000000001 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985526+0000", 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4085000000000001e-05 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985537+0000", 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1143e-05 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985630+0000", 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2417000000000006e-05 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985741+0000", 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011084 2024-09-19T05:44:12.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.985917+0000", 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017616499999999999 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.991540+0000", 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056230630000000002 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214462+0000", 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22292171499999999 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214510+0000", 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8154e-05 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.214543+0000", 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2824999999999998e-05 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.065 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:472 3.31 3:8c3036ad:::10000000002.000000d5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.874750348999999, 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.154846594, 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 472 2024-09-19T05:44:12.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068829+0000", 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068831+0000", 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0140000000000001e-06 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078582+0000", 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097511360000000005 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078597+0000", 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5194000000000001e-05 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078605+0000", 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0860000000000002e-06 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078689+0000", 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4184999999999994e-05 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078786+0000", 2024-09-19T05:44:12.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6094000000000005e-05 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.078912+0000", 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000126749 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.083288+0000", 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004375421 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223574+0000", 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.140286203 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223628+0000", 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4401000000000003e-05 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.223675+0000", 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7110999999999997e-05 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.071 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:496 3.1e 3:7b5946a1:::10000000002.000000ed:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.849802585000001, 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15292847600000001, 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.072 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 496 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093777+0000", 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.093778+0000", 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.319e-06 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099527+0000", 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057487959999999996 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099543+0000", 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6767000000000001e-05 2024-09-19T05:44:12.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099552+0000", 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0919999999999998e-06 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099633+0000", 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1337999999999999e-05 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099734+0000", 2024-09-19T05:44:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101005 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.099877+0000", 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000143067 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.103900+0000", 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040226250000000002 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246621+0000", 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.142720927 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246660+0000", 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9567000000000003e-05 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.246705+0000", 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4972999999999998e-05 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.077 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:500 3.29 3:96fc3913:::10000000002.000000f1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.842970303, 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.153671691, 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 500 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100609+0000", 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.100612+0000", 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.055e-06 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108795+0000", 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081828820000000007 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108807+0000", 2024-09-19T05:44:12.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2468e-05 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108815+0000", 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3690000000000001e-06 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108887+0000", 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2509000000000007e-05 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.108964+0000", 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6787999999999997e-05 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.109095+0000", 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013103499999999999 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.112337+0000", 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0032419950000000001 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254185+0000", 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.141847683 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254240+0000", 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5702999999999998e-05 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.254281+0000", 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0204000000000003e-05 2024-09-19T05:44:12.083 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.084 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.084 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.084 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:12.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.030+0000 7fda6932c640 1 Processor -- start 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.030+0000 7fda6932c640 1 -- start start 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda6932c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 0x7fda640c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda6932c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda640c28f0 con 0x7fda64067950 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda6932c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda64067fb0 con 0x7fda64154ca0 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda6932c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fda640cc850 con 0x7fda64155080 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda637fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fda64067950 0x7fda64165320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:59832/0 (socket says 172.21.15.2:59832) 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda637fe640 1 -- 172.21.15.2:0/3069073162 learned_addr learned my addr 172.21.15.2:0/3069073162 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda627fc640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 0x7fda640c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270723057 0 0) 0x7fda640c28f0 con 0x7fda64067950 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda4c003520 con 0x7fda64067950 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1446192736 0 0) 0x7fda4c003520 con 0x7fda64067950 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 msgr2=0x7fda640c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 0x7fda640c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 >> v1:172.21.15.123:6789/0 conn(0x7fda64154ca0 legacy=0x7fda640c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda640c2b60 con 0x7fda64067950 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7fda5c0019f0 con 0x7fda64067950 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 >> v1:172.21.15.2:6789/0 conn(0x7fda64067950 legacy=0x7fda64165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda4c0039e0 0x7fda4c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda4c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 0x7fda4c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fda640cc850 con 0x7fda4c004390 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c00b9f0 con 0x7fda4c0039e0 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.031+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fda4c00bbf0 con 0x7fda64155080 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda62ffd640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda4c0039e0 0x7fda4c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda627fc640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda4c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda637fe640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 0x7fda4c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda627fc640 1 -- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 msgr2=0x7fda4c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda627fc640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda64155080 0x7fda4c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda627fc640 1 -- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda4c0039e0 msgr2=0x7fda4c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda627fc640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda4c0039e0 0x7fda4c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda627fc640 1 -- 172.21.15.2:0/3069073162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c003520 con 0x7fda4c004390 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.032+0000 7fda62ffd640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda4c0039e0 0x7fda4c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda627fc640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda4c004860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fda58009ac0 tx=0x7fda4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce0b93617c042c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58036070 con 0x7fda4c004390 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda58031400 con 0x7fda4c004390 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda61ffb640 1 -- 172.21.15.2:0/3069073162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58009690 con 0x7fda4c004390 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda6932c640 1 -- 172.21.15.2:0/3069073162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 msgr2=0x7fda4c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda6932c640 1 --2- 172.21.15.2:0/3069073162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda4c004860 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fda58009ac0 tx=0x7fda4c005830 comp rx=0 tx=0).stop 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda637fe640 1 -- 172.21.15.2:0/3069073162 reap_dead start 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda6932c640 1 -- 172.21.15.2:0/3069073162 shutdown_connections 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda6932c640 1 -- 172.21.15.2:0/3069073162 >> 172.21.15.2:0/3069073162 conn(0x7fda6406fcf0 msgr2=0x7fda64070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda6932c640 1 -- 172.21.15.2:0/3069073162 shutdown_connections 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.033+0000 7fda6932c640 1 -- 172.21.15.2:0/3069073162 wait complete. 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 Processor -- start 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 -- start start 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda640735a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda6414e380 0x7fda641467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda6414e7e0 0x7fda64146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fda6406bc10 con 0x7fda4c004390 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda640ccb30 con 0x7fda6414e380 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.035+0000 7fda6932c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fda6406bd90 con 0x7fda6414e7e0 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda640735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda640735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33134/0 (socket says 172.21.15.2:33134) 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 -- 172.21.15.2:0/742762071 learned_addr learned my addr 172.21.15.2:0/742762071 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda637fe640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda6414e7e0 0x7fda64146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda627fc640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda6414e380 0x7fda641467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 -- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda6414e7e0 msgr2=0x7fda64146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda6414e7e0 0x7fda64146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 -- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda6414e380 msgr2=0x7fda641467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda6414e380 0x7fda641467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 -- 172.21.15.2:0/742762071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6414ec80 con 0x7fda4c004390 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda62ffd640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda640735a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda5400fb60 tx=0x7fda5400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5400fe60 con 0x7fda4c004390 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda54018070 con 0x7fda4c004390 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6414f0c0 con 0x7fda4c004390 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5400de50 con 0x7fda4c004390 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.036+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6414f680 con 0x7fda4c004390 2024-09-19T05:44:12.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.037+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda640c3bf0 con 0x7fda4c004390 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fda540050d0 con 0x7fda4c004390 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda4b7fe640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda5006c2e0 0x7fda5006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7fda5408b5d0 con 0x7fda4c004390 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda4b7fe640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fda50074540 0x7fda50076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda627fc640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda5006c2e0 0x7fda5006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda54016e50 con 0x7fda50074540 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fda54092050 con 0x7fda4c004390 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.038+0000 7fda637fe640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fda50074540 0x7fda50076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.039+0000 7fda637fe640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fda50074540 0x7fda50076960 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.039+0000 7fda627fc640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda5006c2e0 0x7fda5006e7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda58033000 tx=0x7fda58008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.039+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda54016e50 con 0x7fda50074540 2024-09-19T05:44:12.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.055+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fda64152520 con 0x7fda50074540 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda4b7fe640 1 -- 172.21.15.2:0/742762071 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fda64152520 con 0x7fda50074540 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fda50074540 msgr2=0x7fda50076960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fda50074540 0x7fda50076960 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda5006c2e0 msgr2=0x7fda5006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda5006c2e0 0x7fda5006e7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fda58033000 tx=0x7fda58008030 comp rx=0 tx=0).stop 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 msgr2=0x7fda640735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 --2- 172.21.15.2:0/742762071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda4c004390 0x7fda640735a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fda5400fb60 tx=0x7fda5400d520 comp rx=0 tx=0).stop 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda637fe640 1 -- 172.21.15.2:0/742762071 reap_dead start 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 shutdown_connections 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 >> 172.21.15.2:0/742762071 conn(0x7fda6406fcf0 msgr2=0x7fda6406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 shutdown_connections 2024-09-19T05:44:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.056+0000 7fda6932c640 1 -- 172.21.15.2:0/742762071 wait complete. 2024-09-19T05:44:12.102 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-19T05:44:12.102 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:12.084+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:12.186 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:12.184+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:12.229 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:12.229 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:12.229 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:12.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.236+0000 7f6fd74a5640 1 Processor -- start 2024-09-19T05:44:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.236+0000 7f6fd74a5640 1 -- start start 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd74a5640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6fd015b2a0 0x7f6fd00718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd74a5640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd00cbc50 0x7f6fd0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd74a5640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd00ca910 con 0x7f6fd00cb870 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd74a5640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd006d930 con 0x7f6fd015b2a0 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd74a5640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd00caed0 con 0x7f6fd00cbc50 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd4a40640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6fd015b2a0 0x7f6fd00718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd4a40640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6fd015b2a0 0x7f6fd00718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33136/0 (socket says 172.21.15.2:33136) 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd4a40640 1 -- 172.21.15.2:0/650238408 learned_addr learned my addr 172.21.15.2:0/650238408 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.237+0000 7f6fd5a42640 1 --2- 172.21.15.2:0/650238408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd00cbc50 0x7f6fd0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fd4a40640 1 -- 172.21.15.2:0/650238408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd00cbc50 msgr2=0x7f6fd0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fd4a40640 1 --2- 172.21.15.2:0/650238408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd00cbc50 0x7f6fd0071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fd4a40640 1 -- 172.21.15.2:0/650238408 >> v1:172.21.15.123:6789/0 conn(0x7f6fd00cb870 legacy=0x7f6fd015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fd4a40640 1 -- 172.21.15.2:0/650238408 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd00cabf0 con 0x7f6fd015b2a0 2024-09-19T05:44:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fd4a40640 1 --2- 172.21.15.2:0/650238408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd015b2a0 0x7f6fd00718a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009870 tx=0x7f6fc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d630eb62f2c113d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fcffff640 1 -- 172.21.15.2:0/650238408 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4037070 con 0x7f6fd015b2a0 2024-09-19T05:44:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fcffff640 1 -- 172.21.15.2:0/650238408 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc4032440 con 0x7f6fd015b2a0 2024-09-19T05:44:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.238+0000 7f6fcffff640 1 -- 172.21.15.2:0/650238408 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4032810 con 0x7f6fd015b2a0 2024-09-19T05:44:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 -- 172.21.15.2:0/650238408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd015b2a0 msgr2=0x7f6fd00718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 --2- 172.21.15.2:0/650238408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd015b2a0 0x7f6fd00718a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009870 tx=0x7f6fc402a270 comp rx=0 tx=0).stop 2024-09-19T05:44:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 -- 172.21.15.2:0/650238408 shutdown_connections 2024-09-19T05:44:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 --2- 172.21.15.2:0/650238408 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd00cbc50 0x7f6fd0071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 --2- 172.21.15.2:0/650238408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd015b2a0 0x7f6fd00718a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 -- 172.21.15.2:0/650238408 >> 172.21.15.2:0/650238408 conn(0x7f6fd006bdf0 msgr2=0x7f6fd006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 -- 172.21.15.2:0/650238408 shutdown_connections 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.239+0000 7f6fd74a5640 1 -- 172.21.15.2:0/650238408 wait complete. 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.240+0000 7f6fd74a5640 1 Processor -- start 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.240+0000 7f6fd74a5640 1 -- start start 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd74a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd00cb870 0x7f6fd00c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd74a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd00cbc50 0x7f6fd01e5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd74a5640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd015b2a0 0x7f6fd01e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd74a5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd006d130 con 0x7f6fd00cb870 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd74a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd006d4a0 con 0x7f6fd00cbc50 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd74a5640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd00ca420 con 0x7f6fd015b2a0 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd00cb870 0x7f6fd00c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd00cb870 0x7f6fd00c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33140/0 (socket says 172.21.15.2:33140) 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 -- 172.21.15.2:0/2725618091 learned_addr learned my addr 172.21.15.2:0/2725618091 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5a42640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd015b2a0 0x7f6fd01e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd4a40640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd00cbc50 0x7f6fd01e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 -- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd015b2a0 msgr2=0x7f6fd01e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6fd015b2a0 0x7f6fd01e57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 -- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd00cbc50 msgr2=0x7f6fd01e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6fd00cbc50 0x7f6fd01e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.241+0000 7f6fd5241640 1 -- 172.21.15.2:0/2725618091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd00c8480 con 0x7f6fd00cb870 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.242+0000 7f6fd5241640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd00cb870 0x7f6fd00c7f40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c760 tx=0x7f6fc000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.242+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0016020 con 0x7f6fd00cb870 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.242+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc0006d50 con 0x7f6fd00cb870 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.242+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0003bb0 con 0x7f6fd00cb870 2024-09-19T05:44:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.242+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd01e4820 con 0x7f6fd00cb870 2024-09-19T05:44:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.242+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd01f1450 con 0x7f6fd00cb870 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.243+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6fd01f1690 con 0x7f6fd00cb870 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.243+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6fc000a850 con 0x7f6fd00cb870 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fcdffb640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fbc06c2b0 0x7f6fbc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f6fc008af90 con 0x7f6fd00cb870 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fd4a40640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fbc06c2b0 0x7f6fbc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fcdffb640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6fbc074510 0x7f6fbc076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fc000de50 con 0x7f6fbc074510 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f6fc008d5b0 con 0x7f6fd00cb870 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fd5a42640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6fbc074510 0x7f6fbc076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fd4a40640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fbc06c2b0 0x7f6fbc06e770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4034000 tx=0x7f6fc402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.244+0000 7f6fd5a42640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6fbc074510 0x7f6fbc076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.245+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6fc000de50 con 0x7f6fbc074510 2024-09-19T05:44:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.261+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f6fd0071d90 con 0x7f6fbc074510 2024-09-19T05:44:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.261+0000 7f6fcdffb640 1 -- 172.21.15.2:0/2725618091 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f6fd0071d90 con 0x7f6fbc074510 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6fbc074510 msgr2=0x7f6fbc076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6fbc074510 0x7f6fbc076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fbc06c2b0 msgr2=0x7f6fbc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6fbc06c2b0 0x7f6fbc06e770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4034000 tx=0x7f6fc402e000 comp rx=0 tx=0).stop 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd00cb870 msgr2=0x7f6fd00c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 --2- 172.21.15.2:0/2725618091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6fd00cb870 0x7f6fd00c7f40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fc000c760 tx=0x7f6fc000cc20 comp rx=0 tx=0).stop 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd5a42640 1 -- 172.21.15.2:0/2725618091 reap_dead start 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 shutdown_connections 2024-09-19T05:44:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 >> 172.21.15.2:0/2725618091 conn(0x7f6fd006bdf0 msgr2=0x7f6fd0154690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 shutdown_connections 2024-09-19T05:44:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.262+0000 7f6fd74a5640 1 -- 172.21.15.2:0/2725618091 wait complete. 2024-09-19T05:44:12.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-19T05:44:12.322 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:12.320+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:12.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.376+0000 7fca51096640 1 Processor -- start 2024-09-19T05:44:12.377 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.376+0000 7fca51096640 1 -- start start 2024-09-19T05:44:12.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.376+0000 7fca51096640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0c5610 0x7fca4c15b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.376+0000 7fca51096640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fca4c0d0c90 0x7fca4c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.376+0000 7fca51096640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4c0cc930 con 0x7fca4c0d08b0 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.376+0000 7fca51096640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fca4c0ccc10 con 0x7fca4c0d0c90 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.376+0000 7fca51096640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c0ccef0 con 0x7fca4c0c5610 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4b59e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fca4c0d0c90 0x7fca4c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4b59e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fca4c0d0c90 0x7fca4c0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33152/0 (socket says 172.21.15.2:33152) 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4b59e640 1 -- 172.21.15.2:0/3039849775 learned_addr learned my addr 172.21.15.2:0/3039849775 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4a59c640 1 --2- 172.21.15.2:0/3039849775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0c5610 0x7fca4c15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4b59e640 1 -- 172.21.15.2:0/3039849775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0c5610 msgr2=0x7fca4c15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.379 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4b59e640 1 --2- 172.21.15.2:0/3039849775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0c5610 0x7fca4c15b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4b59e640 1 -- 172.21.15.2:0/3039849775 >> v1:172.21.15.123:6789/0 conn(0x7fca4c0d08b0 legacy=0x7fca4c0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.377+0000 7fca4b59e640 1 -- 172.21.15.2:0/3039849775 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c0cd1d0 con 0x7fca4c0d0c90 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca4b59e640 1 --2- 172.21.15.2:0/3039849775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c0cb630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca44009e50 tx=0x7fca4402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d043ce46e62d68e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca49d9b640 1 -- 172.21.15.2:0/3039849775 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44036070 con 0x7fca4c0d0c90 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca49d9b640 1 -- 172.21.15.2:0/3039849775 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca44031600 con 0x7fca4c0d0c90 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca49d9b640 1 -- 172.21.15.2:0/3039849775 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca440319d0 con 0x7fca4c0d0c90 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 -- 172.21.15.2:0/3039849775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 msgr2=0x7fca4c0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 --2- 172.21.15.2:0/3039849775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c0cb630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca44009e50 tx=0x7fca4402a7a0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 -- 172.21.15.2:0/3039849775 shutdown_connections 2024-09-19T05:44:12.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 --2- 172.21.15.2:0/3039849775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c0cb630 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 --2- 172.21.15.2:0/3039849775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0c5610 0x7fca4c15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 -- 172.21.15.2:0/3039849775 >> 172.21.15.2:0/3039849775 conn(0x7fca4c06d940 msgr2=0x7fca4c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 -- 172.21.15.2:0/3039849775 shutdown_connections 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.378+0000 7fca51096640 1 -- 172.21.15.2:0/3039849775 wait complete. 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.380+0000 7fca51096640 1 Processor -- start 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.380+0000 7fca51096640 1 -- start start 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c0c5610 0x7fca4c1e1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0d08b0 0x7fca4c14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c1f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c154770 con 0x7fca4c0d0c90 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c0cbd80 con 0x7fca4c0c5610 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c1548f0 con 0x7fca4c0d08b0 2024-09-19T05:44:12.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c1f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c1f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33154/0 (socket says 172.21.15.2:33154) 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 -- 172.21.15.2:0/278619542 learned_addr learned my addr 172.21.15.2:0/278619542 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4a59c640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0d08b0 0x7fca4c14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4ad9d640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c0c5610 0x7fca4c1e1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 -- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0d08b0 msgr2=0x7fca4c14a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fca4c0d08b0 0x7fca4c14a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 -- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c0c5610 msgr2=0x7fca4c1e1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c0c5610 0x7fca4c1e1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 -- 172.21.15.2:0/278619542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c1f1480 con 0x7fca4c0d0c90 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca4b59e640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c1f3a40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca44009f70 tx=0x7fca44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44036050 con 0x7fca4c0d0c90 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca44042070 con 0x7fca4c0d0c90 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44009510 con 0x7fca4c0d0c90 2024-09-19T05:44:12.385 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c153fe0 con 0x7fca4c0d0c90 2024-09-19T05:44:12.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.381+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1f09a0 con 0x7fca4c0d0c90 2024-09-19T05:44:12.386 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:12.387 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:12.387 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:12.387 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:12.385+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:12.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.382+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca4c0cb630 con 0x7fca4c0d0c90 2024-09-19T05:44:12.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.383+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fca44007bf0 con 0x7fca4c0d0c90 2024-09-19T05:44:12.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.383+0000 7fca337fe640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fca3806c2b0 0x7fca3806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7fca440acaf0 con 0x7fca4c0d0c90 2024-09-19T05:44:12.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca337fe640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fca38074510 0x7fca38076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca4ad9d640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fca3806c2b0 0x7fca3806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca38077000 con 0x7fca38074510 2024-09-19T05:44:12.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fca440acf80 con 0x7fca4c0d0c90 2024-09-19T05:44:12.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca4a59c640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fca38074510 0x7fca38076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca4ad9d640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fca3806c2b0 0x7fca3806e770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca3c001780 tx=0x7fca3c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.384+0000 7fca4a59c640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fca38074510 0x7fca38076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.385+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca38077000 con 0x7fca38074510 2024-09-19T05:44:12.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:12.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.401+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fca4c14e930 con 0x7fca38074510 2024-09-19T05:44:12.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca337fe640 1 -- 172.21.15.2:0/278619542 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fca4c14e930 con 0x7fca38074510 2024-09-19T05:44:12.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fca38074510 msgr2=0x7fca38076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fca38074510 0x7fca38076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fca3806c2b0 msgr2=0x7fca3806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fca3806c2b0 0x7fca3806e770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca3c001780 tx=0x7fca3c009040 comp rx=0 tx=0).stop 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 msgr2=0x7fca4c1f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 --2- 172.21.15.2:0/278619542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca4c0d0c90 0x7fca4c1f3a40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca44009f70 tx=0x7fca44004a40 comp rx=0 tx=0).stop 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca4b59e640 1 -- 172.21.15.2:0/278619542 reap_dead start 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 shutdown_connections 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 >> 172.21.15.2:0/278619542 conn(0x7fca4c06d940 msgr2=0x7fca4c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 shutdown_connections 2024-09-19T05:44:12.404 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.402+0000 7fca51096640 1 -- 172.21.15.2:0/278619542 wait complete. 2024-09-19T05:44:12.414 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-19T05:44:12.487 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:12.486+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:12.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.520+0000 7f2594e37640 1 Processor -- start 2024-09-19T05:44:12.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.520+0000 7f2594e37640 1 -- start start 2024-09-19T05:44:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.521+0000 7f2594e37640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590154c20 0x7f25900cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.521+0000 7f2594e37640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2590155000 0x7f2590073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.521+0000 7f2594e37640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f25900caf40 0x7f2590073880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.521+0000 7f2594e37640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f25900ccbc0 con 0x7f2590155000 2024-09-19T05:44:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.521+0000 7f2594e37640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f25900cc600 con 0x7f25900caf40 2024-09-19T05:44:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.521+0000 7f2594e37640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f25900cc8e0 con 0x7f2590154c20 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.521+0000 7f258dd9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2590155000 0x7f2590073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258dd9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2590155000 0x7f2590073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33160/0 (socket says 172.21.15.2:33160) 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258dd9b640 1 -- 172.21.15.2:0/101399199 learned_addr learned my addr 172.21.15.2:0/101399199 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258ed9d640 1 --2- 172.21.15.2:0/101399199 >> v2:172.21.15.123:3300/0 conn(0x7f25900caf40 0x7f2590073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258e59c640 1 --2- 172.21.15.2:0/101399199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590154c20 0x7f25900cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258dd9b640 1 -- 172.21.15.2:0/101399199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590154c20 msgr2=0x7f25900cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258dd9b640 1 --2- 172.21.15.2:0/101399199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590154c20 0x7f25900cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258dd9b640 1 -- 172.21.15.2:0/101399199 >> v2:172.21.15.123:3300/0 conn(0x7f25900caf40 msgr2=0x7f2590073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258dd9b640 1 --2- 172.21.15.2:0/101399199 >> v2:172.21.15.123:3300/0 conn(0x7f25900caf40 0x7f2590073880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.522+0000 7f258dd9b640 1 -- 172.21.15.2:0/101399199 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25900cbc60 con 0x7f2590155000 2024-09-19T05:44:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f258dd9b640 1 --2- 172.21.15.2:0/101399199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2590155000 0x7f2590073340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f25900b61f0 tx=0x7f258402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88162228aa1d683 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f258d59a640 1 -- 172.21.15.2:0/101399199 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584004030 con 0x7f2590155000 2024-09-19T05:44:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f258d59a640 1 -- 172.21.15.2:0/101399199 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25840041d0 con 0x7f2590155000 2024-09-19T05:44:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f258d59a640 1 -- 172.21.15.2:0/101399199 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584003610 con 0x7f2590155000 2024-09-19T05:44:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 -- 172.21.15.2:0/101399199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2590155000 msgr2=0x7f2590073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 --2- 172.21.15.2:0/101399199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2590155000 0x7f2590073340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f25900b61f0 tx=0x7f258402a1c0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 -- 172.21.15.2:0/101399199 shutdown_connections 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 --2- 172.21.15.2:0/101399199 >> v2:172.21.15.123:3300/0 conn(0x7f25900caf40 0x7f2590073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 --2- 172.21.15.2:0/101399199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2590155000 0x7f2590073340 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 --2- 172.21.15.2:0/101399199 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590154c20 0x7f25900cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 -- 172.21.15.2:0/101399199 >> 172.21.15.2:0/101399199 conn(0x7f259006d0e0 msgr2=0x7f259006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 -- 172.21.15.2:0/101399199 shutdown_connections 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.523+0000 7f2594e37640 1 -- 172.21.15.2:0/101399199 wait complete. 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 Processor -- start 2024-09-19T05:44:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 -- start start 2024-09-19T05:44:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25900caf40 0x7f25900b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2590154c20 0x7f25900b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590155000 0x7f25900b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f25900c5770 con 0x7f25900caf40 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25900cb430 con 0x7f2590154c20 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.525+0000 7f2594e37640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f25900c58f0 con 0x7f2590155000 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25900caf40 0x7f25900b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25900caf40 0x7f25900b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33176/0 (socket says 172.21.15.2:33176) 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 -- 172.21.15.2:0/2403151779 learned_addr learned my addr 172.21.15.2:0/2403151779 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258dd9b640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2590154c20 0x7f25900b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258ed9d640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590155000 0x7f25900b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 -- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590155000 msgr2=0x7f25900b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590155000 0x7f25900b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 -- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2590154c20 msgr2=0x7f25900b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2590154c20 0x7f25900b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 -- 172.21.15.2:0/2403151779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25901e5780 con 0x7f25900caf40 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258ed9d640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2590155000 0x7f25900b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:44:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f258e59c640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25900caf40 0x7f25900b40b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f258000e8a0 tx=0x7f258000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580017070 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f258000c940 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25901e53e0 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258000cd30 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.526+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25901e4110 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.527+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25901e51f0 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.528+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f258001a070 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.528+0000 7f257effd640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f257006c2b0 0x7f257006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.528+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f258008b220 con 0x7f25900caf40 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.528+0000 7f258dd9b640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f257006c2b0 0x7f257006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.528+0000 7f257effd640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2570074510 0x7f2570076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.528+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2570077000 con 0x7f2570074510 2024-09-19T05:44:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.528+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f2580092050 con 0x7f25900caf40 2024-09-19T05:44:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.529+0000 7f258ed9d640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2570074510 0x7f2570076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.529+0000 7f258dd9b640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f257006c2b0 0x7f257006e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2584033000 tx=0x7f258402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.529+0000 7f258ed9d640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2570074510 0x7f2570076930 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.530+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2570077000 con 0x7f2570074510 2024-09-19T05:44:12.536 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:12.536 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3 3.12 3:4bec3886:::10000000002.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.253101626999999, 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57240116200000002, 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.537 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:12.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280487+0000", 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.751e-06 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287574+0000", 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070870530000000003 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287592+0000", 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8241e-05 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287601+0000", 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2800000000000003e-06 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287664+0000", 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3474999999999998e-05 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287758+0000", 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3577000000000002e-05 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287882+0000", 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012487199999999999 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.292606+0000", 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047238860000000001 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852756+0000", 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56015009400000004 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852823+0000", 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6138999999999998e-05 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852886+0000", 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3794000000000006e-05 2024-09-19T05:44:12.542 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:27 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.230567628999999, 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55004442399999998, 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.543 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 27 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303023+0000", 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4340000000000001e-06 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316366+0000", 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013343634 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316392+0000", 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6120999999999999e-05 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316402+0000", 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0066e-05 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316514+0000", 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111859 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316686+0000", 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017135699999999999 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316868+0000", 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018244100000000001 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.324019+0000", 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071506060000000003 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852956+0000", 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52893702899999995 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853018+0000", 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2178e-05 2024-09-19T05:44:12.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853064+0000", 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5698999999999997e-05 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:46 3.12 3:489b4446:::10000000002.0000002b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.201991762999999, 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54589824600000003, 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 46 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331598+0000", 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5689999999999999e-06 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347173+0000", 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015575235999999999 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347192+0000", 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9113000000000001e-05 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347202+0000", 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0151e-05 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347265+0000", 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3163000000000005e-05 2024-09-19T05:44:12.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347408+0000", 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014307299999999999 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347532+0000", 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012328400000000001 2024-09-19T05:44:12.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.351950+0000", 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044183620000000003 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877426+0000", 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52547596500000004 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877461+0000", 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877493+0000", 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2020000000000002e-05 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:51 3.12 3:499536aa:::10000000002.00000030:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.170242646999998, 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53664442800000001, 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 51 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363346+0000", 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9709999999999998e-06 2024-09-19T05:44:12.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383174+0000", 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019827860999999999 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383192+0000", 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383200+0000", 2024-09-19T05:44:12.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9589999999999995e-06 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383263+0000", 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2502999999999994e-05 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383391+0000", 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012766399999999999 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383510+0000", 2024-09-19T05:44:12.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118998 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.386613+0000", 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031035260000000001 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899913+0000", 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.51330011600000003 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899958+0000", 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4487000000000003e-05 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899989+0000", 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0886999999999998e-05 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.561 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:157 3.12 3:4b703321:::10000000002.0000009a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.97094916, 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34817550400000002, 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 157 2024-09-19T05:44:12.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562641+0000", 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576231+0000", 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013590286 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576263+0000", 2024-09-19T05:44:12.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1875999999999998e-05 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576270+0000", 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4840000000000001e-06 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576349+0000", 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.975e-05 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576547+0000", 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000198154 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576738+0000", 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019091400000000001 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.581161+0000", 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044224930000000004 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910746+0000", 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32958495700000001 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910782+0000", 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6636e-05 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910813+0000", 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0675e-05 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.567 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:158 3.1 3:82785032:::10000000002.0000009b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.955052027000001, 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.11650379299999999, 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.568 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 158 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578538+0000", 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4939999999999999e-06 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597197+0000", 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018658534000000001 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597218+0000", 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1296000000000001e-05 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597224+0000", 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4020000000000001e-06 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.571 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597274+0000", 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0556000000000002e-05 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597387+0000", 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113119 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597484+0000", 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6346000000000002e-05 2024-09-19T05:44:12.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.600132+0000", 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026487059999999998 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.694929+0000", 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.094797131000000007 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.694990+0000", 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0287000000000001e-05 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.695039+0000", 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8921999999999997e-05 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:184 3.12 3:4bdf515e:::10000000002.000000b5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:12.574 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.912458683000001, 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29608825300000002, 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 184 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621130+0000", 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0789999999999999e-06 2024-09-19T05:44:12.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632344+0000", 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01121422 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632369+0000", 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4712999999999999e-05 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632380+0000", 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1037e-05 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632436+0000", 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5887999999999998e-05 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632544+0000", 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107601 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632641+0000", 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7460000000000005e-05 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.636592+0000", 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039505950000000003 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917116+0000", 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.280524098 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917174+0000", 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7720999999999998e-05 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917216+0000", 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2840999999999999e-05 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:200 3.24 3:2525a577:::10000000002.000000c5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.881992603, 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.176819009, 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.581 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 200 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651597+0000", 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661419+0000", 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009822239 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661445+0000", 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5922999999999999e-05 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661463+0000", 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8501000000000001e-05 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661572+0000", 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108471 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661690+0000", 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118387 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661822+0000", 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013183800000000001 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.666069+0000", 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042470839999999999 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828327+0000", 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16225789600000001 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828368+0000", 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1402000000000001e-05 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828413+0000", 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4712999999999997e-05 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:252 3.24 3:25920220:::10000000002.000000f9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.828397364000001, 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12927656200000001, 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.587 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 252 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705192+0000", 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5849999999999998e-06 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720126+0000", 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01493366 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720143+0000", 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7385e-05 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720154+0000", 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0601000000000001e-05 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720232+0000", 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8584999999999994e-05 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720420+0000", 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018729799999999999 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720631+0000", 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021127400000000001 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.728280+0000", 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076494989999999997 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834381+0000", 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.106101029 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834423+0000", 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1634999999999997e-05 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834466+0000", 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3010999999999999e-05 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:283 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.919623941999999, 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57880034499999999, 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.593 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 283 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613965+0000", 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8759999999999999e-06 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633465+0000", 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019500483999999998 2024-09-19T05:44:12.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633485+0000", 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.984e-05 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633494+0000", 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5520000000000005e-06 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633589+0000", 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5144999999999998e-05 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633713+0000", 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012382300000000001 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633835+0000", 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122705 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.640960+0000", 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071243599999999997 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192691+0000", 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55173159599999999 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192737+0000", 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5487e-05 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192763+0000", 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6477000000000001e-05 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:294 3.24 3:2742a876:::10000000002.00000023:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.900053788999999, 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12659410500000001, 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.599 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 294 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633536+0000", 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.649e-06 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651603+0000", 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018067155000000001 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651628+0000", 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4950999999999999e-05 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651639+0000", 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-05 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651756+0000", 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117209 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651847+0000", 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1372000000000005e-05 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.652064+0000", 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021635099999999999 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.657902+0000", 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058383100000000002 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.759968+0000", 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.102066002 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760054+0000", 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5728000000000003e-05 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760127+0000", 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3248000000000001e-05 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:302 3.12 3:489b4446:::10000000002.0000002b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.881898982999999, 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54116458000000001, 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.605 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 302 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651691+0000", 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1810000000000001e-06 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664738+0000", 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013046540000000001 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664754+0000", 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6019999999999999e-05 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664767+0000", 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3315999999999999e-05 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664845+0000", 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8472000000000004e-05 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664945+0000", 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9804999999999997e-05 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.665089+0000", 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014373700000000001 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.669251+0000", 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041617379999999999 2024-09-19T05:44:12.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192796+0000", 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52354539899999997 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192829+0000", 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2668e-05 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192852+0000", 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3703999999999999e-05 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:307 3.12 3:499536aa:::10000000002.00000030:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.847802482000001, 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.50714811299999996, 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.611 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 307 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685788+0000", 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2480000000000001e-06 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698429+0000", 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012641172000000001 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698451+0000", 2024-09-19T05:44:12.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2172e-05 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698462+0000", 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1141000000000001e-05 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698591+0000", 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012867999999999999 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698704+0000", 2024-09-19T05:44:12.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113409 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698834+0000", 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012981599999999999 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.703334+0000", 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045004600000000004 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192881+0000", 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48954666899999999 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192908+0000", 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6988e-05 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192932+0000", 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4358e-05 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.616 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:397 3.24 3:273fbfa7:::10000000002.0000008a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.723481734, 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24366095600000001, 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.617 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 397 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810107+0000", 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824232+0000", 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014124503 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824247+0000", 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4834000000000001e-05 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824259+0000", 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1769e-05 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824368+0000", 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109722 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824500+0000", 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013183699999999999 2024-09-19T05:44:12.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824766+0000", 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000266016 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.830112+0000", 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053457620000000004 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053694+0000", 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22358192099999999 2024-09-19T05:44:12.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053736+0000", 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2534999999999998e-05 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053766+0000", 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9643999999999999e-05 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:413 3.12 3:4b703321:::10000000002.0000009a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.695844087999999, 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.36390152399999998, 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 413 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837745+0000", 2024-09-19T05:44:12.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.04e-06 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851397+0000", 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013652687 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851412+0000", 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4776999999999999e-05 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851420+0000", 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.43e-06 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851493+0000", 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3583000000000003e-05 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851600+0000", 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010678 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851717+0000", 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117237 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.855412+0000", 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036952439999999999 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201584+0000", 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34617155799999999 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201618+0000", 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3494999999999997e-05 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201644+0000", 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6693e-05 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:422 3.1 3:8350567c:::10000000002.000000a3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.667503945, 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15887042900000001, 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 422 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866086+0000", 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:12.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880367+0000", 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014281046 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880392+0000", 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5054000000000001e-05 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880406+0000", 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4401e-05 2024-09-19T05:44:12.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880532+0000", 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012579600000000001 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880667+0000", 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000135146 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880827+0000", 2024-09-19T05:44:12.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015946299999999999 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.886956+0000", 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061295730000000001 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024837+0000", 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.13788043999999999 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024890+0000", 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3483000000000003e-05 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024953+0000", 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3097999999999998e-05 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:440 3.12 3:4bdf515e:::10000000002.000000b5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:12.634 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.641421166000001, 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.31533669800000003, 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 440 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892170+0000", 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6179999999999997e-06 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900748+0000", 2024-09-19T05:44:12.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085778480000000008 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900763+0000", 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5231999999999999e-05 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900773+0000", 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7240000000000004e-06 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900829+0000", 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5562000000000001e-05 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900902+0000", 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3313999999999996e-05 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.901041+0000", 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000139419 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.903913+0000", 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028715619999999998 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207434+0000", 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30352098799999999 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207468+0000", 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3861e-05 2024-09-19T05:44:12.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207502+0000", 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000003e-05 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:456 3.24 3:2525a577:::10000000002.000000c5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.608384941000001, 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26312323300000001, 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.640 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 456 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925204+0000", 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1110000000000002e-06 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938061+0000", 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012856615 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938083+0000", 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2543000000000001e-05 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938093+0000", 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0084e-05 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938178+0000", 2024-09-19T05:44:12.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4827000000000006e-05 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938280+0000", 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010234900000000001 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938437+0000", 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015626099999999999 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942534+0000", 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004097458 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188244+0000", 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24570961099999999 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188281+0000", 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7134000000000001e-05 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188325+0000", 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4240000000000003e-05 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:487 3.33 3:ce8011cb:::10000000002.000000e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.585629644000001, 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22306556599999999, 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.646 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 487 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:12.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947959+0000", 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1519999999999999e-06 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962107+0000", 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014147649 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962127+0000", 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9868999999999999e-05 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962137+0000", 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0407e-05 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962206+0000", 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9122000000000007e-05 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962312+0000", 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105982 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962445+0000", 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000133108 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966463+0000", 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004017779 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170931+0000", 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.20446781999999999 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170977+0000", 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6245000000000002e-05 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.171023+0000", 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5432999999999997e-05 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:508 3.24 3:25920220:::10000000002.000000f9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 13.560445837, 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21526778799999999, 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:12.652 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 508 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:12.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973144+0000", 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.542e-06 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983914+0000", 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010770711000000001 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983945+0000", 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0648999999999997e-05 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983955+0000", 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8269999999999993e-06 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984017+0000", 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2652999999999998e-05 2024-09-19T05:44:12.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984100+0000", 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3094000000000001e-05 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984208+0000", 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107768 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.988635+0000", 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044265620000000002 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188359+0000", 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19972409799999999 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188389+0000", 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9882999999999998e-05 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:12.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188409+0000", 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0001e-05 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:12.658 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:12.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:12.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.546+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f25900cd9e0 con 0x7f2570074510 2024-09-19T05:44:12.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.546+0000 7f257effd640 1 -- 172.21.15.2:0/2403151779 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f25900cd9e0 con 0x7f2570074510 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2570074510 msgr2=0x7f2570076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f2570074510 0x7f2570076930 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f257006c2b0 msgr2=0x7f257006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f257006c2b0 0x7f257006e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2584033000 tx=0x7f258402e000 comp rx=0 tx=0).stop 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25900caf40 msgr2=0x7f25900b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 --2- 172.21.15.2:0/2403151779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f25900caf40 0x7f25900b40b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f258000e8a0 tx=0x7f258000ed60 comp rx=0 tx=0).stop 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f258ed9d640 1 -- 172.21.15.2:0/2403151779 reap_dead start 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 shutdown_connections 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 >> 172.21.15.2:0/2403151779 conn(0x7f259006d0e0 msgr2=0x7f2590073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 shutdown_connections 2024-09-19T05:44:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.547+0000 7f2594e37640 1 -- 172.21.15.2:0/2403151779 wait complete. 2024-09-19T05:44:12.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-19T05:44:12.663 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:12.658+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:12.720 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:44:12.767 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:12.768 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:12.768 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:12.768 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:12.777 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:12.790 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:12.789+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.806+0000 7f42979f1640 1 Processor -- start 2024-09-19T05:44:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.806+0000 7f42979f1640 1 -- start start 2024-09-19T05:44:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f42979f1640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4290067d10 0x7f42900b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f42979f1640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f429015cb90 0x7f42900c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f42979f1640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42900b4930 con 0x7f42900680f0 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f42979f1640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f42900b4650 con 0x7f4290067d10 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f42979f1640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f42900c90c0 con 0x7f429015cb90 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f4294f8c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f42900680f0 0x7f429015c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:59840/0 (socket says 172.21.15.2:59840) 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f4294f8c640 1 -- 172.21.15.2:0/2022498474 learned_addr learned my addr 172.21.15.2:0/2022498474 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f429578d640 1 --2- 172.21.15.2:0/2022498474 >> v2:172.21.15.2:3300/0 conn(0x7f4290067d10 0x7f42900b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f4295f8e640 1 --2- 172.21.15.2:0/2022498474 >> v2:172.21.15.123:3300/0 conn(0x7f429015cb90 0x7f42900c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f4287fff640 1 -- 172.21.15.2:0/2022498474 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1348394558 0 0) 0x7f42900b4930 con 0x7f42900680f0 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f4287fff640 1 -- 172.21.15.2:0/2022498474 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f427c003520 con 0x7f42900680f0 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f429578d640 1 -- 172.21.15.2:0/2022498474 >> v2:172.21.15.123:3300/0 conn(0x7f429015cb90 msgr2=0x7f42900c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f429578d640 1 --2- 172.21.15.2:0/2022498474 >> v2:172.21.15.123:3300/0 conn(0x7f429015cb90 0x7f42900c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f429578d640 1 -- 172.21.15.2:0/2022498474 >> v1:172.21.15.2:6789/0 conn(0x7f42900680f0 legacy=0x7f429015c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.807+0000 7f429578d640 1 -- 172.21.15.2:0/2022498474 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42900c93a0 con 0x7f4290067d10 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f429578d640 1 --2- 172.21.15.2:0/2022498474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42900b3a90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4290068610 tx=0x7f428c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f8f657d807fa97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f4287fff640 1 -- 172.21.15.2:0/2022498474 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c037070 con 0x7f4290067d10 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f4287fff640 1 -- 172.21.15.2:0/2022498474 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f428c032440 con 0x7f4290067d10 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f4287fff640 1 -- 172.21.15.2:0/2022498474 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c032810 con 0x7f4290067d10 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f42979f1640 1 -- 172.21.15.2:0/2022498474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 msgr2=0x7f42900b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f42979f1640 1 --2- 172.21.15.2:0/2022498474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42900b3a90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4290068610 tx=0x7f428c02a1a0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f42979f1640 1 -- 172.21.15.2:0/2022498474 shutdown_connections 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f42979f1640 1 --2- 172.21.15.2:0/2022498474 >> v2:172.21.15.123:3300/0 conn(0x7f429015cb90 0x7f42900c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f42979f1640 1 --2- 172.21.15.2:0/2022498474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42900b3a90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.809+0000 7f42979f1640 1 -- 172.21.15.2:0/2022498474 >> 172.21.15.2:0/2022498474 conn(0x7f4290070fc0 msgr2=0x7f429006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.810+0000 7f42979f1640 1 -- 172.21.15.2:0/2022498474 shutdown_connections 2024-09-19T05:44:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.810+0000 7f42979f1640 1 -- 172.21.15.2:0/2022498474 wait complete. 2024-09-19T05:44:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 Processor -- start 2024-09-19T05:44:12.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 -- start start 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42901f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42900680f0 0x7f42901f0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f429015cb90 0x7f429014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f42900b41e0 con 0x7f4290067d10 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42900b4060 con 0x7f42900680f0 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.811+0000 7f42979f1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f42900b4360 con 0x7f429015cb90 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42901f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42901f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33194/0 (socket says 172.21.15.2:33194) 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 -- 172.21.15.2:0/1233777443 learned_addr learned my addr 172.21.15.2:0/1233777443 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f4294f8c640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42900680f0 0x7f42901f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f4295f8e640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f429015cb90 0x7f429014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 -- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f429015cb90 msgr2=0x7f429014b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f429015cb90 0x7f429014b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 -- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42900680f0 msgr2=0x7f42901f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42900680f0 0x7f42901f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 -- 172.21.15.2:0/1233777443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4290157b40 con 0x7f4290067d10 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f429578d640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42901f4500 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f428c000c00 tx=0x7f428c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c041020 con 0x7f4290067d10 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f428c03d700 con 0x7f4290067d10 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42901f3a20 con 0x7f4290067d10 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.812+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c037040 con 0x7f4290067d10 2024-09-19T05:44:12.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.813+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42901f3fe0 con 0x7f4290067d10 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.813+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42900b3a90 con 0x7f4290067d10 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f428c044020 con 0x7f4290067d10 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4285ffb640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f427406c2b0 0x7f427406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f428c0aba60 con 0x7f4290067d10 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4285ffb640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4274074510 0x7f4274076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f428c0067f0 con 0x7f4274074510 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f428c0b02c0 con 0x7f4290067d10 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4294f8c640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f427406c2b0 0x7f427406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.815+0000 7f4295f8e640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4274074510 0x7f4274076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.816+0000 7f4294f8c640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f427406c2b0 0x7f427406e770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4290071af0 tx=0x7f4280009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.816+0000 7f4295f8e640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4274074510 0x7f4274076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.817+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f428c0067f0 con 0x7f4274074510 2024-09-19T05:44:12.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.833+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f4290148c50 con 0x7f4274074510 2024-09-19T05:44:12.834 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.833+0000 7f4285ffb640 1 -- 172.21.15.2:0/1233777443 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f4290148c50 con 0x7f4274074510 2024-09-19T05:44:12.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4274074510 msgr2=0x7f4274076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f4274074510 0x7f4274076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f427406c2b0 msgr2=0x7f427406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f427406c2b0 0x7f427406e770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4290071af0 tx=0x7f4280009040 comp rx=0 tx=0).stop 2024-09-19T05:44:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 msgr2=0x7f42901f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 --2- 172.21.15.2:0/1233777443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4290067d10 0x7f42901f4500 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f428c000c00 tx=0x7f428c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f4295f8e640 1 -- 172.21.15.2:0/1233777443 reap_dead start 2024-09-19T05:44:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 shutdown_connections 2024-09-19T05:44:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 >> 172.21.15.2:0/1233777443 conn(0x7f4290070fc0 msgr2=0x7f42900ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 shutdown_connections 2024-09-19T05:44:12.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.834+0000 7f42979f1640 1 -- 172.21.15.2:0/1233777443 wait complete. 2024-09-19T05:44:12.845 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-19T05:44:12.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.850+0000 7f29b6391640 1 Processor -- start 2024-09-19T05:44:12.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.850+0000 7f29b6391640 1 -- start start 2024-09-19T05:44:12.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29b6391640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f29b0154c20 0x7f29b00cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29b6391640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29b6391640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b00ccbc0 con 0x7f29b0155000 2024-09-19T05:44:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29b6391640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f29b00cc600 con 0x7f29b0154c20 2024-09-19T05:44:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29b6391640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f29b00cc8e0 con 0x7f29b00caf40 2024-09-19T05:44:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29af7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f29b0155000 0x7f29b0073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:59848/0 (socket says 172.21.15.2:59848) 2024-09-19T05:44:12.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29af7fe640 1 -- 172.21.15.2:0/1428121490 learned_addr learned my addr 172.21.15.2:0/1428121490 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29b492e640 1 --2- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.851+0000 7f29affff640 1 --2- 172.21.15.2:0/1428121490 >> v2:172.21.15.123:3300/0 conn(0x7f29b0154c20 0x7f29b00cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.852+0000 7f29aeffd640 1 -- 172.21.15.2:0/1428121490 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715321126 0 0) 0x7f29b00ccbc0 con 0x7f29b0155000 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.852+0000 7f29aeffd640 1 -- 172.21.15.2:0/1428121490 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2998003520 con 0x7f29b0155000 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.852+0000 7f29affff640 1 -- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 msgr2=0x7f29b015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.852+0000 7f29affff640 1 --2- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.852+0000 7f29affff640 1 -- 172.21.15.2:0/1428121490 >> v1:172.21.15.2:6789/0 conn(0x7f29b0155000 legacy=0x7f29b0073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.852+0000 7f29affff640 1 -- 172.21.15.2:0/1428121490 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b00cbc60 con 0x7f29b0154c20 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.852+0000 7f29b492e640 1 --2- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29affff640 1 --2- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0154c20 0x7f29b00cd9e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29a00097a0 tx=0x7f29a002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=646ea56526a02d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29aeffd640 1 -- 172.21.15.2:0/1428121490 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0037070 con 0x7f29b0154c20 2024-09-19T05:44:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29aeffd640 1 -- 172.21.15.2:0/1428121490 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a0032440 con 0x7f29b0154c20 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29aeffd640 1 -- 172.21.15.2:0/1428121490 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00327f0 con 0x7f29b0154c20 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29b6391640 1 -- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0154c20 msgr2=0x7f29b00cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29b6391640 1 --2- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0154c20 0x7f29b00cd9e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29a00097a0 tx=0x7f29a002a180 comp rx=0 tx=0).stop 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29b6391640 1 -- 172.21.15.2:0/1428121490 shutdown_connections 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29b6391640 1 --2- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29b6391640 1 --2- 172.21.15.2:0/1428121490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0154c20 0x7f29b00cd9e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.853+0000 7f29b6391640 1 -- 172.21.15.2:0/1428121490 >> 172.21.15.2:0/1428121490 conn(0x7f29b006d0e0 msgr2=0x7f29b006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.854+0000 7f29b6391640 1 -- 172.21.15.2:0/1428121490 shutdown_connections 2024-09-19T05:44:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.854+0000 7f29b6391640 1 -- 172.21.15.2:0/1428121490 wait complete. 2024-09-19T05:44:12.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.855+0000 7f29b6391640 1 Processor -- start 2024-09-19T05:44:12.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.855+0000 7f29b6391640 1 -- start start 2024-09-19T05:44:12.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29b6391640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b00b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29b6391640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29b0154c20 0x7f29b0156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29b6391640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0155000 0x7f29b0156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29b6391640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f29b0073a50 con 0x7f29b0154c20 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29b6391640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00cb430 con 0x7f29b0155000 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29b6391640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f29b0073bd0 con 0x7f29b00caf40 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29b0154c20 0x7f29b0156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29b0154c20 0x7f29b0156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33208/0 (socket says 172.21.15.2:33208) 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 -- 172.21.15.2:0/62379464 learned_addr learned my addr 172.21.15.2:0/62379464 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29affff640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b00b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29b492e640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0155000 0x7f29b0156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 -- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 msgr2=0x7f29b00b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b00b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 -- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0155000 msgr2=0x7f29b0156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0155000 0x7f29b0156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 -- 172.21.15.2:0/62379464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b01f0c90 con 0x7f29b0154c20 2024-09-19T05:44:12.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29affff640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b00b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:12.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29af7fe640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29b0154c20 0x7f29b0156700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c430 tx=0x7f29a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29acff9640 1 -- 172.21.15.2:0/62379464 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4017070 con 0x7f29b0154c20 2024-09-19T05:44:12.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.856+0000 7f29acff9640 1 -- 172.21.15.2:0/62379464 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a4007500 con 0x7f29b0154c20 2024-09-19T05:44:12.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.857+0000 7f29acff9640 1 -- 172.21.15.2:0/62379464 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a40059e0 con 0x7f29b0154c20 2024-09-19T05:44:12.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.857+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b01f0f70 con 0x7f29b0154c20 2024-09-19T05:44:12.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.857+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b01f1530 con 0x7f29b0154c20 2024-09-19T05:44:12.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.858+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b00c5770 con 0x7f29b0154c20 2024-09-19T05:44:12.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.858+0000 7f29acff9640 1 -- 172.21.15.2:0/62379464 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f29a401a070 con 0x7f29b0154c20 2024-09-19T05:44:12.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.859+0000 7f29acff9640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f299c06c2b0 0x7f299c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.859+0000 7f29acff9640 1 -- 172.21.15.2:0/62379464 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f29a408a7a0 con 0x7f29b0154c20 2024-09-19T05:44:12.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.859+0000 7f29affff640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f299c06c2b0 0x7f299c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.860+0000 7f29affff640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f299c06c2b0 0x7f299c06e770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f29a0034000 tx=0x7f29a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.862 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:12.859+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:12.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.863+0000 7f29acff9640 1 -- 172.21.15.2:0/62379464 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f29a4058800 con 0x7f29b0154c20 2024-09-19T05:44:12.881 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:12.881 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:12.881 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:12.881 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:12.881 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:12.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:12.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.954+0000 7f1bd390c640 1 Processor -- start 2024-09-19T05:44:12.955 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.954+0000 7f1bd390c640 1 -- start start 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd390c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1bcc155000 0x7f1bcc0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd390c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc0caf40 0x7f1bcc15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd390c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bcc0ccbc0 con 0x7f1bcc154c20 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd390c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1bcc0cc600 con 0x7f1bcc155000 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd390c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc0cc8e0 con 0x7f1bcc0caf40 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd1ea9640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc0caf40 0x7f1bcc15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd0ea7640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1bcc155000 0x7f1bcc0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd1ea9640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc0caf40 0x7f1bcc15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:47644/0 (socket says 172.21.15.2:47644) 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd16a8640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1bcc154c20 0x7f1bcc073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:59286/0 (socket says 172.21.15.2:59286) 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.955+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2228562182 learned_addr learned my addr 172.21.15.2:0/2228562182 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.956+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2228562182 >> v2:172.21.15.123:3300/0 conn(0x7f1bcc155000 msgr2=0x7f1bcc0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:12.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.956+0000 7f1bd1ea9640 1 --2- 172.21.15.2:0/2228562182 >> v2:172.21.15.123:3300/0 conn(0x7f1bcc155000 0x7f1bcc0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.956+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2228562182 >> v1:172.21.15.123:6789/0 conn(0x7f1bcc154c20 legacy=0x7f1bcc073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.956+0000 7f1bc3fff640 1 -- 172.21.15.2:0/2228562182 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2931017785 0 0) 0x7f1bcc0ccbc0 con 0x7f1bcc154c20 2024-09-19T05:44:12.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.956+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2228562182 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc0cbc60 con 0x7f1bcc0caf40 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.956+0000 7f1bd1ea9640 1 --2- 172.21.15.2:0/2228562182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc0caf40 0x7f1bcc15aae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e50 tx=0x7f1bc402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bde190094ffd010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bc3fff640 1 -- 172.21.15.2:0/2228562182 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4036070 con 0x7f1bcc0caf40 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bc3fff640 1 -- 172.21.15.2:0/2228562182 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc4031600 con 0x7f1bcc0caf40 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bc3fff640 1 -- 172.21.15.2:0/2228562182 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40319d0 con 0x7f1bcc0caf40 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 -- 172.21.15.2:0/2228562182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc0caf40 msgr2=0x7f1bcc15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 --2- 172.21.15.2:0/2228562182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc0caf40 0x7f1bcc15aae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e50 tx=0x7f1bc402a710 comp rx=0 tx=0).stop 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 -- 172.21.15.2:0/2228562182 shutdown_connections 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 --2- 172.21.15.2:0/2228562182 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc0caf40 0x7f1bcc15aae0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 --2- 172.21.15.2:0/2228562182 >> v2:172.21.15.123:3300/0 conn(0x7f1bcc155000 0x7f1bcc0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 -- 172.21.15.2:0/2228562182 >> 172.21.15.2:0/2228562182 conn(0x7f1bcc06d0e0 msgr2=0x7f1bcc06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 -- 172.21.15.2:0/2228562182 shutdown_connections 2024-09-19T05:44:12.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.957+0000 7f1bd390c640 1 -- 172.21.15.2:0/2228562182 wait complete. 2024-09-19T05:44:12.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.958+0000 7f1bd390c640 1 Processor -- start 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.958+0000 7f1bd390c640 1 -- start start 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd390c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bcc0caf40 0x7f1bcc152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd390c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc154c20 0x7f1bcc153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd390c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bcc155000 0x7f1bcc147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd390c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc073a50 con 0x7f1bcc155000 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd390c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc0cb430 con 0x7f1bcc0caf40 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd390c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc073bd0 con 0x7f1bcc154c20 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd1ea9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bcc155000 0x7f1bcc147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd0ea7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc154c20 0x7f1bcc153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd16a8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bcc0caf40 0x7f1bcc152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd0ea7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc154c20 0x7f1bcc153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:47648/0 (socket says 172.21.15.2:47648) 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.959+0000 7f1bd0ea7640 1 -- 172.21.15.2:0/2780906107 learned_addr learned my addr 172.21.15.2:0/2780906107 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:12.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc154c20 msgr2=0x7f1bcc153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd1ea9640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc154c20 0x7f1bcc153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bcc0caf40 msgr2=0x7f1bcc152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd1ea9640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bcc0caf40 0x7f1bcc152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2780906107 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc156c70 con 0x7f1bcc155000 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd0ea7640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1bcc154c20 0x7f1bcc153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd1ea9640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bcc155000 0x7f1bcc147120 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4030000 tx=0x7f1bc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4036050 con 0x7f1bcc155000 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc4042070 con 0x7f1bcc155000 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc14f670 con 0x7f1bcc155000 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4009510 con 0x7f1bcc155000 2024-09-19T05:44:12.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.960+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc156ef0 con 0x7f1bcc155000 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.961+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1bcc146cf0 con 0x7f1bcc155000 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.962+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1bc4007c10 con 0x7f1bcc155000 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.962+0000 7f1bc1ffb640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1bb006c2b0 0x7f1bb006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.962+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f1bc40acb50 con 0x7f1bcc155000 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.962+0000 7f1bd16a8640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1bb006c2b0 0x7f1bb006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.962+0000 7f1bc1ffb640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1bb0074510 0x7f1bb0076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.962+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bc4006b90 con 0x7f1bb0074510 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.962+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f1bc40acfe0 con 0x7f1bcc155000 2024-09-19T05:44:12.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.963+0000 7f1bd0ea7640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1bb0074510 0x7f1bb0076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:12.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.963+0000 7f1bd16a8640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1bb006c2b0 0x7f1bb006e770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000f60 tx=0x7f1bbc007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.963+0000 7f1bd0ea7640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1bb0074510 0x7f1bb0076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:12.966 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:12.963+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:12.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.964+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1bc4006b90 con 0x7f1bb0074510 2024-09-19T05:44:12.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.980+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f1bcc0c5770 con 0x7f1bb0074510 2024-09-19T05:44:12.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.980+0000 7f1bc1ffb640 1 -- 172.21.15.2:0/2780906107 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f1bcc0c5770 con 0x7f1bb0074510 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.980+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1bb0074510 msgr2=0x7f1bb0076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.980+0000 7f1bd390c640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f1bb0074510 0x7f1bb0076930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.980+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1bb006c2b0 msgr2=0x7f1bb006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd390c640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1bb006c2b0 0x7f1bb006e770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc000f60 tx=0x7f1bbc007d50 comp rx=0 tx=0).stop 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bcc155000 msgr2=0x7f1bcc147120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd390c640 1 --2- 172.21.15.2:0/2780906107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bcc155000 0x7f1bcc147120 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4030000 tx=0x7f1bc4004a40 comp rx=0 tx=0).stop 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd1ea9640 1 -- 172.21.15.2:0/2780906107 reap_dead start 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 shutdown_connections 2024-09-19T05:44:12.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 >> 172.21.15.2:0/2780906107 conn(0x7f1bcc06d0e0 msgr2=0x7f1bcc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:12.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 shutdown_connections 2024-09-19T05:44:12.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:12.981+0000 7f1bd390c640 1 -- 172.21.15.2:0/2780906107 wait complete. 2024-09-19T05:44:12.992 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-19T05:44:13.035 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:17 3.21 3:86f42aac:::10000000002.0000000e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.739060467000002, 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.072757916000000006, 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.036 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 17 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293556+0000", 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1830000000000001e-06 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320078+0000", 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.026521807000000001 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320090+0000", 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1873e-05 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320096+0000", 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.852e-06 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320193+0000", 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7145999999999995e-05 2024-09-19T05:44:13.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320321+0000", 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012788799999999999 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320445+0000", 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000124206 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.325801+0000", 2024-09-19T05:44:13.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053561700000000004 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366202+0000", 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.040400806999999997 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366258+0000", 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5971999999999999e-05 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366312+0000", 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4011999999999997e-05 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:20 3.21 3:85380590:::10000000002.00000011:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.712491144000001, 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.094367290000000006, 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.042 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 20 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320125+0000", 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5319999999999999e-06 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345237+0000", 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025112457000000001 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345260+0000", 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3184999999999999e-05 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345274+0000", 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3168e-05 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345351+0000", 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7600000000000002e-05 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345507+0000", 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015602300000000001 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345659+0000", 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015159599999999999 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.351961+0000", 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063018880000000003 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414334+0000", 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.062373799000000001 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414436+0000", 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101795 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414491+0000", 2024-09-19T05:44:13.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4246999999999997e-05 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:23 3.21 3:87fa6b7a:::10000000002.00000014:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.687307396000001, 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.106935525, 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.048 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:13.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345309+0000", 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369545+0000", 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024236034 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369565+0000", 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9928999999999998e-05 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369576+0000", 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0373e-05 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369677+0000", 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101765 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369863+0000", 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000185692 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370039+0000", 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017557 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.374766+0000", 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047269210000000002 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452093+0000", 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.077327075999999995 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452160+0000", 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7805999999999994e-05 2024-09-19T05:44:13.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452243+0000", 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2078999999999998e-05 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:39 3.2e 3:75eea580:::10000000002.00000024:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.662591840000001, 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41578167199999999, 2024-09-19T05:44:13.054 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 39 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370025+0000", 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393960+0000", 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023934536999999999 2024-09-19T05:44:13.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393976+0000", 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6441000000000001e-05 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393989+0000", 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2679e-05 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394066+0000", 2024-09-19T05:44:13.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6919e-05 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394208+0000", 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014247500000000001 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394304+0000", 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5369000000000001e-05 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.399842+0000", 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055380000000000004 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785727+0000", 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.38588529199999999 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785763+0000", 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6494e-05 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785804+0000", 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0729000000000003e-05 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:62 3.c e53/20 3:31d8a57c:::10000000002.0000003b:head v 53'1, mlcod=0'0)", 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.654930636, 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.027856545, 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.060 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:13.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0600000000000001e-07 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400389+0000", 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022704832000000001 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400402+0000", 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2893e-05 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400411+0000", 2024-09-19T05:44:13.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.037e-06 2024-09-19T05:44:13.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400473+0000", 2024-09-19T05:44:13.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:44:13.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400489+0000", 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6651e-05 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.405444+0000", 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049551279999999996 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.405540+0000", 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5969000000000002e-05 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:124 3.21 3:844ac05a:::10000000002.00000079:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.583172600000001, 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.060327543999999997, 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.065 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 124 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449444+0000", 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2239999999999998e-06 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464881+0000", 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015437222 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464904+0000", 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2274999999999999e-05 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464914+0000", 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0657e-05 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464999+0000", 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5031000000000006e-05 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.465166+0000", 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016689400000000001 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.465311+0000", 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145208 2024-09-19T05:44:13.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.470491+0000", 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051794229999999998 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509673+0000", 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.039182002 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509724+0000", 2024-09-19T05:44:13.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1205999999999997e-05 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509769+0000", 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5401999999999997e-05 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:138 3.2e 3:7768043d:::10000000002.00000087:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.567662680000002, 2024-09-19T05:44:13.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34341082099999998, 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 138 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464954+0000", 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482737+0000", 2024-09-19T05:44:13.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017782893000000001 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482757+0000", 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0352000000000001e-05 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482766+0000", 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0119999999999994e-06 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482841+0000", 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4386000000000002e-05 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482989+0000", 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148405 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.483128+0000", 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013918200000000001 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.486268+0000", 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031401229999999999 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808279+0000", 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32201085299999999 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808321+0000", 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1962000000000002e-05 2024-09-19T05:44:13.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808363+0000", 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1434999999999999e-05 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:257 3.21 3:86aaefcc:::10000000002.000000fe:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.533674986000001, 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.11301415099999999, 2024-09-19T05:44:13.077 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 257 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498941+0000", 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9350000000000001e-06 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521369+0000", 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022427589000000001 2024-09-19T05:44:13.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521386+0000", 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6986999999999998e-05 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521393+0000", 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8569999999999997e-06 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521454+0000", 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1571999999999999e-05 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521577+0000", 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122998 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521744+0000", 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016694100000000001 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.527047+0000", 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053028559999999999 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611743+0000", 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.084696294000000005 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611895+0000", 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015154900000000001 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611954+0000", 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8573000000000003e-05 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:273 3.21 3:86f42aac:::10000000002.0000000e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.429279738, 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.13998105, 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.083 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 273 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603338+0000", 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624529+0000", 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.021190186 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624551+0000", 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.213e-05 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624562+0000", 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1452e-05 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624630+0000", 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7738999999999996e-05 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624721+0000", 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1168999999999994e-05 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624854+0000", 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013313600000000001 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.630092+0000", 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052376819999999996 2024-09-19T05:44:13.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743176+0000", 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.113084107 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743260+0000", 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4309999999999997e-05 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743316+0000", 2024-09-19T05:44:13.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5448999999999997e-05 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:276 3.21 3:85380590:::10000000002.00000011:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.408022473000001, 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15545139099999999, 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 276 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:13.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624594+0000", 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0959999999999999e-06 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650750+0000", 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.026156266000000001 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650772+0000", 2024-09-19T05:44:13.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2027000000000001e-05 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650780+0000", 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650866+0000", 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5521999999999999e-05 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650959+0000", 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3524999999999996e-05 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651140+0000", 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018071799999999999 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.658731+0000", 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075914989999999998 2024-09-19T05:44:13.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.779803+0000", 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12107206399999999 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.779912+0000", 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108564 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780043+0000", 2024-09-19T05:44:13.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013128900000000001 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:279 3.21 3:87fa6b7a:::10000000002.00000014:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.381813992, 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17630584099999999, 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.095 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 279 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650803+0000", 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.232e-06 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684038+0000", 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.033235812000000003 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684068+0000", 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9118000000000001e-05 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684080+0000", 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2704999999999999e-05 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684157+0000", 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6879999999999996e-05 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684261+0000", 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010367 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684423+0000", 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016253799999999999 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.689113+0000", 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046892729999999999 2024-09-19T05:44:13.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.826966+0000", 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.137853054 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.827028+0000", 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2030000000000001e-05 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.827106+0000", 2024-09-19T05:44:13.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8529000000000007e-05 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:295 3.2e 3:75eea580:::10000000002.00000024:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.348504105, 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52131509399999998, 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.101 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 295 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684121+0000", 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0455000000000001e-05 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715121+0000", 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031000119 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715142+0000", 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0871999999999999e-05 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715152+0000", 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.064e-05 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715284+0000", 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013133 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715363+0000", 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9676000000000001e-05 2024-09-19T05:44:13.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715467+0000", 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000103816 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.722648+0000", 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071811460000000002 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205361+0000", 2024-09-19T05:44:13.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48271265600000002 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205396+0000", 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4638000000000003e-05 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205425+0000", 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9745999999999999e-05 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:318 3.c e54/20 3:31d8a57c:::10000000002.0000003b:head v 54'5, mlcod=53'3)", 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.320556366, 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035734850999999998, 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712059+0000", 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.471e-06 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742630+0000", 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.030570631000000001 2024-09-19T05:44:13.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742640+0000", 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0264e-05 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742648+0000", 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1890000000000001e-06 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742738+0000", 2024-09-19T05:44:13.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0172000000000003e-05 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742760+0000", 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2005e-05 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747734+0000", 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049739600000000004 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747793+0000", 2024-09-19T05:44:13.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9159000000000001e-05 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:321 3.b 3:d381a89f:::10000000002.0000003e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.317437125, 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.038149887, 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.111 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 321 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715179+0000", 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.08e-06 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746761+0000", 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031581559000000002 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746774+0000", 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2994e-05 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746798+0000", 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4443999999999998e-05 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746852+0000", 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3492000000000002e-05 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746956+0000", 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104455 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747084+0000", 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012780800000000001 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.752122+0000", 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050380809999999998 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753229+0000", 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011063399999999999 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753281+0000", 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2618999999999999e-05 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753327+0000", 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6014999999999999e-05 2024-09-19T05:44:13.116 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:324 3.b 3:d0493c82:::10000000002.00000041:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.285786343, 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035384118999999999, 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.117 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 324 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746830+0000", 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0839999999999999e-06 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774044+0000", 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.027214075000000001 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774064+0000", 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0020000000000001e-05 2024-09-19T05:44:13.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774074+0000", 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3789999999999995e-06 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774161+0000", 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7663000000000004e-05 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774271+0000", 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110229 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774421+0000", 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014956099999999999 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780042+0000", 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056212520000000002 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782100+0000", 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002057513 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782160+0000", 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0513999999999997e-05 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782212+0000", 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1829000000000001e-05 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:339 3.b 3:d24a1766:::10000000002.00000050:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.25851621, 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.032212151000000001, 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 339 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.123 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774100+0000", 2024-09-19T05:44:13.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.968e-06 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796613+0000", 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022512880999999998 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796670+0000", 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7011e-05 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796685+0000", 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5208000000000001e-05 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796884+0000", 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000198641 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.797116+0000", 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232368 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.797291+0000", 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017451899999999999 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.802626+0000", 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005335528 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806209+0000", 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035828069999999999 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806260+0000", 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0961999999999998e-05 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806310+0000", 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0257999999999999e-05 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.128 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:380 3.21 3:844ac05a:::10000000002.00000079:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.235865303000001, 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.063018490999999996, 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 380 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796752+0000", 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.078e-06 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816450+0000", 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019697531000000001 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816475+0000", 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5420000000000001e-05 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816505+0000", 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0199e-05 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816597+0000", 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1739999999999999e-05 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816725+0000", 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000128116 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816874+0000", 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148913 2024-09-19T05:44:13.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.821849+0000", 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004974493 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859654+0000", 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.037805041999999997 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859717+0000", 2024-09-19T05:44:13.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859768+0000", 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0772000000000001e-05 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:394 3.2e 3:7768043d:::10000000002.00000087:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.216056093000001, 2024-09-19T05:44:13.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.400247509, 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 394 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816560+0000", 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.745e-06 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833002+0000", 2024-09-19T05:44:13.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016442149999999999 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833022+0000", 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9446999999999999e-05 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833032+0000", 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0287e-05 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833104+0000", 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2365000000000003e-05 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833189+0000", 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4470999999999998e-05 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833328+0000", 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138913 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837647+0000", 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043193600000000004 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216711+0000", 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.379063539 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216765+0000", 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4302999999999998e-05 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216806+0000", 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0929000000000001e-05 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:402 3.b 3:d1738051:::10000000002.0000008f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.199556171999999, 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035654618999999999, 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.140 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 402 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833060+0000", 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7209999999999999e-06 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861464+0000", 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.028403690999999998 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861483+0000", 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8947999999999999e-05 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861494+0000", 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1168e-05 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861660+0000", 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016661000000000001 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861769+0000", 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108205 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861892+0000", 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012325699999999999 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868420+0000", 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065282420000000001 2024-09-19T05:44:13.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868597+0000", 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177195 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868658+0000", 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0844000000000002e-05 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868713+0000", 2024-09-19T05:44:13.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4737999999999997e-05 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:513 3.21 3:86aaefcc:::10000000002.000000fe:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 14.171088341000001, 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.104537124, 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:13.146 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 513 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861528+0000", 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.875e-06 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877329+0000", 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015801329999999999 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877342+0000", 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2311e-05 2024-09-19T05:44:13.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877347+0000", 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8220000000000003e-06 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877397+0000", 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9227999999999997e-05 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877488+0000", 2024-09-19T05:44:13.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1828000000000002e-05 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877601+0000", 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011255399999999999 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.882452+0000", 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048512929999999996 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.965958+0000", 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.083506094000000003 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966010+0000", 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1544e-05 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966063+0000", 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3245000000000003e-05 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.151 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:13.152 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:13.152 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:13.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.051+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f29b00b32c0 con 0x7f29b0154c20 2024-09-19T05:44:13.154 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:13.152+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:13.155 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.134+0000 7f4676b52640 1 Processor -- start 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.134+0000 7f4676b52640 1 -- start start 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f4676b52640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4670155000 0x7f46700cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f4676b52640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46700ccbc0 con 0x7f46700caf40 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f4676b52640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46700cc600 con 0x7f4670154c20 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f4676b52640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f46700cc8e0 con 0x7f4670155000 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f46750ef640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f46700caf40 0x7f467015aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:59852/0 (socket says 172.21.15.2:59852) 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f46750ef640 1 -- 172.21.15.2:0/1791917720 learned_addr learned my addr 172.21.15.2:0/1791917720 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466ffff640 1 --2- 172.21.15.2:0/1791917720 >> v2:172.21.15.123:3300/0 conn(0x7f4670155000 0x7f46700cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818990670 0 0) 0x7f46700ccbc0 con 0x7f46700caf40 2024-09-19T05:44:13.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4658003520 con 0x7f46700caf40 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3422732250 0 0) 0x7f46700cc600 con 0x7f4670154c20 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46700ccbc0 con 0x7f4670154c20 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 729171495 0 0) 0x7f4658003520 con 0x7f46700caf40 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 >> v2:172.21.15.123:3300/0 conn(0x7f4670155000 msgr2=0x7f46700cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 --2- 172.21.15.2:0/1791917720 >> v2:172.21.15.123:3300/0 conn(0x7f4670155000 0x7f46700cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 >> v1:172.21.15.123:6789/0 conn(0x7f4670154c20 legacy=0x7f4670073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46700cbc60 con 0x7f46700caf40 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f4668001670 con 0x7f46700caf40 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466ffff640 1 --2- 172.21.15.2:0/1791917720 >> v2:172.21.15.123:3300/0 conn(0x7f4670155000 0x7f46700cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 >> v1:172.21.15.2:6789/0 conn(0x7f46700caf40 legacy=0x7f467015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4658003b40 0x7f4658003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.135+0000 7f466f7fe640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4658004800 0x7f4658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466f7fe640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46580051c0 0x7f465800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46700cc8e0 con 0x7f4658004800 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f465800c040 con 0x7f46580051c0 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f465800c240 con 0x7f4658003b40 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466ffff640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4658004800 0x7f4658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f46750ef640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46580051c0 0x7f465800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f46748ee640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4658003b40 0x7f4658003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466ffff640 1 -- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4658003b40 msgr2=0x7f4658003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466ffff640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4658003b40 0x7f4658003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466ffff640 1 -- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46580051c0 msgr2=0x7f465800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466ffff640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46580051c0 0x7f465800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f466ffff640 1 -- 172.21.15.2:0/1791917720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4658003520 con 0x7f4658004800 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.136+0000 7f46750ef640 1 -- 172.21.15.2:0/1791917720 reap_dead start 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f466ffff640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4658004800 0x7f4658004c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f466400e320 tx=0x7f4658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e6d8311d5f42d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664009e30 con 0x7f4658004800 2024-09-19T05:44:13.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466402e620 con 0x7f4658004800 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f466f7fe640 1 -- 172.21.15.2:0/1791917720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466402e9d0 con 0x7f4658004800 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f4676b52640 1 -- 172.21.15.2:0/1791917720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4658004800 msgr2=0x7f4658004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f4676b52640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4658004800 0x7f4658004c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f466400e320 tx=0x7f4658005830 comp rx=0 tx=0).stop 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f4676b52640 1 -- 172.21.15.2:0/1791917720 shutdown_connections 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f4676b52640 1 --2- 172.21.15.2:0/1791917720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4658004800 0x7f4658004c80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f4676b52640 1 -- 172.21.15.2:0/1791917720 >> 172.21.15.2:0/1791917720 conn(0x7f467006d0e0 msgr2=0x7f467006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f4676b52640 1 -- 172.21.15.2:0/1791917720 shutdown_connections 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.137+0000 7f4676b52640 1 -- 172.21.15.2:0/1791917720 wait complete. 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.139+0000 7f4676b52640 1 Processor -- start 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.139+0000 7f4676b52640 1 -- start start 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.139+0000 7f4676b52640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4658004800 0x7f46701e4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.139+0000 7f4676b52640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46701555c0 0x7f46700b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f4676b52640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46700b5d30 0x7f4670148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f4676b52640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f46700c5770 con 0x7f46701555c0 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f4676b52640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46700cb430 con 0x7f4658004800 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f4676b52640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f46700c58f0 con 0x7f46700b5d30 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46701555c0 0x7f46700b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46701555c0 0x7f46700b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33234/0 (socket says 172.21.15.2:33234) 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 -- 172.21.15.2:0/3460747463 learned_addr learned my addr 172.21.15.2:0/3460747463 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f46750ef640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46700b5d30 0x7f4670148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f46748ee640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4658004800 0x7f46701e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 -- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46700b5d30 msgr2=0x7f4670148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46700b5d30 0x7f4670148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 -- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4658004800 msgr2=0x7f46701e4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4658004800 0x7f46701e4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 -- 172.21.15.2:0/3460747463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46701e0d20 con 0x7f46701555c0 2024-09-19T05:44:13.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f46750ef640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46700b5d30 0x7f4670148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f46748ee640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4658004800 0x7f46701e4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466ffff640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46701555c0 0x7f46700b52a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f466402b0a0 tx=0x7f4664009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664009e30 con 0x7f46701555c0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466402e420 con 0x7f46701555c0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664036c60 con 0x7f46701555c0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4670154340 con 0x7f46701555c0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.140+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701e5350 con 0x7f46701555c0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.141+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46701e5dd0 con 0x7f46701555c0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.142+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4664048020 con 0x7f46701555c0 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.142+0000 7f466d7fa640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f465c06c2b0 0x7f465c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.142+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5741+0+0 (secure 0 0 0) 0x7f46640ac3e0 con 0x7f46701555c0 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.142+0000 7f46748ee640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f465c06c2b0 0x7f465c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.143+0000 7f466d7fa640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f465c074510 0x7f465c076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.143+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f465c077000 con 0x7f465c074510 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.143+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f46640b3050 con 0x7f46701555c0 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.143+0000 7f46750ef640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f465c074510 0x7f465c076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.143+0000 7f46748ee640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f465c06c2b0 0x7f465c06e770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46600055e0 tx=0x7f4660009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.143+0000 7f46750ef640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f465c074510 0x7f465c076930 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.144+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f465c077000 con 0x7f465c074510 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f46701e5140 con 0x7f465c074510 2024-09-19T05:44:13.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f466d7fa640 1 -- 172.21.15.2:0/3460747463 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f46701e5140 con 0x7f465c074510 2024-09-19T05:44:13.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f465c074510 msgr2=0x7f465c076930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f465c074510 0x7f465c076930 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f465c06c2b0 msgr2=0x7f465c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f465c06c2b0 0x7f465c06e770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46600055e0 tx=0x7f4660009040 comp rx=0 tx=0).stop 2024-09-19T05:44:13.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46701555c0 msgr2=0x7f46700b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 --2- 172.21.15.2:0/3460747463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46701555c0 0x7f46700b52a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f466402b0a0 tx=0x7f4664009750 comp rx=0 tx=0).stop 2024-09-19T05:44:13.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f46750ef640 1 -- 172.21.15.2:0/3460747463 reap_dead start 2024-09-19T05:44:13.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 shutdown_connections 2024-09-19T05:44:13.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 >> 172.21.15.2:0/3460747463 conn(0x7f467006d0e0 msgr2=0x7f4670151070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:13.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.160+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 shutdown_connections 2024-09-19T05:44:13.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.161+0000 7f4676b52640 1 -- 172.21.15.2:0/3460747463 wait complete. 2024-09-19T05:44:13.254 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:13.253+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:13.268 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:13.268 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:13.268 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:13.268 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:13.276 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:13.355 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:13.354+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:13.423 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:13.423 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:13.423 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:13.423 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:13.423 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:13.432 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:13.456 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:13.455+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:13.541 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:13.541 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:12 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.253535703000001, 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.301330818, 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 12 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285078+0000", 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8979999999999998e-06 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288881+0000", 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038028900000000002 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288908+0000", 2024-09-19T05:44:13.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6831000000000001e-05 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288922+0000", 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4647999999999999e-05 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289034+0000", 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011129 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289158+0000", 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012398199999999999 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289283+0000", 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012596100000000001 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302442+0000", 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01315887 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586307+0000", 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28386498300000002 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586357+0000", 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9406000000000001e-05 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586405+0000", 2024-09-19T05:44:13.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8059000000000002e-05 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:16 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.239411962999998, 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28733763699999998, 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 16 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:13.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299338+0000", 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013993499999999999 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305742+0000", 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064047180000000002 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305795+0000", 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2988000000000002e-05 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305808+0000", 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.307e-05 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305883+0000", 2024-09-19T05:44:13.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4265999999999997e-05 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306078+0000", 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195281 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306298+0000", 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000219662 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.319662+0000", 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0133641 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586451+0000", 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26678922700000002 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586494+0000", 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3102999999999997e-05 2024-09-19T05:44:13.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586535+0000", 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1287e-05 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:36 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.222073576, 2024-09-19T05:44:13.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54259264799999996, 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316539+0000", 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1310000000000002e-06 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326313+0000", 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0097740320000000002 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326341+0000", 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8178e-05 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326354+0000", 2024-09-19T05:44:13.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2767e-05 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326454+0000", 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9741999999999995e-05 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326598+0000", 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000144454 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326740+0000", 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014189899999999999 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.381404+0000", 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054663549999999998 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859031+0000", 2024-09-19T05:44:13.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47762663900000002 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859074+0000", 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3578999999999997e-05 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859129+0000", 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4676999999999998e-05 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:40 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.212184229000002, 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29103886899999998, 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 40 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326428+0000", 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9160000000000001e-06 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343603+0000", 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017174545999999999 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343619+0000", 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6260999999999999e-05 2024-09-19T05:44:13.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343630+0000", 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0774999999999999e-05 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343729+0000", 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8694000000000001e-05 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343846+0000", 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117588 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343969+0000", 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012271600000000001 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.376370+0000", 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032401093999999998 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617348+0000", 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24097808100000001 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617405+0000", 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6577999999999999e-05 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617464+0000", 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9620000000000002e-05 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:101 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.071498526999999, 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.392178684, 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 101 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467114+0000", 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.269e-06 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.477820+0000", 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010705696000000001 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.500766+0000", 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022946191000000001 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508150+0000", 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073836930000000002 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508253+0000", 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103222 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508541+0000", 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028815600000000002 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508725+0000", 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183766 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.543910+0000", 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035184897 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859196+0000", 2024-09-19T05:44:13.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31528614500000002 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859243+0000", 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6727000000000002e-05 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859290+0000", 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6922000000000002e-05 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:268 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.938757392999999, 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37283424700000001, 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 268 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599857+0000", 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.228e-06 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604581+0000", 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047240240000000003 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604601+0000", 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0738e-05 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604613+0000", 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621794+0000", 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017181279000000001 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621899+0000", 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104218 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.622059+0000", 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015997200000000001 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629055+0000", 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069966560000000004 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972534+0000", 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34347880400000003 2024-09-19T05:44:13.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972610+0000", 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6398999999999998e-05 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972687+0000", 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6038999999999995e-05 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:272 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.922205440999999, 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40633524799999998, 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 272 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616407+0000", 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.993e-06 2024-09-19T05:44:13.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631509+0000", 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015101787 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631525+0000", 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5829999999999999e-05 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631535+0000", 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0224000000000001e-05 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631631+0000", 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5696e-05 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631752+0000", 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000121605 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.632659+0000", 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090671499999999999 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.708071+0000", 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075412347000000005 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022627+0000", 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31455541599999998 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022684+0000", 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7543000000000001e-05 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022739+0000", 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5092e-05 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:292 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.890942690999999, 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53039040900000001, 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 292 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647670+0000", 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:13.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657999+0000", 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010329481 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667975+0000", 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0099757179999999997 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667984+0000", 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8470000000000002e-06 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668049+0000", 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4999000000000006e-05 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668145+0000", 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6514000000000005e-05 2024-09-19T05:44:13.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668295+0000", 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015011399999999999 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.729044+0000", 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060748914000000001 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.177943+0000", 2024-09-19T05:44:13.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44889912500000001 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178002+0000", 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8829000000000002e-05 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178057+0000", 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5284999999999997e-05 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:296 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.86676258, 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43521636699999999, 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 296 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.591 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671851+0000", 2024-09-19T05:44:13.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.659e-06 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688220+0000", 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016369595000000001 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688250+0000", 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9499e-05 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688262+0000", 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2031999999999999e-05 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688539+0000", 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027760399999999998 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688662+0000", 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122031 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688813+0000", 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015106399999999999 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.765112+0000", 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076299661000000005 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106912+0000", 2024-09-19T05:44:13.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34180015600000002 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106973+0000", 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0927999999999997e-05 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.107063+0000", 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0137999999999997e-05 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:288 3.31 e54/20 3:8d4d39bf:::10000000002.0000001d:head v 54'5, mlcod=53'3)", 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.840691782, 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48371873900000001, 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:13.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697920+0000", 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7719999999999999e-06 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099856+0000", 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40193617399999998 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099878+0000", 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2306999999999998e-05 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099891+0000", 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132365+0000", 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032473872000000001 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132403+0000", 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8396999999999998e-05 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181593+0000", 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049190325 2024-09-19T05:44:13.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181637+0000", 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3368000000000002e-05 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:337 3.34 3:2c8ffcca:::10000000002.0000004e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.760615082999999, 2024-09-19T05:44:13.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24486184699999999, 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 337 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777997+0000", 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.17e-06 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781851+0000", 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038539429999999999 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781870+0000", 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9012e-05 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781882+0000", 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1864e-05 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781987+0000", 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105691 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782084+0000", 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7072999999999997e-05 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782457+0000", 2024-09-19T05:44:13.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037225700000000002 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799269+0000", 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016812247999999998 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022778+0000", 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22350931499999999 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022823+0000", 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4502999999999997e-05 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022856+0000", 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3770999999999999e-05 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:357 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:13.607 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.698530087, 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33810015799999998, 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 357 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840082+0000", 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2029999999999999e-06 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842172+0000", 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020904890000000001 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842193+0000", 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1058999999999999e-05 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842207+0000", 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3444e-05 2024-09-19T05:44:13.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848376+0000", 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061690909999999998 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848486+0000", 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109979 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848644+0000", 2024-09-19T05:44:13.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015821699999999999 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.894080+0000", 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045435709999999997 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178104+0000", 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28402411700000002 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178148+0000", 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4492e-05 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178180+0000", 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1356999999999998e-05 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:363 3.3f 3:ff5df6dc:::10000000002.00000068:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.613 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.688228733999999, 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32356367800000002, 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 363 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850383+0000", 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0600000000000002e-06 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862283+0000", 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011899873 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862330+0000", 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862352+0000", 2024-09-19T05:44:13.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1141e-05 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.877929+0000", 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015577153 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878045+0000", 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011612000000000001 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:13.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878204+0000", 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158832 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896926+0000", 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018722714000000001 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173814+0000", 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27688780099999999 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173871+0000", 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6907000000000003e-05 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173945+0000", 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3530999999999998e-05 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:365 3.c 3:3214729d:::10000000002.0000006a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.650208334, 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27874868600000002, 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 365 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888405+0000", 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7620000000000001e-06 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892569+0000", 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041637540000000004 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892661+0000", 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2134999999999998e-05 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892675+0000", 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4173e-05 2024-09-19T05:44:13.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908596+0000", 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015920815000000001 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908712+0000", 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011617599999999999 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908878+0000", 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165935 2024-09-19T05:44:13.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972488+0000", 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063609938000000005 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167039+0000", 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19455101699999999 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167097+0000", 2024-09-19T05:44:13.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7776999999999999e-05 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167150+0000", 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3204000000000001e-05 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:372 3.34 3:2ebe227a:::10000000002.00000071:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:13.625 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.63650543, 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.264918877, 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 372 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:13.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902109+0000", 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6990000000000002e-06 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910547+0000", 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084382120000000005 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910566+0000", 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9111e-05 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910579+0000", 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2330999999999999e-05 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923326+0000", 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012747596 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923501+0000", 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000174802 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923775+0000", 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027407199999999999 2024-09-19T05:44:13.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023005+0000", 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099229554999999997 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.157725+0000", 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13472034199999999 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.158104+0000", 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037887499999999998 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167023+0000", 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089192820000000006 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:315 3.1c e54/38 3:394f8a26:::10000000002.00000038:head v 54'8, mlcod=53'5)", 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:13.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.633085025, 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28226446599999999, 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:13.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905527+0000", 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5739999999999998e-06 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939034+0000", 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033507185000000002 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939067+0000", 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2186e-05 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939084+0000", 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6982000000000001e-05 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972466+0000", 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033381985000000003 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972491+0000", 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5789e-05 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187219+0000", 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21472724700000001 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187789+0000", 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000570518 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:373 3.36 3:6d44e697:::10000000002.00000072:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.628001868, 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29574129399999999, 2024-09-19T05:44:13.636 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 373 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910611+0000", 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5779999999999999e-06 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.919588+0000", 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089768220000000006 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938096+0000", 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018507920000000001 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938107+0000", 2024-09-19T05:44:13.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0488e-05 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944322+0000", 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062149900000000001 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944546+0000", 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022402300000000001 2024-09-19T05:44:13.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944884+0000", 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033806299999999999 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.074757+0000", 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.129873715 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206239+0000", 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13148132000000001 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206302+0000", 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3046000000000006e-05 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206349+0000", 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7329e-05 2024-09-19T05:44:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:348 3.1c e54/38 3:3aa14ed1:::10000000002.00000059:head v 54'9, mlcod=53'5)", 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.593938607, 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24318706500000001, 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944672+0000", 2024-09-19T05:44:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1790000000000001e-06 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947488+0000", 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028159130000000002 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947504+0000", 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5367e-05 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947511+0000", 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3300000000000001e-06 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006421+0000", 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058910000999999997 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006451+0000", 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0576e-05 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187796+0000", 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18134434799999999 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187858+0000", 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2351e-05 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:395 3.1c e54/38 3:384ad1d9:::10000000002.00000088:head v 54'10, mlcod=53'5)", 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.591081719, 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24038785900000001, 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947529+0000", 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7400000000000002e-07 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959658+0000", 2024-09-19T05:44:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012129493999999999 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959675+0000", 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6996000000000001e-05 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959687+0000", 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2143999999999999e-05 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023041+0000", 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063353484000000002 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023062+0000", 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1171000000000001e-05 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187862+0000", 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.164799521 2024-09-19T05:44:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187916+0000", 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4175000000000002e-05 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:393 3.36 3:6ccb00e9:::10000000002.00000086:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.551616860999999, 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24664645599999999, 2024-09-19T05:44:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 393 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986998+0000", 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9899999999999997e-06 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995844+0000", 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0088464340000000002 2024-09-19T05:44:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995859+0000", 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4329e-05 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995866+0000", 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037368+0000", 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041501563999999998 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037440+0000", 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1773999999999994e-05 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037555+0000", 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011548399999999999 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117549+0000", 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079993623 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233527+0000", 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11597800699999999 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233591+0000", 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3789999999999997e-05 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233639+0000", 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8606999999999997e-05 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:13.660 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:13.660 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:13.658+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.708+0000 7f29acff9640 1 -- 172.21.15.2:0/62379464 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v61) ==== 87+0+0 (secure 0 0 0) 0x7f29a405cb00 con 0x7f29b0154c20 2024-09-19T05:44:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:44:13.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f299c06c2b0 msgr2=0x7f299c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f299c06c2b0 0x7f299c06e770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f29a0034000 tx=0x7f29a002e000 comp rx=0 tx=0).stop 2024-09-19T05:44:13.712 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29b0154c20 msgr2=0x7f29b0156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29b0154c20 0x7f29b0156700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c430 tx=0x7f29a400c830 comp rx=0 tx=0).stop 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 shutdown_connections 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f299c06c2b0 0x7f299c06e770 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b0155000 0x7f29b0156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29b0154c20 0x7f29b0156700 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 --2- 172.21.15.2:0/62379464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29b00caf40 0x7f29b00b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 >> 172.21.15.2:0/62379464 conn(0x7f29b006d0e0 msgr2=0x7f29b014f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 shutdown_connections 2024-09-19T05:44:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.711+0000 7f29b6391640 1 -- 172.21.15.2:0/62379464 wait complete. 2024-09-19T05:44:13.760 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:13.758+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:13.810 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:13.810 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:13.810 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:13.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:13.861 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:13.860+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:13.874 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:13.927 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:13.927 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:13.927 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:13.927 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:13.928 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:13.937 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:13.961 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:13.960+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:13.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.975+0000 7fed1195f640 1 Processor -- start 2024-09-19T05:44:13.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.975+0000 7fed1195f640 1 -- start start 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.976+0000 7fed1195f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c0cd850 0x7fed0c0c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.976+0000 7fed1195f640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fed0c15e9b0 0x7fed0c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.976+0000 7fed1195f640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed0c0cb3c0 con 0x7fed0c0cd470 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.976+0000 7fed1195f640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fed0c0cc4d0 con 0x7fed0c15e9b0 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.976+0000 7fed1195f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c0cb6a0 con 0x7fed0c0cd850 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.976+0000 7fed0b7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fed0c15e9b0 0x7fed0c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.977+0000 7fed0b7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fed0c15e9b0 0x7fed0c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33248/0 (socket says 172.21.15.2:33248) 2024-09-19T05:44:13.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.977+0000 7fed0b7fe640 1 -- 172.21.15.2:0/142667747 learned_addr learned my addr 172.21.15.2:0/142667747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:13.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.977+0000 7fed03fff640 1 --2- 172.21.15.2:0/142667747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c0cd850 0x7fed0c0c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.977+0000 7fed0b7fe640 1 -- 172.21.15.2:0/142667747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c0cd850 msgr2=0x7fed0c0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:13.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.977+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/142667747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c0cd850 0x7fed0c0c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.977+0000 7fed0b7fe640 1 -- 172.21.15.2:0/142667747 >> v1:172.21.15.123:6789/0 conn(0x7fed0c0cd470 legacy=0x7fed0c1548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.977+0000 7fed0b7fe640 1 -- 172.21.15.2:0/142667747 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0cc720 con 0x7fed0c15e9b0 2024-09-19T05:44:13.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/142667747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c15e9b0 0x7fed0c15aae0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed04009e50 tx=0x7fed0402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6c8f9702dd1eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed0a7fc640 1 -- 172.21.15.2:0/142667747 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed04036070 con 0x7fed0c15e9b0 2024-09-19T05:44:13.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed0a7fc640 1 -- 172.21.15.2:0/142667747 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed04031600 con 0x7fed0c15e9b0 2024-09-19T05:44:13.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed0a7fc640 1 -- 172.21.15.2:0/142667747 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed040319d0 con 0x7fed0c15e9b0 2024-09-19T05:44:13.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed1195f640 1 -- 172.21.15.2:0/142667747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c15e9b0 msgr2=0x7fed0c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed1195f640 1 --2- 172.21.15.2:0/142667747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c15e9b0 0x7fed0c15aae0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed04009e50 tx=0x7fed0402a7a0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed1195f640 1 -- 172.21.15.2:0/142667747 shutdown_connections 2024-09-19T05:44:13.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed1195f640 1 --2- 172.21.15.2:0/142667747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c15e9b0 0x7fed0c15aae0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed1195f640 1 --2- 172.21.15.2:0/142667747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c0cd850 0x7fed0c0c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.978+0000 7fed1195f640 1 -- 172.21.15.2:0/142667747 >> 172.21.15.2:0/142667747 conn(0x7fed0c067b30 msgr2=0x7fed0c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:13.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.979+0000 7fed1195f640 1 -- 172.21.15.2:0/142667747 shutdown_connections 2024-09-19T05:44:13.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.979+0000 7fed1195f640 1 -- 172.21.15.2:0/142667747 wait complete. 2024-09-19T05:44:13.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.980+0000 7fed1195f640 1 Processor -- start 2024-09-19T05:44:13.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.980+0000 7fed1195f640 1 -- start start 2024-09-19T05:44:13.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed1195f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c0cd470 0x7fed0c1e3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed1195f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed0c0cd850 0x7fed0c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed1195f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c15e9b0 0x7fed0c0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed1195f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c155130 con 0x7fed0c0cd470 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed1195f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed0c154fb0 con 0x7fed0c0cd850 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed1195f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fed0c0c58b0 con 0x7fed0c15e9b0 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c0cd470 0x7fed0c1e3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c0cd470 0x7fed0c1e3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33258/0 (socket says 172.21.15.2:33258) 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 -- 172.21.15.2:0/918186371 learned_addr learned my addr 172.21.15.2:0/918186371 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed03fff640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed0c0cd850 0x7fed0c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c15e9b0 0x7fed0c0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c15e9b0 msgr2=0x7fed0c0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.985 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed0c15e9b0 0x7fed0c0b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed0c0cd850 msgr2=0x7fed0c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed0c0cd850 0x7fed0c0b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0c0b4bd0 con 0x7fed0c0cd470 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c0cd470 0x7fed0c1e3c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c430 tx=0x7fecfc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0052d0 con 0x7fed0c0cd470 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.982+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc005490 con 0x7fed0c0cd470 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.983+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed0c0b6550 con 0x7fed0c0cd470 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.983+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc007260 con 0x7fed0c0cd470 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.983+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed0c15cac0 con 0x7fed0c0cd470 2024-09-19T05:44:13.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.983+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fed0c15c5d0 con 0x7fed0c0cd470 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.984+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fecfc007400 con 0x7fed0c0cd470 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed08ff9640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecec06c2e0 0x7fecec06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5741+0+0 (secure 0 0 0) 0x7fecfc08a830 con 0x7fed0c0cd470 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v61) ==== 54+0+16 (secure 0 0 0) 0x7fecfc08f240 con 0x7fed0c0cd470 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed03fff640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecec06c2e0 0x7fecec06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fed0c15d1a0 0x7fed0c1f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed0c0c52d0 con 0x7fed0c15d1a0 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fed0c15d1a0 0x7fed0c1f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.985+0000 7fed03fff640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecec06c2e0 0x7fecec06e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed0c15b820 tx=0x7fecf4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.986+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fed0c15d1a0 0x7fed0c1f3910 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:13.988 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:13.986+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c0c52d0 con 0x7fed0c15d1a0 2024-09-19T05:44:14.002 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.001+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c0cbdf0 con 0x7fed0c15d1a0 2024-09-19T05:44:14.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.002+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c0cbdf0 con 0x7fed0c15d1a0 2024-09-19T05:44:14.004 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:14.004 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:14.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.003+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fed0c1e64f0 0x7fed0c1f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:14.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.003+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fed0c1594d0 con 0x7fed0c1e64f0 2024-09-19T05:44:14.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.003+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fed0c1e64f0 0x7fed0c1f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:14.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.003+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fed0c1e64f0 0x7fed0c1f4360 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:14.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.004+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c1594d0 con 0x7fed0c1e64f0 2024-09-19T05:44:14.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.018+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c15af60 con 0x7fed0c1e64f0 2024-09-19T05:44:14.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.020+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c15af60 con 0x7fed0c1e64f0 2024-09-19T05:44:14.021 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:14.022 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:14.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.020+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fed0c14d080 0x7fed0c14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:14.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.020+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fed0c14d9c0 con 0x7fed0c14d080 2024-09-19T05:44:14.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.020+0000 7fed03fff640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fed0c14d080 0x7fed0c14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:14.022 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.021+0000 7fed03fff640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fed0c14d080 0x7fed0c14d480 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:14.023 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.021+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fecf4011e70 con 0x7fed0c14d080 2024-09-19T05:44:14.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.035+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c062900 con 0x7fed0c14d080 2024-09-19T05:44:14.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.037+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c062900 con 0x7fed0c14d080 2024-09-19T05:44:14.038 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:14.039 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:14.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.037+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fed0c1e7090 0x7fed0c1e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:14.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.037+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fed0c1e4320 con 0x7fed0c1e7090 2024-09-19T05:44:14.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.037+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fed0c1e7090 0x7fed0c1e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:14.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.037+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fed0c1e7090 0x7fed0c1e74f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:14.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.038+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c1e4320 con 0x7fed0c1e7090 2024-09-19T05:44:14.049 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:14.049 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:14.052 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.051+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c1e4320 con 0x7fed0c1e7090 2024-09-19T05:44:14.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.053+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c1e4320 con 0x7fed0c1e7090 2024-09-19T05:44:14.054 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:14.055 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:14.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.053+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fed0c02c640 0x7fed0c02caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:14.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.053+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fed0c1e7f40 con 0x7fed0c02c640 2024-09-19T05:44:14.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.054+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fed0c02c640 0x7fed0c02caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:14.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.054+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fed0c02c640 0x7fed0c02caa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:14.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.055+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c1e7f40 con 0x7fed0c02c640 2024-09-19T05:44:14.059 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:14.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.069+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c1e7f40 con 0x7fed0c02c640 2024-09-19T05:44:14.071 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:14.070+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:14.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.071+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c1e7f40 con 0x7fed0c02c640 2024-09-19T05:44:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:14.073 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:14.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.072+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fed0c02d4f0 0x7fed0c206eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.072+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fed0c1e7f40 con 0x7fed0c02d4f0 2024-09-19T05:44:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.072+0000 7fed03fff640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fed0c02d4f0 0x7fed0c206eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.072+0000 7fed03fff640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fed0c02d4f0 0x7fed0c206eb0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.073+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c1e7f40 con 0x7fed0c02d4f0 2024-09-19T05:44:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.087+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c1e7f40 con 0x7fed0c02d4f0 2024-09-19T05:44:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.089+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c1e7f40 con 0x7fed0c02d4f0 2024-09-19T05:44:14.090 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.089+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fed0c207900 0x7fed0c209cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.089+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fed0c02db80 con 0x7fed0c207900 2024-09-19T05:44:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.089+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fed0c207900 0x7fed0c209cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.090+0000 7fed0b7fe640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fed0c207900 0x7fed0c209cf0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:14.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.090+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c02db80 con 0x7fed0c207900 2024-09-19T05:44:14.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.104+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c02db50 con 0x7fed0c207900 2024-09-19T05:44:14.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.106+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c02db50 con 0x7fed0c207900 2024-09-19T05:44:14.107 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.106+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fed0c20a7a0 0x7fed0c20abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.106+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fed0c20fce0 con 0x7fed0c20a7a0 2024-09-19T05:44:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.107+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fed0c20a7a0 0x7fed0c20abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.107+0000 7fed0affd640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fed0c20a7a0 0x7fed0c20abe0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:14.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.108+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed0c20fce0 con 0x7fed0c20a7a0 2024-09-19T05:44:14.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.122+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fed0c20b230 con 0x7fed0c20a7a0 2024-09-19T05:44:14.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.123+0000 7fed08ff9640 1 -- 172.21.15.2:0/918186371 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fed0c20b230 con 0x7fed0c20a7a0 2024-09-19T05:44:14.125 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:14.126 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:14.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fed0c15d1a0 msgr2=0x7fed0c1f3910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fed0c15d1a0 0x7fed0c1f3910 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fed0c1e64f0 msgr2=0x7fed0c1f4360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fed0c1e64f0 0x7fed0c1f4360 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fed0c14d080 msgr2=0x7fed0c14d480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fed0c14d080 0x7fed0c14d480 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fed0c1e7090 msgr2=0x7fed0c1e74f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fed0c1e7090 0x7fed0c1e74f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fed0c02c640 msgr2=0x7fed0c02caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fed0c02c640 0x7fed0c02caa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed0b7fe640 1 -- 172.21.15.2:0/918186371 reap_dead start 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fed0c02d4f0 msgr2=0x7fed0c206eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fed0c02d4f0 0x7fed0c206eb0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed0b7fe640 1 -- 172.21.15.2:0/918186371 reap_dead start 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fed0c207900 msgr2=0x7fed0c209cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fed0c207900 0x7fed0c209cf0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fed0c20a7a0 msgr2=0x7fed0c20abe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed0b7fe640 1 -- 172.21.15.2:0/918186371 reap_dead start 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fed0c20a7a0 0x7fed0c20abe0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecec06c2e0 msgr2=0x7fecec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecec06c2e0 0x7fecec06e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed0c15b820 tx=0x7fecf4009190 comp rx=0 tx=0).stop 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c0cd470 msgr2=0x7fed0c1e3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c0cd470 0x7fed0c1e3c70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c430 tx=0x7fecfc00c8f0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 shutdown_connections 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fed0c20a7a0 0x7fed0c20abe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fecec06c2e0 0x7fecec06e7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 --2- 172.21.15.2:0/918186371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed0c0cd470 0x7fed0c1e3c70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 >> 172.21.15.2:0/918186371 conn(0x7fed0c067b30 msgr2=0x7fed0c15a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 shutdown_connections 2024-09-19T05:44:14.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:14.124+0000 7fed1195f640 1 -- 172.21.15.2:0/918186371 wait complete. 2024-09-19T05:44:14.162 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:14.161+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:14.186 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:14.187 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:14.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:14.263 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:14.262+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:14.316 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:14.316 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:14.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:14.371 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:14.367+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:14.467 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:14.464+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:14.581 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:14.565+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:14.669 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:14.667+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:14.753 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:14.753 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:14.768 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:14.767+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:14.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:14.873 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:14.871+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:14.970 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:14.968+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:15.025 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:15.026 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 rewriters = 1218023.88 kB/sec 2024-09-19T05:44:15.026 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 rewriters = 1217732.51 kB/sec 2024-09-19T05:44:15.026 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 1218023.88 kB/sec 2024-09-19T05:44:15.026 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 1218023.88 kB/sec 2024-09-19T05:44:15.026 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 1218023.88 kB/sec 2024-09-19T05:44:15.026 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:44:15.046 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.046 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:15.071 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:15.070+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:15.172 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:15.170+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:15.202 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.202 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:15.273 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:15.272+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:15.324 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.325 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:15.377 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:15.376+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:15.442 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.442 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.451 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:15.492 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:15.490+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:15.576 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:15.574+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:15.594 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.594 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.603 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:15.676 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:15.675+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:15.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:44:15.746 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.747 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:15.776 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:15.776+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:15.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.834+0000 7faefef31640 1 Processor -- start 2024-09-19T05:44:15.835 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.834+0000 7faefef31640 1 -- start start 2024-09-19T05:44:15.836 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefef31640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7faef815b2a0 0x7faef80718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefef31640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7faef80cbc50 0x7faef8071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefef31640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef80ca910 con 0x7faef80cb870 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefef31640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7faef806d930 con 0x7faef80cbc50 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefef31640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7faef80caed0 con 0x7faef815b2a0 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefd4ce640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7faef80cbc50 0x7faef8071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefd4ce640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7faef80cbc50 0x7faef8071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33264/0 (socket says 172.21.15.2:33264) 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefd4ce640 1 -- 172.21.15.2:0/2420023414 learned_addr learned my addr 172.21.15.2:0/2420023414 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faef7fff640 1 --2- 172.21.15.2:0/2420023414 >> v2:172.21.15.123:3300/0 conn(0x7faef815b2a0 0x7faef80718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefd4ce640 1 -- 172.21.15.2:0/2420023414 >> v2:172.21.15.123:3300/0 conn(0x7faef815b2a0 msgr2=0x7faef80718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefd4ce640 1 --2- 172.21.15.2:0/2420023414 >> v2:172.21.15.123:3300/0 conn(0x7faef815b2a0 0x7faef80718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefd4ce640 1 -- 172.21.15.2:0/2420023414 >> v1:172.21.15.123:6789/0 conn(0x7faef80cb870 legacy=0x7faef815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:15.837 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.835+0000 7faefd4ce640 1 -- 172.21.15.2:0/2420023414 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef80cabf0 con 0x7faef80cbc50 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.836+0000 7faefd4ce640 1 --2- 172.21.15.2:0/2420023414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cbc50 0x7faef8071de0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faef80c8bc0 tx=0x7faef002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741aa0e0300721bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.836+0000 7faef77fe640 1 -- 172.21.15.2:0/2420023414 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0036070 con 0x7faef80cbc50 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.836+0000 7faef77fe640 1 -- 172.21.15.2:0/2420023414 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faef0031600 con 0x7faef80cbc50 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.836+0000 7faef77fe640 1 -- 172.21.15.2:0/2420023414 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00319d0 con 0x7faef80cbc50 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 -- 172.21.15.2:0/2420023414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cbc50 msgr2=0x7faef8071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 --2- 172.21.15.2:0/2420023414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cbc50 0x7faef8071de0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faef80c8bc0 tx=0x7faef002a730 comp rx=0 tx=0).stop 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 -- 172.21.15.2:0/2420023414 shutdown_connections 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 --2- 172.21.15.2:0/2420023414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cbc50 0x7faef8071de0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 --2- 172.21.15.2:0/2420023414 >> v2:172.21.15.123:3300/0 conn(0x7faef815b2a0 0x7faef80718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 -- 172.21.15.2:0/2420023414 >> 172.21.15.2:0/2420023414 conn(0x7faef806bdf0 msgr2=0x7faef806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 -- 172.21.15.2:0/2420023414 shutdown_connections 2024-09-19T05:44:15.839 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.837+0000 7faefef31640 1 -- 172.21.15.2:0/2420023414 wait complete. 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.838+0000 7faefef31640 1 Processor -- start 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.838+0000 7faefef31640 1 -- start start 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefef31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cb870 0x7faef8153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefef31640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faef80cbc50 0x7faef8145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefef31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faef815b2a0 0x7faef81464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefef31640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faef806d130 con 0x7faef80cb870 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefef31640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faef806d4a0 con 0x7faef815b2a0 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefef31640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7faef80ca420 con 0x7faef80cbc50 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cb870 0x7faef8153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cb870 0x7faef8153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33272/0 (socket says 172.21.15.2:33272) 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 -- 172.21.15.2:0/355960523 learned_addr learned my addr 172.21.15.2:0/355960523 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefd4ce640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faef815b2a0 0x7faef81464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:15.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faef7fff640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faef80cbc50 0x7faef8145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 -- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faef80cbc50 msgr2=0x7faef8145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faef80cbc50 0x7faef8145f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 -- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faef815b2a0 msgr2=0x7faef81464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faef815b2a0 0x7faef81464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.839+0000 7faefcccd640 1 -- 172.21.15.2:0/355960523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef81f4b70 con 0x7faef80cb870 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.840+0000 7faefcccd640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cb870 0x7faef8153610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c400 tx=0x7faeec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.840+0000 7faef57fa640 1 -- 172.21.15.2:0/355960523 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0052d0 con 0x7faef80cb870 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.840+0000 7faef57fa640 1 -- 172.21.15.2:0/355960523 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec005490 con 0x7faef80cb870 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.840+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef81f0b90 con 0x7faef80cb870 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.840+0000 7faef57fa640 1 -- 172.21.15.2:0/355960523 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec007260 con 0x7faef80cb870 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.840+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81f1150 con 0x7faef80cb870 2024-09-19T05:44:15.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.841+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef815ab90 con 0x7faef80cb870 2024-09-19T05:44:15.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.842+0000 7faef57fa640 1 -- 172.21.15.2:0/355960523 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faeec007400 con 0x7faef80cb870 2024-09-19T05:44:15.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.842+0000 7faef57fa640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faedc06c2b0 0x7faedc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:15.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.842+0000 7faef57fa640 1 -- 172.21.15.2:0/355960523 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5770+0+0 (secure 0 0 0) 0x7faeec08ae30 con 0x7faef80cb870 2024-09-19T05:44:15.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.842+0000 7faef7fff640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faedc06c2b0 0x7faedc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:15.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.843+0000 7faef7fff640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faedc06c2b0 0x7faedc06e770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faef8148460 tx=0x7faee4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:15.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:15.846+0000 7faef57fa640 1 -- 172.21.15.2:0/355960523 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faeec058f10 con 0x7faef80cb870 2024-09-19T05:44:15.862 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.862 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:15.877 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:15.876+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:15.973 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:15.974 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:15.978 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:15.977+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:15.982 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:16.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.030+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7faef814adf0 con 0x7faef80cb870 2024-09-19T05:44:16.079 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:16.078+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:16.085 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:16.085 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:16.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:16.140 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:16.185 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:16.184+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:16.196 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:16.196 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:16.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.240+0000 7ff6be2a6640 1 Processor -- start 2024-09-19T05:44:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.240+0000 7ff6be2a6640 1 -- start start 2024-09-19T05:44:16.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6be2a6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 0x7ff6b80c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.242 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6be2a6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff6b80c8c10 0x7ff6b80c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6be2a6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6b80c6350 con 0x7ff6b8070470 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6be2a6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b80c7700 con 0x7ff6b80c8c10 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6be2a6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b8070230 con 0x7ff6b8154cf0 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6bc843640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff6b80c8c10 0x7ff6b80c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6bc843640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff6b80c8c10 0x7ff6b80c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33286/0 (socket says 172.21.15.2:33286) 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6bc843640 1 -- 172.21.15.2:0/370427269 learned_addr learned my addr 172.21.15.2:0/370427269 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/370427269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 0x7ff6b80c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6bc843640 1 -- 172.21.15.2:0/370427269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 msgr2=0x7ff6b80c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6bc843640 1 --2- 172.21.15.2:0/370427269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 0x7ff6b80c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6bc843640 1 -- 172.21.15.2:0/370427269 >> v1:172.21.15.123:6789/0 conn(0x7ff6b8070470 legacy=0x7ff6b815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6b6ffd640 1 -- 172.21.15.2:0/370427269 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560355410 0 0) 0x7ff6ac001110 con 0x7ff6b8070470 2024-09-19T05:44:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.241+0000 7ff6bc843640 1 -- 172.21.15.2:0/370427269 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b806ff50 con 0x7ff6b80c8c10 2024-09-19T05:44:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.242+0000 7ff6bc843640 1 --2- 172.21.15.2:0/370427269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b80c8c10 0x7ff6b80c80d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009e50 tx=0x7ff6b002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ed159bbe99a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.242+0000 7ff6b6ffd640 1 -- 172.21.15.2:0/370427269 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0036070 con 0x7ff6b80c8c10 2024-09-19T05:44:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.242+0000 7ff6b6ffd640 1 -- 172.21.15.2:0/370427269 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b0031600 con 0x7ff6b80c8c10 2024-09-19T05:44:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.242+0000 7ff6b6ffd640 1 -- 172.21.15.2:0/370427269 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00319d0 con 0x7ff6b80c8c10 2024-09-19T05:44:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 -- 172.21.15.2:0/370427269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b80c8c10 msgr2=0x7ff6b80c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/370427269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b80c8c10 0x7ff6b80c80d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009e50 tx=0x7ff6b002a7a0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 -- 172.21.15.2:0/370427269 shutdown_connections 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/370427269 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b80c8c10 0x7ff6b80c80d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/370427269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 0x7ff6b80c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 -- 172.21.15.2:0/370427269 >> 172.21.15.2:0/370427269 conn(0x7ff6b80cc610 msgr2=0x7ff6b80cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 -- 172.21.15.2:0/370427269 shutdown_connections 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.243+0000 7ff6be2a6640 1 -- 172.21.15.2:0/370427269 wait complete. 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.244+0000 7ff6be2a6640 1 Processor -- start 2024-09-19T05:44:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.244+0000 7ff6be2a6640 1 -- start start 2024-09-19T05:44:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6be2a6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b8070470 0x7ff6b8157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6be2a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6b80c8c10 0x7ff6b8157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6be2a6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 0x7ff6b814e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.248 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6be2a6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b81550d0 con 0x7ff6b8070470 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6be2a6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b80c7390 con 0x7ff6b80c8c10 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6be2a6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b815e630 con 0x7ff6b8154cf0 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b8070470 0x7ff6b8157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b8070470 0x7ff6b8157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33290/0 (socket says 172.21.15.2:33290) 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 -- 172.21.15.2:0/1354874707 learned_addr learned my addr 172.21.15.2:0/1354874707 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b77fe640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6b80c8c10 0x7ff6b8157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6bc843640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 0x7ff6b814e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 msgr2=0x7ff6b814e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff6b8154cf0 0x7ff6b814e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6b80c8c10 msgr2=0x7ff6b8157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6b80c8c10 0x7ff6b8157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b81e6640 con 0x7ff6b8070470 2024-09-19T05:44:16.249 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.245+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b8070470 0x7ff6b8157a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e870 tx=0x7ff6a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.246+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8017070 con 0x7ff6b8070470 2024-09-19T05:44:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.246+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a800c940 con 0x7ff6b8070470 2024-09-19T05:44:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.246+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b81e5f80 con 0x7ff6b8070470 2024-09-19T05:44:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.246+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800cd30 con 0x7ff6b8070470 2024-09-19T05:44:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.246+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b8150e20 con 0x7ff6b8070470 2024-09-19T05:44:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.247+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff6b815ea40 con 0x7ff6b8070470 2024-09-19T05:44:16.250 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.247+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff6a801a070 con 0x7ff6b8070470 2024-09-19T05:44:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6b4ff9640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6a406c2e0 0x7ff6a406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5770+0+0 (secure 0 0 0) 0x7ff6a808b270 con 0x7ff6b8070470 2024-09-19T05:44:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v63) ==== 54+0+16 (secure 0 0 0) 0x7ff6b81e6640 con 0x7ff6b8070470 2024-09-19T05:44:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6b77fe640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6a406c2e0 0x7ff6a406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff6b814e000 0x7ff6b806f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff6b81508f0 con 0x7ff6b814e000 2024-09-19T05:44:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6bc843640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff6b814e000 0x7ff6b806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.248+0000 7ff6b77fe640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6a406c2e0 0x7ff6a406e7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8072440 tx=0x7ff6ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.249+0000 7ff6bc843640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff6b814e000 0x7ff6b806f330 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.249+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b81508f0 con 0x7ff6b814e000 2024-09-19T05:44:16.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.263+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b80c6970 con 0x7ff6b814e000 2024-09-19T05:44:16.266 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.265+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b80c6970 con 0x7ff6b814e000 2024-09-19T05:44:16.266 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:16.267 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:16.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.265+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff6b81f45f0 0x7ff6b81f0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.265+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff6b8153be0 con 0x7ff6b81f45f0 2024-09-19T05:44:16.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.265+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff6b81f45f0 0x7ff6b81f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.265+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff6b81f45f0 0x7ff6b81f0900 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.267 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.266+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b8153be0 con 0x7ff6b81f45f0 2024-09-19T05:44:16.280 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:16.280+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.281+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b80ca890 con 0x7ff6b81f45f0 2024-09-19T05:44:16.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.282+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b80ca890 con 0x7ff6b81f45f0 2024-09-19T05:44:16.283 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.282+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff6b81f49d0 0x7ff6b81f1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.282+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff6b81e5cd0 con 0x7ff6b81f49d0 2024-09-19T05:44:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.283+0000 7ff6b77fe640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff6b81f49d0 0x7ff6b81f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.283+0000 7ff6b77fe640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff6b81f49d0 0x7ff6b81f1350 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.283+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b81e5cd0 con 0x7ff6b81f49d0 2024-09-19T05:44:16.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.297+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b8062880 con 0x7ff6b81f49d0 2024-09-19T05:44:16.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.299+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b8062880 con 0x7ff6b81f49d0 2024-09-19T05:44:16.300 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:16.300 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:16.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.299+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff6b8063360 0x7ff6b815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.299+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff6b815c050 con 0x7ff6b8063360 2024-09-19T05:44:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.299+0000 7ff6bc843640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff6b8063360 0x7ff6b815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.299+0000 7ff6bc843640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff6b8063360 0x7ff6b815bb10 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.300+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b815c050 con 0x7ff6b8063360 2024-09-19T05:44:16.307 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:16.307 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:16.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.313+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b815c530 con 0x7ff6b8063360 2024-09-19T05:44:16.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:16.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.315+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b815c530 con 0x7ff6b8063360 2024-09-19T05:44:16.317 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.315+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff6b802c640 0x7ff6b802caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.315+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff6b815c050 con 0x7ff6b802c640 2024-09-19T05:44:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.315+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff6b802c640 0x7ff6b802caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.316+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff6b802c640 0x7ff6b802caa0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.316+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b815c050 con 0x7ff6b802c640 2024-09-19T05:44:16.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.330+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b815c050 con 0x7ff6b802c640 2024-09-19T05:44:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.332+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b815c050 con 0x7ff6b802c640 2024-09-19T05:44:16.333 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.332+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7ff6b802d4f0 0x7ff6b8207200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.332+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff6b802d970 con 0x7ff6b802d4f0 2024-09-19T05:44:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.332+0000 7ff6b77fe640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7ff6b802d4f0 0x7ff6b8207200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.333+0000 7ff6b77fe640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7ff6b802d4f0 0x7ff6b8207200 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.333+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b802d970 con 0x7ff6b802d4f0 2024-09-19T05:44:16.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.347+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b802d970 con 0x7ff6b802d4f0 2024-09-19T05:44:16.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.350+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b802d970 con 0x7ff6b802d4f0 2024-09-19T05:44:16.351 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:16.352 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:16.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.350+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff6b8207c50 0x7ff6b820a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.350+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff6b802d970 con 0x7ff6b8207c50 2024-09-19T05:44:16.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.350+0000 7ff6bc843640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff6b8207c50 0x7ff6b820a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.352 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.350+0000 7ff6bc843640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff6b8207c50 0x7ff6b820a040 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.351+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b802d970 con 0x7ff6b8207c50 2024-09-19T05:44:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.365+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b802d970 con 0x7ff6b8207c50 2024-09-19T05:44:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.366+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b802d970 con 0x7ff6b8207c50 2024-09-19T05:44:16.368 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:16.368 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:16.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.367+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff6b820aa90 0x7ff6b820ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:16.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.367+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff6b820fdb0 con 0x7ff6b820aa90 2024-09-19T05:44:16.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.367+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff6b820aa90 0x7ff6b820ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:16.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.367+0000 7ff6b7fff640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff6b820aa90 0x7ff6b820ce80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:16.369 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.368+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff6b820fdb0 con 0x7ff6b820aa90 2024-09-19T05:44:16.381 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:16.380+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:16.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.381+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff6b820fd80 con 0x7ff6b820aa90 2024-09-19T05:44:16.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6b4ff9640 1 -- 172.21.15.2:0/1354874707 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff6b820fd80 con 0x7ff6b820aa90 2024-09-19T05:44:16.384 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff6b814e000 msgr2=0x7ff6b806f330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff6b814e000 0x7ff6b806f330 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff6b81f45f0 msgr2=0x7ff6b81f0900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff6b81f45f0 0x7ff6b81f0900 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff6b81f49d0 msgr2=0x7ff6b81f1350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff6b81f49d0 0x7ff6b81f1350 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff6b8063360 msgr2=0x7ff6b815bb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff6b8063360 0x7ff6b815bb10 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff6b802c640 msgr2=0x7ff6b802caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff6b802c640 0x7ff6b802caa0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7ff6b802d4f0 msgr2=0x7ff6b8207200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7ff6b802d4f0 0x7ff6b8207200 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff6b8207c50 msgr2=0x7ff6b820a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff6b8207c50 0x7ff6b820a040 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6bc843640 1 -- 172.21.15.2:0/1354874707 reap_dead start 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff6b820aa90 msgr2=0x7ff6b820ce80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff6b820aa90 0x7ff6b820ce80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.383+0000 7ff6bc843640 1 -- 172.21.15.2:0/1354874707 reap_dead start 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6bc843640 1 -- 172.21.15.2:0/1354874707 reap_dead start 2024-09-19T05:44:16.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6bc843640 1 -- 172.21.15.2:0/1354874707 reap_dead start 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6a406c2e0 msgr2=0x7ff6a406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6a406c2e0 0x7ff6a406e7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8072440 tx=0x7ff6ac009040 comp rx=0 tx=0).stop 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b8070470 msgr2=0x7ff6b8157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b8070470 0x7ff6b8157a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800e870 tx=0x7ff6a800ed30 comp rx=0 tx=0).stop 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 shutdown_connections 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6a406c2e0 0x7ff6a406e7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 --2- 172.21.15.2:0/1354874707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff6b8070470 0x7ff6b8157a70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 >> 172.21.15.2:0/1354874707 conn(0x7ff6b80cc610 msgr2=0x7ff6b80cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 shutdown_connections 2024-09-19T05:44:16.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.384+0000 7ff6be2a6640 1 -- 172.21.15.2:0/1354874707 wait complete. 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3 3.12 3:4bec3886:::10000000002.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:16.425 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.141761639999999, 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57240116200000002, 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:16.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280487+0000", 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.751e-06 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287574+0000", 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070870530000000003 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287592+0000", 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8241e-05 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287601+0000", 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2800000000000003e-06 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287664+0000", 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3474999999999998e-05 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287758+0000", 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3577000000000002e-05 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287882+0000", 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012487199999999999 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.292606+0000", 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047238860000000001 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852756+0000", 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56015009400000004 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852823+0000", 2024-09-19T05:44:16.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6138999999999998e-05 2024-09-19T05:44:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852886+0000", 2024-09-19T05:44:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3794000000000006e-05 2024-09-19T05:44:16.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:27 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.119227641999998, 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55004442399999998, 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.432 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 27 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303023+0000", 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4340000000000001e-06 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316366+0000", 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013343634 2024-09-19T05:44:16.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316392+0000", 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6120999999999999e-05 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316402+0000", 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0066e-05 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316514+0000", 2024-09-19T05:44:16.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111859 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316686+0000", 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017135699999999999 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316868+0000", 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018244100000000001 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.324019+0000", 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071506060000000003 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852956+0000", 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52893702899999995 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853018+0000", 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2178e-05 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853064+0000", 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5698999999999997e-05 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:46 3.12 3:489b4446:::10000000002.0000002b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.090651776000001, 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54589824600000003, 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.438 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 46 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:16.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331598+0000", 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5689999999999999e-06 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347173+0000", 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015575235999999999 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347192+0000", 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9113000000000001e-05 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347202+0000", 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0151e-05 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347265+0000", 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3163000000000005e-05 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347408+0000", 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014307299999999999 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347532+0000", 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012328400000000001 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.351950+0000", 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044183620000000003 2024-09-19T05:44:16.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877426+0000", 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52547596500000004 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877461+0000", 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877493+0000", 2024-09-19T05:44:16.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2020000000000002e-05 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:51 3.12 3:499536aa:::10000000002.00000030:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.058902660000001, 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53664442800000001, 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.444 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 51 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363344+0000", 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.363346+0000", 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9709999999999998e-06 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383174+0000", 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019827860999999999 2024-09-19T05:44:16.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383192+0000", 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8456000000000001e-05 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383200+0000", 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9589999999999995e-06 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383263+0000", 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2502999999999994e-05 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383391+0000", 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012766399999999999 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.383510+0000", 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118998 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.386613+0000", 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031035260000000001 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899913+0000", 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.51330011600000003 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899958+0000", 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4487000000000003e-05 2024-09-19T05:44:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.899989+0000", 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0886999999999998e-05 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:157 3.12 3:4b703321:::10000000002.0000009a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.859609172999999, 2024-09-19T05:44:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34817550400000002, 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 157 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562638+0000", 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.562641+0000", 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2789999999999999e-06 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576231+0000", 2024-09-19T05:44:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013590286 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576263+0000", 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1875999999999998e-05 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576270+0000", 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4840000000000001e-06 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576349+0000", 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.975e-05 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576547+0000", 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000198154 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.576738+0000", 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019091400000000001 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.581161+0000", 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044224930000000004 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910746+0000", 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32958495700000001 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910782+0000", 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6636e-05 2024-09-19T05:44:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.910813+0000", 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0675e-05 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:158 3.1 3:82785032:::10000000002.0000009b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.84371204, 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.11650379299999999, 2024-09-19T05:44:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 158 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578535+0000", 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.578538+0000", 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4939999999999999e-06 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597197+0000", 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018658534000000001 2024-09-19T05:44:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597218+0000", 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1296000000000001e-05 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597224+0000", 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4020000000000001e-06 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597274+0000", 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0556000000000002e-05 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597387+0000", 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113119 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.597484+0000", 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6346000000000002e-05 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.600132+0000", 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026487059999999998 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.694929+0000", 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.094797131000000007 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.694990+0000", 2024-09-19T05:44:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0287000000000001e-05 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.695039+0000", 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8921999999999997e-05 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:184 3.12 3:4bdf515e:::10000000002.000000b5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.801118696, 2024-09-19T05:44:16.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29608825300000002, 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 184 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621128+0000", 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.621130+0000", 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0789999999999999e-06 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632344+0000", 2024-09-19T05:44:16.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01121422 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632369+0000", 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4712999999999999e-05 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632380+0000", 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1037e-05 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632436+0000", 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5887999999999998e-05 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632544+0000", 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107601 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.632641+0000", 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7460000000000005e-05 2024-09-19T05:44:16.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.636592+0000", 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039505950000000003 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917116+0000", 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.280524098 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917174+0000", 2024-09-19T05:44:16.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7720999999999998e-05 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.917216+0000", 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2840999999999999e-05 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:200 3.24 3:2525a577:::10000000002.000000c5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:16.468 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.770652616, 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.176819009, 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 200 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:16.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651594+0000", 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.651597+0000", 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5550000000000001e-06 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661419+0000", 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.009822239 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661445+0000", 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5922999999999999e-05 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661463+0000", 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8501000000000001e-05 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661572+0000", 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108471 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661690+0000", 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118387 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.661822+0000", 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013183800000000001 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.666069+0000", 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042470839999999999 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828327+0000", 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16225789600000001 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828368+0000", 2024-09-19T05:44:16.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1402000000000001e-05 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.828413+0000", 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4712999999999997e-05 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:252 3.24 3:25920220:::10000000002.000000f9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:16.474 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 22.717057377, 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12927656200000001, 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 252 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705189+0000", 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.705192+0000", 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5849999999999998e-06 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720126+0000", 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01493366 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720143+0000", 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7385e-05 2024-09-19T05:44:16.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720154+0000", 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0601000000000001e-05 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720232+0000", 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8584999999999994e-05 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720420+0000", 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018729799999999999 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.720631+0000", 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021127400000000001 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.728280+0000", 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076494989999999997 2024-09-19T05:44:16.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834381+0000", 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.106101029 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834423+0000", 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1634999999999997e-05 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.834466+0000", 2024-09-19T05:44:16.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3010999999999999e-05 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:283 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.808283955, 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57880034499999999, 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.481 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 283 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:16.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613965+0000", 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8759999999999999e-06 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633465+0000", 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019500483999999998 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633485+0000", 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.984e-05 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633494+0000", 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5520000000000005e-06 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633589+0000", 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5144999999999998e-05 2024-09-19T05:44:16.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633713+0000", 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012382300000000001 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633835+0000", 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122705 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.640960+0000", 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071243599999999997 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192691+0000", 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55173159599999999 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192737+0000", 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5487e-05 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192763+0000", 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6477000000000001e-05 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:294 3.24 3:2742a876:::10000000002.00000023:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.788713802, 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12659410500000001, 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.487 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 294 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633533+0000", 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633536+0000", 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.649e-06 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651603+0000", 2024-09-19T05:44:16.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018067155000000001 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651628+0000", 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4950999999999999e-05 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651639+0000", 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.113e-05 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.490 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651756+0000", 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117209 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651847+0000", 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1372000000000005e-05 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.652064+0000", 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021635099999999999 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.657902+0000", 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058383100000000002 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.759968+0000", 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.102066002 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760054+0000", 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5728000000000003e-05 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760127+0000", 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3248000000000001e-05 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:302 3.12 3:489b4446:::10000000002.0000002b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.770558995999998, 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54116458000000001, 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.493 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 302 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651688+0000", 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651691+0000", 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1810000000000001e-06 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664738+0000", 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013046540000000001 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664754+0000", 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6019999999999999e-05 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664767+0000", 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3315999999999999e-05 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664845+0000", 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8472000000000004e-05 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.496 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.664945+0000", 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9804999999999997e-05 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.665089+0000", 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014373700000000001 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.669251+0000", 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041617379999999999 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192796+0000", 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52354539899999997 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192829+0000", 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2668e-05 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192852+0000", 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3703999999999999e-05 2024-09-19T05:44:16.498 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:307 3.12 3:499536aa:::10000000002.00000030:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.736462495000001, 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.50714811299999996, 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.499 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 307 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685784+0000", 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.685788+0000", 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2480000000000001e-06 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698429+0000", 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012641172000000001 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698451+0000", 2024-09-19T05:44:16.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2172e-05 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698462+0000", 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1141000000000001e-05 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698591+0000", 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012867999999999999 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698704+0000", 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113409 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.698834+0000", 2024-09-19T05:44:16.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012981599999999999 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.703334+0000", 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045004600000000004 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192881+0000", 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48954666899999999 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192908+0000", 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6988e-05 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192932+0000", 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4358e-05 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:397 3.24 3:273fbfa7:::10000000002.0000008a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.612141746999999, 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24366095600000001, 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 397 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.506 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810105+0000", 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.810107+0000", 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.413e-06 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824232+0000", 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014124503 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824247+0000", 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4834000000000001e-05 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824259+0000", 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1769e-05 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824368+0000", 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109722 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824500+0000", 2024-09-19T05:44:16.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013183699999999999 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.824766+0000", 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000266016 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.830112+0000", 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053457620000000004 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053694+0000", 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22358192099999999 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053736+0000", 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2534999999999998e-05 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.053766+0000", 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9643999999999999e-05 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.511 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:413 3.12 3:4b703321:::10000000002.0000009a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.584504101, 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.36390152399999998, 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.512 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 413 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837743+0000", 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837745+0000", 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.04e-06 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851397+0000", 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013652687 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851412+0000", 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4776999999999999e-05 2024-09-19T05:44:16.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851420+0000", 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.43e-06 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851493+0000", 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3583000000000003e-05 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851600+0000", 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010678 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.851717+0000", 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117237 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.855412+0000", 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036952439999999999 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201584+0000", 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34617155799999999 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201618+0000", 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3494999999999997e-05 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.201644+0000", 2024-09-19T05:44:16.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6693e-05 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:422 3.1 3:8350567c:::10000000002.000000a3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.556163957999999, 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15887042900000001, 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.518 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 422 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866083+0000", 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.866086+0000", 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880367+0000", 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014281046 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880392+0000", 2024-09-19T05:44:16.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5054000000000001e-05 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880406+0000", 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4401e-05 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880532+0000", 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012579600000000001 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880667+0000", 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000135146 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.880827+0000", 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015946299999999999 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.886956+0000", 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061295730000000001 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024837+0000", 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.13788043999999999 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024890+0000", 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3483000000000003e-05 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.024953+0000", 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3097999999999998e-05 2024-09-19T05:44:16.523 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:440 3.12 3:4bdf515e:::10000000002.000000b5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.530081179, 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.31533669800000003, 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.524 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 440 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892166+0000", 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.892170+0000", 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6179999999999997e-06 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900748+0000", 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085778480000000008 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900763+0000", 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5231999999999999e-05 2024-09-19T05:44:16.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900773+0000", 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7240000000000004e-06 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900829+0000", 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5562000000000001e-05 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.900902+0000", 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3313999999999996e-05 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.901041+0000", 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000139419 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.903913+0000", 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0028715619999999998 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207434+0000", 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30352098799999999 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207468+0000", 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3861e-05 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.207502+0000", 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4570000000000003e-05 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.530 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:456 3.24 3:2525a577:::10000000002.000000c5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.497044954, 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.26312323300000001, 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.532 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 456 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925202+0000", 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.925204+0000", 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1110000000000002e-06 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938061+0000", 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012856615 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938083+0000", 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2543000000000001e-05 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938093+0000", 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0084e-05 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938178+0000", 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4827000000000006e-05 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938280+0000", 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010234900000000001 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.938437+0000", 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015626099999999999 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.942534+0000", 2024-09-19T05:44:16.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004097458 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188244+0000", 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.24570961099999999 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.537 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188281+0000", 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7134000000000001e-05 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188325+0000", 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4240000000000003e-05 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.538 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:487 3.33 3:ce8011cb:::10000000002.000000e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.474289657, 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.22306556599999999, 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 487 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.539 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947957+0000", 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.947959+0000", 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1519999999999999e-06 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962107+0000", 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014147649 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962127+0000", 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9868999999999999e-05 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962137+0000", 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0407e-05 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962206+0000", 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9122000000000007e-05 2024-09-19T05:44:16.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962312+0000", 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105982 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.962445+0000", 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000133108 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966463+0000", 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004017779 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170931+0000", 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.20446781999999999 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170977+0000", 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6245000000000002e-05 2024-09-19T05:44:16.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.171023+0000", 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5432999999999997e-05 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:508 3.24 3:25920220:::10000000002.000000f9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 17.449105849999999, 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21526778799999999, 2024-09-19T05:44:16.545 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 508 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973141+0000", 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973144+0000", 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.542e-06 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983914+0000", 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010770711000000001 2024-09-19T05:44:16.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983945+0000", 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0648999999999997e-05 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.983955+0000", 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8269999999999993e-06 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:16.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984017+0000", 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2652999999999998e-05 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984100+0000", 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3094000000000001e-05 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.984208+0000", 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107768 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.988635+0000", 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044265620000000002 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188359+0000", 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19972409799999999 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188389+0000", 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9882999999999998e-05 2024-09-19T05:44:16.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.188409+0000", 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0001e-05 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:16.551 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:16.553 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:16.553 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:16.552+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:16.653 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:16.652+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:16.697 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:16.697 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:16.697 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:16.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:16.754 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:16.753+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:16.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.755+0000 7faef57fa640 1 -- 172.21.15.2:0/355960523 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v64) ==== 97+0+0 (secure 0 0 0) 0x7faeec05d210 con 0x7faef80cb870 2024-09-19T05:44:16.757 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faedc06c2b0 msgr2=0x7faedc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faedc06c2b0 0x7faedc06e770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faef8148460 tx=0x7faee4009190 comp rx=0 tx=0).stop 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cb870 msgr2=0x7faef8153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cb870 0x7faef8153610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c400 tx=0x7faeec00c8c0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 shutdown_connections 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faedc06c2b0 0x7faedc06e770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faef815b2a0 0x7faef81464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faef80cbc50 0x7faef8145f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 --2- 172.21.15.2:0/355960523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faef80cb870 0x7faef8153610 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:16.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 >> 172.21.15.2:0/355960523 conn(0x7faef806bdf0 msgr2=0x7faef8154690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:16.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.758+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 shutdown_connections 2024-09-19T05:44:16.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:16.759+0000 7faefef31640 1 -- 172.21.15.2:0/355960523 wait complete. 2024-09-19T05:44:16.850 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:16.851 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:16.851 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:16.855 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:16.854+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:16.859 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:16.957 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:16.955+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:17.007 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:17.007 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:17 3.21 3:86f42aac:::10000000002.0000000e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.710726325, 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.072757916000000006, 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 17 2024-09-19T05:44:17.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:17.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293554+0000", 2024-09-19T05:44:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.293556+0000", 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1830000000000001e-06 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320078+0000", 2024-09-19T05:44:17.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.026521807000000001 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320090+0000", 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1873e-05 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320096+0000", 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.852e-06 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320193+0000", 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7145999999999995e-05 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320321+0000", 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012788799999999999 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.013 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320445+0000", 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000124206 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.325801+0000", 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053561700000000004 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366202+0000", 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.040400806999999997 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366258+0000", 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5971999999999999e-05 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.366312+0000", 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4011999999999997e-05 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:20 3.21 3:85380590:::10000000002.00000011:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:17.015 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.684157001999999, 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.094367290000000006, 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 20 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320123+0000", 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.320125+0000", 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5319999999999999e-06 2024-09-19T05:44:17.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345237+0000", 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025112457000000001 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345260+0000", 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3184999999999999e-05 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345274+0000", 2024-09-19T05:44:17.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3168e-05 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345351+0000", 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7600000000000002e-05 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345507+0000", 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015602300000000001 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345659+0000", 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015159599999999999 2024-09-19T05:44:17.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.351961+0000", 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063018880000000003 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414334+0000", 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.062373799000000001 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414436+0000", 2024-09-19T05:44:17.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101795 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.414491+0000", 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4246999999999997e-05 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:23 3.21 3:87fa6b7a:::10000000002.00000014:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:17.021 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.658973253999999, 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.106935525, 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 23 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:17.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345307+0000", 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.345309+0000", 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369545+0000", 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.024236034 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369565+0000", 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9928999999999998e-05 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369576+0000", 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0373e-05 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369677+0000", 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101765 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.369863+0000", 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000185692 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370039+0000", 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017557 2024-09-19T05:44:17.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.374766+0000", 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047269210000000002 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452093+0000", 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.077327075999999995 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452160+0000", 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7805999999999994e-05 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.452243+0000", 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2078999999999998e-05 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.027 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:39 3.2e 3:75eea580:::10000000002.00000024:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.634257697999999, 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41578167199999999, 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 39 2024-09-19T05:44:17.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370023+0000", 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.370025+0000", 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7369999999999998e-06 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393960+0000", 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023934536999999999 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393976+0000", 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6441000000000001e-05 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.393989+0000", 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2679e-05 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394066+0000", 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6919e-05 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394208+0000", 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014247500000000001 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.394304+0000", 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5369000000000001e-05 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.399842+0000", 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055380000000000004 2024-09-19T05:44:17.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785727+0000", 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.38588529199999999 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785763+0000", 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6494e-05 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.785804+0000", 2024-09-19T05:44:17.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0729000000000003e-05 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:62 3.c e53/20 3:31d8a57c:::10000000002.0000003b:head v 53'1, mlcod=0'0)", 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.626596494000001, 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.027856545, 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.034 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.377684+0000", 2024-09-19T05:44:17.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0600000000000001e-07 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400389+0000", 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022704832000000001 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400402+0000", 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2893e-05 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.036 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400411+0000", 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.037e-06 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400473+0000", 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1629000000000002e-05 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.400489+0000", 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6651e-05 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.405444+0000", 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049551279999999996 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.405540+0000", 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5969000000000002e-05 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:124 3.21 3:844ac05a:::10000000002.00000079:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.554838457999999, 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.060327543999999997, 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 124 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449442+0000", 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.449444+0000", 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2239999999999998e-06 2024-09-19T05:44:17.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464881+0000", 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015437222 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464904+0000", 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2274999999999999e-05 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.041 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464914+0000", 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0657e-05 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464999+0000", 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5031000000000006e-05 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.465166+0000", 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016689400000000001 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.465311+0000", 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000145208 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.470491+0000", 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051794229999999998 2024-09-19T05:44:17.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509673+0000", 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.039182002 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509724+0000", 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1205999999999997e-05 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.509769+0000", 2024-09-19T05:44:17.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5401999999999997e-05 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:138 3.2e 3:7768043d:::10000000002.00000087:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.539328537999999, 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34341082099999998, 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.045 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 138 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464952+0000", 2024-09-19T05:44:17.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.464954+0000", 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2180000000000002e-06 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482737+0000", 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017782893000000001 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482757+0000", 2024-09-19T05:44:17.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0352000000000001e-05 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482766+0000", 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0119999999999994e-06 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482841+0000", 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4386000000000002e-05 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.482989+0000", 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148405 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.483128+0000", 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013918200000000001 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.486268+0000", 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0031401229999999999 2024-09-19T05:44:17.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808279+0000", 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32201085299999999 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808321+0000", 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1962000000000002e-05 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.808363+0000", 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1434999999999999e-05 2024-09-19T05:44:17.050 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:257 3.21 3:86aaefcc:::10000000002.000000fe:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 23.505340843999999, 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.11301415099999999, 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.051 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 257 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498939+0000", 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.498941+0000", 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9350000000000001e-06 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521369+0000", 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022427589000000001 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521386+0000", 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6986999999999998e-05 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521393+0000", 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8569999999999997e-06 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521454+0000", 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1571999999999999e-05 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521577+0000", 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122998 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.521744+0000", 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016694100000000001 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.527047+0000", 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053028559999999999 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611743+0000", 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.084696294000000005 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611895+0000", 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015154900000000001 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.611954+0000", 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8573000000000003e-05 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.057 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:273 3.21 3:86f42aac:::10000000002.0000000e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.400945596, 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.13998105, 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 273 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603335+0000", 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.603338+0000", 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6899999999999998e-06 2024-09-19T05:44:17.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624529+0000", 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.021190186 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624551+0000", 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.213e-05 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624562+0000", 2024-09-19T05:44:17.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1452e-05 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624630+0000", 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7738999999999996e-05 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624721+0000", 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1168999999999994e-05 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624854+0000", 2024-09-19T05:44:17.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013313600000000001 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.630092+0000", 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0052376819999999996 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743176+0000", 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.113084107 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743260+0000", 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4309999999999997e-05 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.743316+0000", 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5448999999999997e-05 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:276 3.21 3:85380590:::10000000002.00000011:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.379688331000001, 2024-09-19T05:44:17.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15545139099999999, 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 276 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:17.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624592+0000", 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.624594+0000", 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0959999999999999e-06 2024-09-19T05:44:17.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650750+0000", 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.026156266000000001 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650772+0000", 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2027000000000001e-05 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650780+0000", 2024-09-19T05:44:17.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8210000000000002e-06 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650866+0000", 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5521999999999999e-05 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650959+0000", 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3524999999999996e-05 2024-09-19T05:44:17.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.651140+0000", 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018071799999999999 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.658731+0000", 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075914989999999998 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.779803+0000", 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.12107206399999999 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.779912+0000", 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108564 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780043+0000", 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013128900000000001 2024-09-19T05:44:17.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:279 3.21 3:87fa6b7a:::10000000002.00000014:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.353479849999999, 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17630584099999999, 2024-09-19T05:44:17.070 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 279 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650800+0000", 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.650803+0000", 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.232e-06 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684038+0000", 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.033235812000000003 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684068+0000", 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9118000000000001e-05 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684080+0000", 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2704999999999999e-05 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684157+0000", 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6879999999999996e-05 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684261+0000", 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010367 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684423+0000", 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016253799999999999 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.689113+0000", 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046892729999999999 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.826966+0000", 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.137853054 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.827028+0000", 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2030000000000001e-05 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.827106+0000", 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8529000000000007e-05 2024-09-19T05:44:17.076 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:295 3.2e 3:75eea580:::10000000002.00000024:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.077 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.320169963000001, 2024-09-19T05:44:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52131509399999998, 2024-09-19T05:44:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 295 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684110+0000", 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.684121+0000", 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0455000000000001e-05 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715121+0000", 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031000119 2024-09-19T05:44:17.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715142+0000", 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0871999999999999e-05 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715152+0000", 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.064e-05 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715284+0000", 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013133 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715363+0000", 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9676000000000001e-05 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715467+0000", 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000103816 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.722648+0000", 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071811460000000002 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205361+0000", 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48271265600000002 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205396+0000", 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4638000000000003e-05 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.205425+0000", 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9745999999999999e-05 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:318 3.c e54/20 3:31d8a57c:::10000000002.0000003b:head v 54'5, mlcod=53'3)", 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.292222224, 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035734850999999998, 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.084 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712058+0000", 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.712059+0000", 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.471e-06 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742630+0000", 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.030570631000000001 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742640+0000", 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0264e-05 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742648+0000", 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1890000000000001e-06 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742738+0000", 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0172000000000003e-05 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.742760+0000", 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2005e-05 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747734+0000", 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049739600000000004 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747793+0000", 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9159000000000001e-05 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:321 3.b 3:d381a89f:::10000000002.0000003e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:17.088 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.289102982999999, 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.038149887, 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 321 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715177+0000", 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.715179+0000", 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.08e-06 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746761+0000", 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.031581559000000002 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746774+0000", 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2994e-05 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746798+0000", 2024-09-19T05:44:17.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4443999999999998e-05 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746852+0000", 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3492000000000002e-05 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746956+0000", 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104455 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.747084+0000", 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012780800000000001 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.752122+0000", 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050380809999999998 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753229+0000", 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0011063399999999999 2024-09-19T05:44:17.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753281+0000", 2024-09-19T05:44:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2618999999999999e-05 2024-09-19T05:44:17.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.753327+0000", 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6014999999999999e-05 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:324 3.b 3:d0493c82:::10000000002.00000041:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.096 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.257452201, 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035384118999999999, 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 324 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746828+0000", 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.746830+0000", 2024-09-19T05:44:17.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0839999999999999e-06 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774044+0000", 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.027214075000000001 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774064+0000", 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0020000000000001e-05 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774074+0000", 2024-09-19T05:44:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3789999999999995e-06 2024-09-19T05:44:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774161+0000", 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7663000000000004e-05 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774271+0000", 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110229 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774421+0000", 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014956099999999999 2024-09-19T05:44:17.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.780042+0000", 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056212520000000002 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782100+0000", 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002057513 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782160+0000", 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0513999999999997e-05 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.782212+0000", 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1829000000000001e-05 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:339 3.b 3:d24a1766:::10000000002.00000050:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.103 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.230182068000001, 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.032212151000000001, 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 339 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774098+0000", 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.774100+0000", 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.968e-06 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796613+0000", 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.022512880999999998 2024-09-19T05:44:17.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796670+0000", 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7011e-05 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796685+0000", 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5208000000000001e-05 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796884+0000", 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000198641 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.797116+0000", 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000232368 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.797291+0000", 2024-09-19T05:44:17.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017451899999999999 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.802626+0000", 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005335528 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806209+0000", 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035828069999999999 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806260+0000", 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0961999999999998e-05 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.806310+0000", 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0257999999999999e-05 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.112 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:380 3.21 3:844ac05a:::10000000002.00000079:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.207531160999999, 2024-09-19T05:44:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.063018490999999996, 2024-09-19T05:44:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.113 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 380 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796749+0000", 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.796752+0000", 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.078e-06 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816450+0000", 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019697531000000001 2024-09-19T05:44:17.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816475+0000", 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5420000000000001e-05 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816505+0000", 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0199e-05 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816597+0000", 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1739999999999999e-05 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816725+0000", 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000128116 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816874+0000", 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148913 2024-09-19T05:44:17.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.821849+0000", 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004974493 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859654+0000", 2024-09-19T05:44:17.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.037805041999999997 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859717+0000", 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.859768+0000", 2024-09-19T05:44:17.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0772000000000001e-05 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:394 3.2e 3:7768043d:::10000000002.00000087:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.187721951, 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.400247509, 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.122 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 394 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816558+0000", 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.816560+0000", 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.745e-06 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833002+0000", 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016442149999999999 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833022+0000", 2024-09-19T05:44:17.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9446999999999999e-05 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833032+0000", 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0287e-05 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833104+0000", 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2365000000000003e-05 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833189+0000", 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4470999999999998e-05 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833328+0000", 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138913 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.837647+0000", 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043193600000000004 2024-09-19T05:44:17.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216711+0000", 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.379063539 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216765+0000", 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4302999999999998e-05 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.216806+0000", 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0929000000000001e-05 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:402 3.b 3:d1738051:::10000000002.0000008f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.128 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.171222029999999, 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.035654618999999999, 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 402 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833058+0000", 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.833060+0000", 2024-09-19T05:44:17.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7209999999999999e-06 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861464+0000", 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.028403690999999998 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861483+0000", 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8947999999999999e-05 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861494+0000", 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1168e-05 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861660+0000", 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016661000000000001 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861769+0000", 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108205 2024-09-19T05:44:17.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861892+0000", 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012325699999999999 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868420+0000", 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065282420000000001 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868597+0000", 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177195 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868658+0000", 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0844000000000002e-05 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.868713+0000", 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4737999999999997e-05 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.134 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:513 3.21 3:86aaefcc:::10000000002.000000fe:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 18.142754198999999, 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.104537124, 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.135 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 513 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861526+0000", 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.861528+0000", 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.875e-06 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877329+0000", 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015801329999999999 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877342+0000", 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2311e-05 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877347+0000", 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8220000000000003e-06 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877397+0000", 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9227999999999997e-05 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877488+0000", 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1828000000000002e-05 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.877601+0000", 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011255399999999999 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.882452+0000", 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048512929999999996 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.965958+0000", 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.083506094000000003 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966010+0000", 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1544e-05 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.966063+0000", 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3245000000000003e-05 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:17.140 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:17.141 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:17.145 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:17.143+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:17.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:17.245 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:17.244+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:17.259 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:17.259 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:17.259 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:17.259 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:17.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:17.346 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:17.345+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:17.415 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:17.415 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:17.415 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:17.416 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:17.416 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:17.425 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:17.469 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:17.468+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:17.535 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:12 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:17.535 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.247241109000001, 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.301330818, 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 12 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285078+0000", 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8979999999999998e-06 2024-09-19T05:44:17.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288881+0000", 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038028900000000002 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288908+0000", 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6831000000000001e-05 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288922+0000", 2024-09-19T05:44:17.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4647999999999999e-05 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289034+0000", 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011129 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289158+0000", 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012398199999999999 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289283+0000", 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012596100000000001 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302442+0000", 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01315887 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586307+0000", 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28386498300000002 2024-09-19T05:44:17.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586357+0000", 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9406000000000001e-05 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586405+0000", 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8059000000000002e-05 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:16 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.233117368999999, 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28733763699999998, 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 16 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299338+0000", 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013993499999999999 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305742+0000", 2024-09-19T05:44:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064047180000000002 2024-09-19T05:44:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305795+0000", 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2988000000000002e-05 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305808+0000", 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.307e-05 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305883+0000", 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4265999999999997e-05 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306078+0000", 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195281 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306298+0000", 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000219662 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.319662+0000", 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0133641 2024-09-19T05:44:17.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586451+0000", 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26678922700000002 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586494+0000", 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3102999999999997e-05 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586535+0000", 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1287e-05 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:36 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:17.549 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.215778982, 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54259264799999996, 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:17.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316539+0000", 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1310000000000002e-06 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326313+0000", 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0097740320000000002 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326341+0000", 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8178e-05 2024-09-19T05:44:17.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326354+0000", 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2767e-05 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326454+0000", 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9741999999999995e-05 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326598+0000", 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000144454 2024-09-19T05:44:17.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326740+0000", 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014189899999999999 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.381404+0000", 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054663549999999998 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859031+0000", 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47762663900000002 2024-09-19T05:44:17.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859074+0000", 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3578999999999997e-05 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859129+0000", 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4676999999999998e-05 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:40 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.205889634999998, 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29103886899999998, 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 40 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:17.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326428+0000", 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9160000000000001e-06 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343603+0000", 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017174545999999999 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343619+0000", 2024-09-19T05:44:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6260999999999999e-05 2024-09-19T05:44:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343630+0000", 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0774999999999999e-05 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343729+0000", 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8694000000000001e-05 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343846+0000", 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117588 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343969+0000", 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012271600000000001 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.376370+0000", 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032401093999999998 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617348+0000", 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24097808100000001 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617405+0000", 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6577999999999999e-05 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617464+0000", 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9620000000000002e-05 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:101 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.065203932999999, 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.392178684, 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 101 2024-09-19T05:44:17.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467114+0000", 2024-09-19T05:44:17.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.269e-06 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.477820+0000", 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010705696000000001 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.500766+0000", 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022946191000000001 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508150+0000", 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073836930000000002 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508253+0000", 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103222 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508541+0000", 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028815600000000002 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508725+0000", 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183766 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.543910+0000", 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035184897 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859196+0000", 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31528614500000002 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859243+0000", 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6727000000000002e-05 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859290+0000", 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6922000000000002e-05 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:268 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.932462799, 2024-09-19T05:44:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37283424700000001, 2024-09-19T05:44:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 268 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599857+0000", 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.228e-06 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604581+0000", 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047240240000000003 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604601+0000", 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0738e-05 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604613+0000", 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621794+0000", 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017181279000000001 2024-09-19T05:44:17.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621899+0000", 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104218 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.622059+0000", 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015997200000000001 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629055+0000", 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069966560000000004 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972534+0000", 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34347880400000003 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972610+0000", 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6398999999999998e-05 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972687+0000", 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6038999999999995e-05 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:272 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.915910846999999, 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40633524799999998, 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 272 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:17.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616407+0000", 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.993e-06 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631509+0000", 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015101787 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631525+0000", 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5829999999999999e-05 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631535+0000", 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0224000000000001e-05 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631631+0000", 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5696e-05 2024-09-19T05:44:17.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631752+0000", 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000121605 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.632659+0000", 2024-09-19T05:44:17.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090671499999999999 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.708071+0000", 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075412347000000005 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022627+0000", 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31455541599999998 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022684+0000", 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7543000000000001e-05 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022739+0000", 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5092e-05 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:292 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:17.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.884648096999999, 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53039040900000001, 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 292 2024-09-19T05:44:17.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.585 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:17.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:17.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647670+0000", 2024-09-19T05:44:17.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:17.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657999+0000", 2024-09-19T05:44:17.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010329481 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667975+0000", 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0099757179999999997 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667984+0000", 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8470000000000002e-06 2024-09-19T05:44:17.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668049+0000", 2024-09-19T05:44:17.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4999000000000006e-05 2024-09-19T05:44:17.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668145+0000", 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6514000000000005e-05 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668295+0000", 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015011399999999999 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.729044+0000", 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060748914000000001 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.177943+0000", 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44889912500000001 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178002+0000", 2024-09-19T05:44:17.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8829000000000002e-05 2024-09-19T05:44:17.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178057+0000", 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5284999999999997e-05 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:296 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.860467986, 2024-09-19T05:44:17.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43521636699999999, 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 296 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671851+0000", 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.659e-06 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688220+0000", 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016369595000000001 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688250+0000", 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9499e-05 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688262+0000", 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2031999999999999e-05 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688539+0000", 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027760399999999998 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688662+0000", 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122031 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688813+0000", 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015106399999999999 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.765112+0000", 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076299661000000005 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106912+0000", 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34180015600000002 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106973+0000", 2024-09-19T05:44:17.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0927999999999997e-05 2024-09-19T05:44:17.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.107063+0000", 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0137999999999997e-05 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:288 3.31 e54/20 3:8d4d39bf:::10000000002.0000001d:head v 54'5, mlcod=53'3)", 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:17.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.834397188000001, 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48371873900000001, 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697920+0000", 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7719999999999999e-06 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099856+0000", 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40193617399999998 2024-09-19T05:44:17.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099878+0000", 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2306999999999998e-05 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099891+0000", 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132365+0000", 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032473872000000001 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132403+0000", 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8396999999999998e-05 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181593+0000", 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049190325 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181637+0000", 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3368000000000002e-05 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:337 3.34 3:2c8ffcca:::10000000002.0000004e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.754320489000001, 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24486184699999999, 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 337 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:17.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777997+0000", 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.17e-06 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781851+0000", 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038539429999999999 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781870+0000", 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9012e-05 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781882+0000", 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1864e-05 2024-09-19T05:44:17.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781987+0000", 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105691 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782084+0000", 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7072999999999997e-05 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782457+0000", 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037225700000000002 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799269+0000", 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016812247999999998 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022778+0000", 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22350931499999999 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022823+0000", 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4502999999999997e-05 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022856+0000", 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3770999999999999e-05 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:357 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.692235492999998, 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33810015799999998, 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 357 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840082+0000", 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2029999999999999e-06 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842172+0000", 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020904890000000001 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842193+0000", 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1058999999999999e-05 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842207+0000", 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3444e-05 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848376+0000", 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061690909999999998 2024-09-19T05:44:17.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848486+0000", 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109979 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848644+0000", 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015821699999999999 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.894080+0000", 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045435709999999997 2024-09-19T05:44:17.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178104+0000", 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28402411700000002 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178148+0000", 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4492e-05 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178180+0000", 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1356999999999998e-05 2024-09-19T05:44:17.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:363 3.3f 3:ff5df6dc:::10000000002.00000068:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.681934139999999, 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32356367800000002, 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 363 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:17.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850383+0000", 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0600000000000002e-06 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862283+0000", 2024-09-19T05:44:17.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011899873 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862330+0000", 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862352+0000", 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1141e-05 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.877929+0000", 2024-09-19T05:44:17.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015577153 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878045+0000", 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011612000000000001 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878204+0000", 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158832 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896926+0000", 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018722714000000001 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173814+0000", 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27688780099999999 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173871+0000", 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6907000000000003e-05 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173945+0000", 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3530999999999998e-05 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:365 3.c 3:3214729d:::10000000002.0000006a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.625 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.643913739999999, 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27874868600000002, 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 365 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:17.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888405+0000", 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7620000000000001e-06 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892569+0000", 2024-09-19T05:44:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041637540000000004 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892661+0000", 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2134999999999998e-05 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892675+0000", 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4173e-05 2024-09-19T05:44:17.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908596+0000", 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015920815000000001 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908712+0000", 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011617599999999999 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908878+0000", 2024-09-19T05:44:17.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165935 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972488+0000", 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063609938000000005 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167039+0000", 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19455101699999999 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167097+0000", 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7776999999999999e-05 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167150+0000", 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3204000000000001e-05 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:372 3.34 3:2ebe227a:::10000000002.00000071:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.630210836, 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.264918877, 2024-09-19T05:44:17.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 372 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902109+0000", 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6990000000000002e-06 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910547+0000", 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084382120000000005 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910566+0000", 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9111e-05 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910579+0000", 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2330999999999999e-05 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923326+0000", 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012747596 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923501+0000", 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000174802 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923775+0000", 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027407199999999999 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023005+0000", 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099229554999999997 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.157725+0000", 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13472034199999999 2024-09-19T05:44:17.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.158104+0000", 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037887499999999998 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167023+0000", 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089192820000000006 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.637 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:315 3.1c e54/38 3:394f8a26:::10000000002.00000038:head v 54'8, mlcod=53'5)", 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.626790431, 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28226446599999999, 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905527+0000", 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5739999999999998e-06 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939034+0000", 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033507185000000002 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939067+0000", 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2186e-05 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939084+0000", 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6982000000000001e-05 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972466+0000", 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033381985000000003 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972491+0000", 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5789e-05 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187219+0000", 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21472724700000001 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187789+0000", 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000570518 2024-09-19T05:44:17.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:373 3.36 3:6d44e697:::10000000002.00000072:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.621707273999998, 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29574129399999999, 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 373 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910611+0000", 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5779999999999999e-06 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.919588+0000", 2024-09-19T05:44:17.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089768220000000006 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938096+0000", 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018507920000000001 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938107+0000", 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0488e-05 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944322+0000", 2024-09-19T05:44:17.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062149900000000001 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944546+0000", 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022402300000000001 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944884+0000", 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033806299999999999 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.074757+0000", 2024-09-19T05:44:17.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.129873715 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206239+0000", 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13148132000000001 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206302+0000", 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3046000000000006e-05 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206349+0000", 2024-09-19T05:44:17.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7329e-05 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:348 3.1c e54/38 3:3aa14ed1:::10000000002.00000059:head v 54'9, mlcod=53'5)", 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.587644012999998, 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24318706500000001, 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944672+0000", 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1790000000000001e-06 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947488+0000", 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028159130000000002 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947504+0000", 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5367e-05 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947511+0000", 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3300000000000001e-06 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006421+0000", 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058910000999999997 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006451+0000", 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0576e-05 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187796+0000", 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18134434799999999 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187858+0000", 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2351e-05 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:395 3.1c e54/38 3:384ad1d9:::10000000002.00000088:head v 54'10, mlcod=53'5)", 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.584787124999998, 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24038785900000001, 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:17.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947529+0000", 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7400000000000002e-07 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959658+0000", 2024-09-19T05:44:17.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012129493999999999 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959675+0000", 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6996000000000001e-05 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959687+0000", 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2143999999999999e-05 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023041+0000", 2024-09-19T05:44:17.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063353484000000002 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023062+0000", 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1171000000000001e-05 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187862+0000", 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.164799521 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187916+0000", 2024-09-19T05:44:17.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4175000000000002e-05 2024-09-19T05:44:17.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.657 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:393 3.36 3:6ccb00e9:::10000000002.00000086:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.545322267, 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24664645599999999, 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:17.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 393 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:17.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986998+0000", 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9899999999999997e-06 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995844+0000", 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0088464340000000002 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995859+0000", 2024-09-19T05:44:17.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4329e-05 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995866+0000", 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037368+0000", 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041501563999999998 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037440+0000", 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1773999999999994e-05 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037555+0000", 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011548399999999999 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117549+0000", 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079993623 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233527+0000", 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11597800699999999 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233591+0000", 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3789999999999997e-05 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233639+0000", 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8606999999999997e-05 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.663 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:17.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:17.664 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:17.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:17.666 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:17.664+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:17.669 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:17.669 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 readers = 8942921.00 kB/sec 2024-09-19T05:44:17.669 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 readers = 8925715.73 kB/sec 2024-09-19T05:44:17.669 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 8942921.00 kB/sec 2024-09-19T05:44:17.669 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 8942921.00 kB/sec 2024-09-19T05:44:17.669 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 8942921.00 kB/sec 2024-09-19T05:44:17.669 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:44:17.765 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:17.765+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:17.776 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:17.776 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:17.776 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:17.776 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:17.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:17.867 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:17.865+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:17.932 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:17.932 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:17.932 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:17.933 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:17.933 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:17.941 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:17.986 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:17.985+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:18.051 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:18.051 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:18.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:18.051 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:18.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:44:18.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 115.168966756, 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:44:18.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:44:18.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:44:18.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:7 3.2a 3:557f550e:::10000000002.00000004:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.765518450999998, 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27494626900000002, 2024-09-19T05:44:18.059 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 7 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:18.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282495+0000", 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4609999999999998e-06 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286920+0000", 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044248229999999996 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286972+0000", 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1659000000000001e-05 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286986+0000", 2024-09-19T05:44:18.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287117+0000", 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131328 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287358+0000", 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024141700000000001 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287543+0000", 2024-09-19T05:44:18.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018436700000000001 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299025+0000", 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011482327000000001 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557305+0000", 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25828001099999998 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557360+0000", 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5074000000000002e-05 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557439+0000", 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9018999999999997e-05 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:92 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.632370432999998, 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37051118500000002, 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415644+0000", 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.422983+0000", 2024-09-19T05:44:18.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073390779999999997 2024-09-19T05:44:18.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423020+0000", 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7660000000000002e-05 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423036+0000", 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5359999999999999e-05 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423261+0000", 2024-09-19T05:44:18.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225217 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423458+0000", 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196555 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423737+0000", 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027922100000000002 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.435447+0000", 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011709855999999999 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.785961+0000", 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35051446400000003 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786086+0000", 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012537599999999999 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786152+0000", 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691000000000005e-05 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.072 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:104 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.618144110999999, 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28556600799999998, 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429870+0000", 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-06 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446619+0000", 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016749447000000001 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446650+0000", 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0218000000000001e-05 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446666+0000", 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6920999999999999e-05 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448857+0000", 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021902699999999998 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449011+0000", 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015441700000000001 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449168+0000", 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156527 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489828+0000", 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040659908000000002 2024-09-19T05:44:18.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715285+0000", 2024-09-19T05:44:18.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22545759500000001 2024-09-19T05:44:18.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715345+0000", 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0269999999999997e-05 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715433+0000", 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7814999999999999e-05 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:108 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.601293093999999, 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26890963299999998, 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.080 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446721+0000", 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8250000000000001e-06 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451943+0000", 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052222140000000002 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451961+0000", 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8117e-05 2024-09-19T05:44:18.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451970+0000", 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7420000000000003e-06 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473034+0000", 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021064282 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473242+0000", 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020716699999999999 2024-09-19T05:44:18.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473446+0000", 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204396 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489832+0000", 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016385994000000001 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715506+0000", 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22567427700000001 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715577+0000", 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0548000000000003e-05 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715628+0000", 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070999999999999e-05 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.085 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:139 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.584481688, 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.328123891, 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.086 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463532+0000", 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0339999999999999e-06 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468331+0000", 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047995099999999999 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468365+0000", 2024-09-19T05:44:18.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3667000000000001e-05 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468382+0000", 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6651e-05 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468915+0000", 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053293299999999997 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469199+0000", 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028406900000000002 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469482+0000", 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028390000000000002 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.485026+0000", 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015543509 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791538+0000", 2024-09-19T05:44:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30651197400000002 2024-09-19T05:44:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791602+0000", 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3780000000000003e-05 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791654+0000", 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1863999999999997e-05 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:148 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.566031654, 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34695000399999998, 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 148 2024-09-19T05:44:18.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481982+0000", 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6309999999999999e-06 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489765+0000", 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077821310000000003 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489778+0000", 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3604999999999999e-05 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489787+0000", 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489875+0000", 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8039000000000002e-05 2024-09-19T05:44:18.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490101+0000", 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022636299999999999 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490426+0000", 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032453799999999997 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.506325+0000", 2024-09-19T05:44:18.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015899149000000001 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828797+0000", 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32247239900000002 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828865+0000", 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7521999999999995e-05 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828930+0000", 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4795999999999994e-05 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:156 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.558176447000001, 2024-09-19T05:44:18.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26935146300000001, 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:18.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489837+0000", 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2689999999999998e-06 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495152+0000", 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053150209999999996 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495356+0000", 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203537 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495373+0000", 2024-09-19T05:44:18.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496071+0000", 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069810200000000003 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496246+0000", 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017547299999999999 2024-09-19T05:44:18.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496469+0000", 2024-09-19T05:44:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022283200000000001 2024-09-19T05:44:18.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.540840+0000", 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044371318999999999 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759082+0000", 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21824185500000001 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759137+0000", 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645000000000003e-05 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759186+0000", 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9499000000000002e-05 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:185 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:18.106 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.545496603, 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32688410400000001, 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 185 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:18.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502518+0000", 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.509973+0000", 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074553919999999999 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510031+0000", 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8356999999999997e-05 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510046+0000", 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932e-05 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510135+0000", 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8548000000000007e-05 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510886+0000", 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075159700000000001 2024-09-19T05:44:18.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511079+0000", 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019261500000000001 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532794+0000", 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021714989000000001 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829312+0000", 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29651747699999997 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829360+0000", 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8396000000000002e-05 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829399+0000", 2024-09-19T05:44:18.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8961999999999997e-05 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:264 3.1c 3:3ae0713d:::10000000002.00000005:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.455577893000001, 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29060550499999999, 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 264 2024-09-19T05:44:18.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592436+0000", 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5629999999999999e-06 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595027+0000", 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025912600000000002 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595055+0000", 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8164999999999999e-05 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595069+0000", 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3953e-05 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595156+0000", 2024-09-19T05:44:18.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6981e-05 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595260+0000", 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010363 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595417+0000", 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015732100000000001 2024-09-19T05:44:18.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.605722+0000", 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010305099999999999 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882929+0000", 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27720663499999998 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882988+0000", 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9237000000000002e-05 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.883039+0000", 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0659999999999999e-05 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:315 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.416617573, 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55644120600000002, 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 315 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631396+0000", 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5490000000000001e-06 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636257+0000", 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048607859999999998 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636331+0000", 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3992000000000006e-05 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636355+0000", 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3813999999999999e-05 2024-09-19T05:44:18.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636445+0000", 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0180999999999996e-05 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636569+0000", 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123991 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636849+0000", 2024-09-19T05:44:18.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000279664 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659414+0000", 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022565053000000002 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187715+0000", 2024-09-19T05:44:18.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52830067000000003 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187783+0000", 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8509000000000004e-05 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187835+0000", 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1996999999999997e-05 2024-09-19T05:44:18.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:347 3.2a 3:5747858d:::10000000002.00000058:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.127 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.408544747000001, 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48789592399999998, 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 347 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639470+0000", 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.145e-06 2024-09-19T05:44:18.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644022+0000", 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045525940000000001 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646323+0000", 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023007879999999998 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646342+0000", 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8766999999999998e-05 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646435+0000", 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3394999999999996e-05 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646564+0000", 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012848800000000001 2024-09-19T05:44:18.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646738+0000", 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017418499999999999 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.741685+0000", 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094946825999999998 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127231+0000", 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.385546526 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127301+0000", 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9741999999999997e-05 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127363+0000", 2024-09-19T05:44:18.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1468000000000002e-05 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:348 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.401642256999999, 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54159870300000001, 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 348 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646372+0000", 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9110000000000002e-06 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.678772+0000", 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032399869999999997 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681572+0000", 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002799714 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681592+0000", 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0298000000000001e-05 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682777+0000", 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011847190000000001 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682909+0000", 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000132146 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.683102+0000", 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019280499999999999 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.702631+0000", 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019529271000000001 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187923+0000", 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48529252 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187948+0000", 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5004999999999999e-05 2024-09-19T05:44:18.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187968+0000", 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9443999999999999e-05 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:360 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.140 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.358265842000002, 2024-09-19T05:44:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43778335400000001, 2024-09-19T05:44:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 360 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:18.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689748+0000", 2024-09-19T05:44:18.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7690000000000001e-06 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692563+0000", 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028147329999999998 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692593+0000", 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9476999999999999e-05 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692616+0000", 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3340999999999999e-05 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723279+0000", 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030662759000000001 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723430+0000", 2024-09-19T05:44:18.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151247 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723629+0000", 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199426 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793213+0000", 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069583435999999999 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127443+0000", 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33423067000000001 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127491+0000", 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8000999999999997e-05 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127529+0000", 2024-09-19T05:44:18.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7494999999999999e-05 2024-09-19T05:44:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.148 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:364 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.355362124999999, 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43506039200000002, 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 364 2024-09-19T05:44:18.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692652+0000", 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9280000000000002e-06 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697100+0000", 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004447678 2024-09-19T05:44:18.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701619+0000", 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045192829999999998 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701638+0000", 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8365000000000001e-05 2024-09-19T05:44:18.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742561+0000", 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040923560999999997 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742686+0000", 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012482899999999999 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742839+0000", 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015348 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808887+0000", 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066047260999999996 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127632+0000", 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31874497099999999 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127672+0000", 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0244999999999998e-05 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127710+0000", 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7790999999999998e-05 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:395 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.342620280999999, 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48275294299999999, 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 395 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.156 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705394+0000", 2024-09-19T05:44:18.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7800000000000001e-06 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720169+0000", 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014775346 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720185+0000", 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5721000000000001e-05 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720194+0000", 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4609999999999997e-06 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720294+0000", 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100005 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720392+0000", 2024-09-19T05:44:18.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7547999999999995e-05 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720531+0000", 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000139099 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.767968+0000", 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047436846999999997 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188062+0000", 2024-09-19T05:44:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42009412899999998 2024-09-19T05:44:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188107+0000", 2024-09-19T05:44:18.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5269000000000003e-05 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188144+0000", 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6738e-05 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:404 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.165 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.327183115, 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46750329699999998, 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 404 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720832+0000", 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8469999999999998e-06 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726318+0000", 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005485775 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726490+0000", 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172339 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726505+0000", 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4467999999999999e-05 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743219+0000", 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016714046 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743320+0000", 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101548 2024-09-19T05:44:18.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743459+0000", 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013899499999999999 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.775025+0000", 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031565658000000003 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188247+0000", 2024-09-19T05:44:18.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.413221583 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188297+0000", 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0586e-05 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188332+0000", 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4452000000000002e-05 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:412 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.309564310999999, 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40023217100000003, 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 412 2024-09-19T05:44:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:18.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738450+0000", 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7e-06 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.758682+0000", 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020232357999999999 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772625+0000", 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013943232 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.774996+0000", 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023703890000000001 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793240+0000", 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018244646999999999 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793347+0000", 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106276 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793511+0000", 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016406099999999999 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808895+0000", 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015384296 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138570+0000", 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32967460500000001 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138630+0000", 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0087999999999998e-05 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138679+0000", 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9518999999999998e-05 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:441 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.253855763000001, 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39435722000000001, 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 441 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794159+0000", 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.579e-06 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.802843+0000", 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0086840579999999997 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.807986+0000", 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051424890000000001 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808000+0000", 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808113+0000", 2024-09-19T05:44:18.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011294900000000001 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808200+0000", 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6792999999999994e-05 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808612+0000", 2024-09-19T05:44:18.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041192799999999999 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827676+0000", 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019063875000000001 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188432+0000", 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36075675699999998 2024-09-19T05:44:18.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188479+0000", 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6149000000000001e-05 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188513+0000", 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4310999999999997e-05 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:454 3.37 3:ec162ecd:::10000000002.000000c3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.215092291000001, 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32323945999999998, 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 454 2024-09-19T05:44:18.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832922+0000", 2024-09-19T05:44:18.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6280000000000001e-06 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839384+0000", 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064625180000000004 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850436+0000", 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011051365 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850464+0000", 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8719000000000001e-05 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850532+0000", 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7212000000000006e-05 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:18.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850663+0000", 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013160799999999999 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850877+0000", 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000213716 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871083+0000", 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020205864 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156086+0000", 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.285003532 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156125+0000", 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8569999999999998e-05 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156159+0000", 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3728e-05 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.192 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.192 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:18.192 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:44:18.192 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:44:18.192 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-19T05:44:18.193 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-19T05:44:18.194 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:18.195 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:18.192+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:18.294 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:18.293+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.337+0000 7f1a865b7640 1 Processor -- start 2024-09-19T05:44:18.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.337+0000 7f1a865b7640 1 -- start start 2024-09-19T05:44:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a865b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a800c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a865b7640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a800c74c0 0x7f1a800cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a865b7640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a800c4e30 con 0x7f1a80151d70 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a865b7640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1a800c8350 con 0x7f1a800c74c0 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a865b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1a801577f0 con 0x7f1a80151990 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a800c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a84b54640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a800c74c0 0x7f1a800cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a84b54640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a800c74c0 0x7f1a800cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:59024/0 (socket says 172.21.15.2:59024) 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a84b54640 1 -- 172.21.15.2:0/2545909842 learned_addr learned my addr 172.21.15.2:0/2545909842 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a800c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:47710/0 (socket says 172.21.15.2:47710) 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a77fff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1a80151d70 0x7f1a800c87e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:59328/0 (socket says 172.21.15.2:59328) 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a7ffff640 1 -- 172.21.15.2:0/2545909842 >> v2:172.21.15.123:3300/0 conn(0x7f1a800c74c0 msgr2=0x7f1a800cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a7ffff640 1 --2- 172.21.15.2:0/2545909842 >> v2:172.21.15.123:3300/0 conn(0x7f1a800c74c0 0x7f1a800cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a7ffff640 1 -- 172.21.15.2:0/2545909842 >> v1:172.21.15.123:6789/0 conn(0x7f1a80151d70 legacy=0x7f1a800c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a7ffff640 1 -- 172.21.15.2:0/2545909842 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80070960 con 0x7f1a80151990 2024-09-19T05:44:18.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.338+0000 7f1a84b54640 1 --2- 172.21.15.2:0/2545909842 >> v2:172.21.15.123:3300/0 conn(0x7f1a800c74c0 0x7f1a800cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.339+0000 7f1a7ffff640 1 --2- 172.21.15.2:0/2545909842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a800c47f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a700097a0 tx=0x7f1a7002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f86c25db26abb93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.339+0000 7f1a7f7fe640 1 -- 172.21.15.2:0/2545909842 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70037070 con 0x7f1a80151990 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.339+0000 7f1a7f7fe640 1 -- 172.21.15.2:0/2545909842 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a70032440 con 0x7f1a80151990 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.339+0000 7f1a7f7fe640 1 -- 172.21.15.2:0/2545909842 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70032810 con 0x7f1a80151990 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 -- 172.21.15.2:0/2545909842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 msgr2=0x7f1a800c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2545909842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a800c47f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a700097a0 tx=0x7f1a7002a180 comp rx=0 tx=0).stop 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 -- 172.21.15.2:0/2545909842 shutdown_connections 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2545909842 >> v2:172.21.15.123:3300/0 conn(0x7f1a800c74c0 0x7f1a800cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2545909842 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a800c47f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 -- 172.21.15.2:0/2545909842 >> 172.21.15.2:0/2545909842 conn(0x7f1a80072a60 msgr2=0x7f1a80072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 -- 172.21.15.2:0/2545909842 shutdown_connections 2024-09-19T05:44:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.340+0000 7f1a865b7640 1 -- 172.21.15.2:0/2545909842 wait complete. 2024-09-19T05:44:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.341+0000 7f1a865b7640 1 Processor -- start 2024-09-19T05:44:18.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.341+0000 7f1a865b7640 1 -- start start 2024-09-19T05:44:18.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a865b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a800c74c0 0x7f1a801ddba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a865b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a801f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a865b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800cc450 0x7f1a801e3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a865b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80071200 con 0x7f1a800c74c0 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a865b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80070f20 con 0x7f1a800cc450 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a865b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8006fb80 con 0x7f1a80151990 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a800c74c0 0x7f1a801ddba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a7ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a800c74c0 0x7f1a801ddba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33300/0 (socket says 172.21.15.2:33300) 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a7ffff640 1 -- 172.21.15.2:0/2479654022 learned_addr learned my addr 172.21.15.2:0/2479654022 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a84b54640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800cc450 0x7f1a801e3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.342+0000 7f1a77fff640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a801f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7ffff640 1 -- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 msgr2=0x7f1a801f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7ffff640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a801f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7ffff640 1 -- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800cc450 msgr2=0x7f1a801e3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7ffff640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800cc450 0x7f1a801e3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7ffff640 1 -- 172.21.15.2:0/2479654022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a801f1360 con 0x7f1a800c74c0 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7ffff640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a800c74c0 0x7f1a801ddba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a7002a1c0 tx=0x7f1a700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7d7fa640 1 -- 172.21.15.2:0/2479654022 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70037040 con 0x7f1a800c74c0 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7d7fa640 1 -- 172.21.15.2:0/2479654022 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a70042070 con 0x7f1a800c74c0 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a801f22a0 con 0x7f1a800c74c0 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a7d7fa640 1 -- 172.21.15.2:0/2479654022 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70030920 con 0x7f1a800c74c0 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.343+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a801de0e0 con 0x7f1a800c74c0 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:18.347 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:18.348 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:18.348 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:18.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.344+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a800c7bf0 con 0x7f1a800c74c0 2024-09-19T05:44:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.345+0000 7f1a7d7fa640 1 -- 172.21.15.2:0/2479654022 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1a70030ac0 con 0x7f1a800c74c0 2024-09-19T05:44:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.345+0000 7f1a7d7fa640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1a6006c2b0 0x7f1a6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.345+0000 7f1a7d7fa640 1 -- 172.21.15.2:0/2479654022 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5770+0+0 (secure 0 0 0) 0x7f1a700abc80 con 0x7f1a800c74c0 2024-09-19T05:44:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.345+0000 7f1a77fff640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1a6006c2b0 0x7f1a6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.346+0000 7f1a77fff640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1a6006c2b0 0x7f1a6006e770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a80072eb0 tx=0x7f1a68007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.349+0000 7f1a7d7fa640 1 -- 172.21.15.2:0/2479654022 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1a70079df0 con 0x7f1a800c74c0 2024-09-19T05:44:18.355 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:18.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:18.397 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:18.395+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:18.496 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:18.495+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:18.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.497+0000 7f779d68d640 1 Processor -- start 2024-09-19T05:44:18.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.497+0000 7f779d68d640 1 -- start start 2024-09-19T05:44:18.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f779d68d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f77980cd4e0 0x7f77980b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f779d68d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f779815ebe0 con 0x7f779815aa90 2024-09-19T05:44:18.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f779d68d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77980b2e00 con 0x7f77980cd8c0 2024-09-19T05:44:18.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f779d68d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f779815eec0 con 0x7f77980cd4e0 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f7796ffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f77980cd4e0 0x7f77980b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f77967fc640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f779815aa90 0x7f779815df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:59862/0 (socket says 172.21.15.2:59862) 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f77967fc640 1 -- 172.21.15.2:0/3624869125 learned_addr learned my addr 172.21.15.2:0/3624869125 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3624869125 >> v2:172.21.15.2:3300/0 conn(0x7f77980cd4e0 0x7f77980b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33314/0 (socket says 172.21.15.2:33314) 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f7796ffd640 1 -- 172.21.15.2:0/3624869125 >> v1:172.21.15.123:6789/0 conn(0x7f77980cd8c0 legacy=0x7f77980c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f7796ffd640 1 -- 172.21.15.2:0/3624869125 >> v1:172.21.15.2:6789/0 conn(0x7f779815aa90 legacy=0x7f779815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f7796ffd640 1 -- 172.21.15.2:0/3624869125 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77980b30e0 con 0x7f77980cd4e0 2024-09-19T05:44:18.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.498+0000 7f7795ffb640 1 -- 172.21.15.2:0/3624869125 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 25053075 0 0) 0x7f779815ebe0 con 0x7f779815aa90 2024-09-19T05:44:18.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.499+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3624869125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd4e0 0x7f77980b3bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f77880097a0 tx=0x7f778802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760b7601fcb4e3e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.499+0000 7f7795ffb640 1 -- 172.21.15.2:0/3624869125 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788037070 con 0x7f77980cd4e0 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.499+0000 7f7795ffb640 1 -- 172.21.15.2:0/3624869125 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7788032440 con 0x7f77980cd4e0 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.499+0000 7f7795ffb640 1 -- 172.21.15.2:0/3624869125 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788032810 con 0x7f77980cd4e0 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.500+0000 7f779d68d640 1 -- 172.21.15.2:0/3624869125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd4e0 msgr2=0x7f77980b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.500+0000 7f779d68d640 1 --2- 172.21.15.2:0/3624869125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd4e0 0x7f77980b3bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f77880097a0 tx=0x7f778802a230 comp rx=0 tx=0).stop 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.500+0000 7f779d68d640 1 -- 172.21.15.2:0/3624869125 shutdown_connections 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.500+0000 7f779d68d640 1 --2- 172.21.15.2:0/3624869125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd4e0 0x7f77980b3bc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.500+0000 7f779d68d640 1 -- 172.21.15.2:0/3624869125 >> 172.21.15.2:0/3624869125 conn(0x7f7798067b30 msgr2=0x7f7798067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.500+0000 7f779d68d640 1 -- 172.21.15.2:0/3624869125 shutdown_connections 2024-09-19T05:44:18.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.500+0000 7f779d68d640 1 -- 172.21.15.2:0/3624869125 wait complete. 2024-09-19T05:44:18.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.501+0000 7f779d68d640 1 Processor -- start 2024-09-19T05:44:18.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.501+0000 7f779d68d640 1 -- start start 2024-09-19T05:44:18.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f779d68d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77980cd4e0 0x7f7798150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f779d68d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd8c0 0x7f7798153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f779d68d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779815aa90 0x7f779814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f779d68d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f779815e640 con 0x7f77980cd8c0 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f779d68d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f77980b36a0 con 0x7f779815aa90 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f779d68d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f779815e7c0 con 0x7f77980cd4e0 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77967fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd8c0 0x7f7798153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77967fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd8c0 0x7f7798153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33316/0 (socket says 172.21.15.2:33316) 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77967fc640 1 -- 172.21.15.2:0/3911031410 learned_addr learned my addr 172.21.15.2:0/3911031410 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77980cd4e0 0x7f7798150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77977fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779815aa90 0x7f779814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77967fc640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77980cd4e0 msgr2=0x7f7798150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77967fc640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f77980cd4e0 0x7f7798150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77967fc640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779815aa90 msgr2=0x7f779814d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.502+0000 7f77967fc640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f779815aa90 0x7f779814d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.503+0000 7f77967fc640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77981516b0 con 0x7f77980cd8c0 2024-09-19T05:44:18.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.503+0000 7f77967fc640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd8c0 0x7f7798153000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c420 tx=0x7f778c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.503+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0074e0 con 0x7f77980cd8c0 2024-09-19T05:44:18.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.503+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778c0047e0 con 0x7f77980cd8c0 2024-09-19T05:44:18.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.503+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c004c10 con 0x7f77980cd8c0 2024-09-19T05:44:18.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.503+0000 7f779d68d640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77981f4370 con 0x7f77980cd8c0 2024-09-19T05:44:18.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.503+0000 7f779d68d640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779815bf30 con 0x7f77980cd8c0 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.504+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7770005270 con 0x7f77980cd8c0 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.505+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f778c011070 con 0x7f77980cd8c0 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f777f7fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f778406c2b0 0x7f778406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5770+0+0 (secure 0 0 0) 0x7f778c08ae30 con 0x7f77980cd8c0 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f778406c2b0 0x7f778406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:18.508 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:18.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v64) ==== 54+0+16 (secure 0 0 0) 0x7f778c08d4b0 con 0x7f77980cd8c0 2024-09-19T05:44:18.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7770000fa0 0x7f77700045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7770005550 con 0x7f7770000fa0 2024-09-19T05:44:18.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f778406c2b0 0x7f778406e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7788034000 tx=0x7f778802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.506+0000 7f77977fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7770000fa0 0x7f77700045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.507+0000 7f77977fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7770000fa0 0x7f77700045e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.508+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7770005550 con 0x7f7770000fa0 2024-09-19T05:44:18.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:18.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.523+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7770004fe0 con 0x7f7770000fa0 2024-09-19T05:44:18.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.526+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7770004fe0 con 0x7f7770000fa0 2024-09-19T05:44:18.527 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.526+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7770001d00 0x7f7770002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.526+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f777000b390 con 0x7f7770001d00 2024-09-19T05:44:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.526+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7770001d00 0x7f7770002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.527+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7770001d00 0x7f7770002990 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.527+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f777000b390 con 0x7f7770001d00 2024-09-19T05:44:18.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.531+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f1a800c4820 con 0x7f1a800c74c0 2024-09-19T05:44:18.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.533+0000 7f1a7d7fa640 1 -- 172.21.15.2:0/2479654022 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v64) ==== 81+0+34615 (secure 0 0 0) 0x7f1a7007e0f0 con 0x7f1a800c74c0 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 64, 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "896e162c-4658-4743-b70e-7ec2b5bce950", 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-19T05:42:09.734568+0000", 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-19T05:44:16.752754+0000", 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-19T05:42:20.842106+0000", 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-19T05:43:19.394170+0000", 2024-09-19T05:44:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5806080, 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "nodeep-scrub", 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 6, 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-19T05:44:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "squid", 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:21.863894+0000", 2024-09-19T05:44:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-19T05:44:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "18", 2024-09-19T05:44:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:44:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:44:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-19T05:44:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 6.9000000953674316, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 6.9000000953674316, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.28999999165534973, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-19T05:44:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_metadata", 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:30.912746+0000", 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:44:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-19T05:44:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "21", 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:44:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:44:18.547 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:44:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_bias": 4, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 16, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_priority": 5 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": "cephfs" 2024-09-19T05:44:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.9700000286102295, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.9700000286102295, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1.1399999856948853, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2.25, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2.25, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "cephfs_data", 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-19T05:42:31.312255+0000", 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-19T05:44:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "is_stretch_pool": false, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 64, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 64, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 64, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 64, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 64, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-19T05:44:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "22", 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-19T05:44:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-19T05:44:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-19T05:44:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 64, 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "target_size_ratio": 0.89999997615814209 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs": { 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "data": "cephfs" 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "score_type": "Fair distribution", 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.5399999618530273, 2024-09-19T05:44:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.5399999618530273, 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1.1399999856948853, 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "665c2ac5-a964-4f38-8fa6-f4e95cac9272", 2024-09-19T05:44:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 58, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-19T05:44:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 288510134 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/288510134", 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/288510134", 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/288510134", 2024-09-19T05:44:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/288510134", 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "a4443f63-2c47-4c6f-a2df-6765b78afcc2", 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 0, 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 41, 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2074594312 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2074594312", 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2074594312", 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2074594312", 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2074594312", 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "61465313-1ba1-4f26-b488-6ea73981419b", 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 15, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 40, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2644191985 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/2644191985", 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/2644191985", 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/2644191985", 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/2644191985", 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "fa9d27d5-68d4-4467-a2a5-79f767a071aa", 2024-09-19T05:44:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 58, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3985782072 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/3985782072", 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/3985782072", 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/3985782072", 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/3985782072", 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "85a040ff-ad8b-44ff-b82e-a36b05d83087", 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 58, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6816", 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6817", 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6818", 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6819", 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6822", 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6823", 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6820", 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6821", 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2654957182 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6817/2654957182", 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6819/2654957182", 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6823/2654957182", 2024-09-19T05:44:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6821/2654957182", 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "f90f8fa4-4afc-4272-89eb-5e90123155d9", 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 58, 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6800", 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6801", 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6802", 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6803", 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6806", 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6807", 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6804", 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6805", 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3690741328 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6801/3690741328", 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6803/3690741328", 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6807/3690741328", 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6805/3690741328", 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "48561580-ba9a-4096-9722-16c308f414ee", 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 20, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6824", 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6825", 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6826", 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6827", 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6830", 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6831", 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6828", 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6829", 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 593730548 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6825/593730548", 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6827/593730548", 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6831/593730548", 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6829/593730548", 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "d1ee7357-25a5-432c-b96a-e808139a1cf3", 2024-09-19T05:44:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 14, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 62, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6808", 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6809", 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6810", 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6811", 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6814", 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6815", 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6812", 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.123:6813", 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1657801827 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.123:6809/1657801827", 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.123:6811/1657801827", 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.123:6815/1657801827", 2024-09-19T05:44:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.123:6813/1657801827", 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.148620+0000", 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.921396+0000", 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.980898+0000", 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.184322+0000", 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.139258+0000", 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:17.479047+0000", 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.432111+0000", 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-19T05:44:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540701547738038271, 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-19T05:42:18.402960+0000", 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [ 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.f", 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 0, 2024-09-19T05:44:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 7 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-19T05:44:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-19T05:44:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-19T05:44:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-19T05:44:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:18.612 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.535+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1a6006c2b0 msgr2=0x7f1a6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.535+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1a6006c2b0 0x7f1a6006e770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a80072eb0 tx=0x7f1a68007e80 comp rx=0 tx=0).stop 2024-09-19T05:44:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.535+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a800c74c0 msgr2=0x7f1a801ddba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.535+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a800c74c0 0x7f1a801ddba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1a7002a1c0 tx=0x7f1a700046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 shutdown_connections 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1a6006c2b0 0x7f1a6006e770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a800cc450 0x7f1a801e3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1a80151990 0x7f1a801f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 --2- 172.21.15.2:0/2479654022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1a800c74c0 0x7f1a801ddba0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 >> 172.21.15.2:0/2479654022 conn(0x7f1a80072a60 msgr2=0x7f1a800cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 shutdown_connections 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.536+0000 7f1a865b7640 1 -- 172.21.15.2:0/2479654022 wait complete. 2024-09-19T05:44:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.543+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f77700031a0 con 0x7f7770001d00 2024-09-19T05:44:18.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.545+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f77700031a0 con 0x7f7770001d00 2024-09-19T05:44:18.615 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.545+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7770003c20 0x7f7770009c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.546+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f777000a3d0 con 0x7f7770003c20 2024-09-19T05:44:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.546+0000 7f77967fc640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7770003c20 0x7f7770009c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.546+0000 7f77967fc640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7770003c20 0x7f7770009c80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.617 INFO:tasks.thrashosds.thrasher:j is {'epoch': 64, 'fsid': '896e162c-4658-4743-b70e-7ec2b5bce950', 'created': '2024-09-19T05:42:09.734568+0000', 'modified': '2024-09-19T05:44:16.752754+0000', 'last_up_change': '2024-09-19T05:42:20.842106+0000', 'last_in_change': '2024-09-19T05:43:19.394170+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T05:42:21.863894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T05:42:30.912746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9700000286102295, 'score_stable': 1.9700000286102295, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T05:42:31.312255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '665c2ac5-a964-4f38-8fa6-f4e95cac9272', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 288510134}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 288510134}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 288510134}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 288510134}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 288510134}]}, 'public_addr': '172.21.15.2:6809/288510134', 'cluster_addr': '172.21.15.2:6811/288510134', 'heartbeat_back_addr': '172.21.15.2:6815/288510134', 'heartbeat_front_addr': '172.21.15.2:6813/288510134', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a4443f63-2c47-4c6f-a2df-6765b78afcc2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2074594312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2074594312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2074594312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2074594312}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2074594312}]}, 'public_addr': '172.21.15.2:6817/2074594312', 'cluster_addr': '172.21.15.2:6819/2074594312', 'heartbeat_back_addr': '172.21.15.2:6823/2074594312', 'heartbeat_front_addr': '172.21.15.2:6821/2074594312', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '61465313-1ba1-4f26-b488-6ea73981419b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 2644191985}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 2644191985}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 2644191985}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 2644191985}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 2644191985}]}, 'public_addr': '172.21.15.2:6825/2644191985', 'cluster_addr': '172.21.15.2:6827/2644191985', 'heartbeat_back_addr': '172.21.15.2:6831/2644191985', 'heartbeat_front_addr': '172.21.15.2:6829/2644191985', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'fa9d27d5-68d4-4467-a2a5-79f767a071aa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 3985782072}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 3985782072}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 3985782072}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 3985782072}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 3985782072}]}, 'public_addr': '172.21.15.2:6801/3985782072', 'cluster_addr': '172.21.15.2:6803/3985782072', 'heartbeat_back_addr': '172.21.15.2:6807/3985782072', 'heartbeat_front_addr': '172.21.15.2:6805/3985782072', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '85a040ff-ad8b-44ff-b82e-a36b05d83087', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6816', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6817', 'nonce': 2654957182}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6818', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6819', 'nonce': 2654957182}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6822', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6823', 'nonce': 2654957182}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6820', 'nonce': 2654957182}, {'type': 'v1', 'addr': '172.21.15.123:6821', 'nonce': 2654957182}]}, 'public_addr': '172.21.15.123:6817/2654957182', 'cluster_addr': '172.21.15.123:6819/2654957182', 'heartbeat_back_addr': '172.21.15.123:6823/2654957182', 'heartbeat_front_addr': '172.21.15.123:6821/2654957182', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'f90f8fa4-4afc-4272-89eb-5e90123155d9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6800', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6801', 'nonce': 3690741328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6802', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6803', 'nonce': 3690741328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6806', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6807', 'nonce': 3690741328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6804', 'nonce': 3690741328}, {'type': 'v1', 'addr': '172.21.15.123:6805', 'nonce': 3690741328}]}, 'public_addr': '172.21.15.123:6801/3690741328', 'cluster_addr': '172.21.15.123:6803/3690741328', 'heartbeat_back_addr': '172.21.15.123:6807/3690741328', 'heartbeat_front_addr': '172.21.15.123:6805/3690741328', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '48561580-ba9a-4096-9722-16c308f414ee', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6824', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6825', 'nonce': 593730548}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6826', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6827', 'nonce': 593730548}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6830', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6831', 'nonce': 593730548}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6828', 'nonce': 593730548}, {'type': 'v1', 'addr': '172.21.15.123:6829', 'nonce': 593730548}]}, 'public_addr': '172.21.15.123:6825/593730548', 'cluster_addr': '172.21.15.123:6827/593730548', 'heartbeat_back_addr': '172.21.15.123:6831/593730548', 'heartbeat_front_addr': '172.21.15.123:6829/593730548', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'd1ee7357-25a5-432c-b96a-e808139a1cf3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6808', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6809', 'nonce': 1657801827}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6810', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6811', 'nonce': 1657801827}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6814', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6815', 'nonce': 1657801827}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6812', 'nonce': 1657801827}, {'type': 'v1', 'addr': '172.21.15.123:6813', 'nonce': 1657801827}]}, 'public_addr': '172.21.15.123:6809/1657801827', 'cluster_addr': '172.21.15.123:6811/1657801827', 'heartbeat_back_addr': '172.21.15.123:6815/1657801827', 'heartbeat_front_addr': '172.21.15.123:6813/1657801827', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.148620+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.921396+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.980898+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.184322+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.139258+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:17.479047+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.432111+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-19T05:42:18.402960+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 0, 'to': 7}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-19T05:44:18.618 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:44:18.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.547+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f777000a3d0 con 0x7f7770003c20 2024-09-19T05:44:18.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.561+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f777000a8e0 con 0x7f7770003c20 2024-09-19T05:44:18.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.562+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f777000a8e0 con 0x7f7770003c20 2024-09-19T05:44:18.619 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.562+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f777000d550 0x7f777000f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.562+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f7770012960 con 0x7f777000d550 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.563+0000 7f77977fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f777000d550 0x7f777000f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.563+0000 7f77977fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f777000d550 0x7f777000f940 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.564+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7770012960 con 0x7f777000d550 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.577+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7770010240 con 0x7f777000d550 2024-09-19T05:44:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.579+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7770010240 con 0x7f777000d550 2024-09-19T05:44:18.621 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:18.621 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.579+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f77700120f0 0x7f7770010450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.579+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f77700162c0 con 0x7f77700120f0 2024-09-19T05:44:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.580+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f77700120f0 0x7f7770010450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.580+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f77700120f0 0x7f7770010450 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.581+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77700162c0 con 0x7f77700120f0 2024-09-19T05:44:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.610+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7770010990 con 0x7f77700120f0 2024-09-19T05:44:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.612+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7770010990 con 0x7f77700120f0 2024-09-19T05:44:18.622 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:18.612+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:18.623 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:18.623 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:18.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.612+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f77700110b0 0x7f7770011530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.612+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f77700194d0 con 0x7f77700110b0 2024-09-19T05:44:18.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.613+0000 7f77967fc640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f77700110b0 0x7f7770011530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.613+0000 7f77967fc640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f77700110b0 0x7f7770011530 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.614+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77700194d0 con 0x7f77700110b0 2024-09-19T05:44:18.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.629+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7770016c40 con 0x7f77700110b0 2024-09-19T05:44:18.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.631+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7770016c40 con 0x7f77700110b0 2024-09-19T05:44:18.633 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:18.634 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:18.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.632+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f77700172c0 0x7f7770017700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.632+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f777001c8c0 con 0x7f77700172c0 2024-09-19T05:44:18.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.632+0000 7f77977fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f77700172c0 0x7f7770017700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.633+0000 7f77977fe640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f77700172c0 0x7f7770017700 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.633+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f777001c8c0 con 0x7f77700172c0 2024-09-19T05:44:18.646 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:18.646 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:18.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.647+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7770017d50 con 0x7f77700172c0 2024-09-19T05:44:18.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.649+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7770017d50 con 0x7f77700172c0 2024-09-19T05:44:18.650 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.649+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f77700183d0 0x7f77700187f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.649+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f7770018eb0 con 0x7f77700183d0 2024-09-19T05:44:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.650+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f77700183d0 0x7f77700187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.650+0000 7f7796ffd640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f77700183d0 0x7f77700187f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.651+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7770018eb0 con 0x7f77700183d0 2024-09-19T05:44:18.656 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:18.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.665+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7770000950 con 0x7f77700183d0 2024-09-19T05:44:18.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777f7fe640 1 -- 172.21.15.2:0/3911031410 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7770000950 con 0x7f77700183d0 2024-09-19T05:44:18.668 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7770000fa0 msgr2=0x7f77700045e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7770000fa0 0x7f77700045e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7770001d00 msgr2=0x7f7770002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7770001d00 0x7f7770002990 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7770003c20 msgr2=0x7f7770009c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7770003c20 0x7f7770009c80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f777000d550 msgr2=0x7f777000f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f77977fe640 1 -- 172.21.15.2:0/3911031410 reap_dead start 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f777000d550 0x7f777000f940 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f77700120f0 msgr2=0x7f7770010450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f77700120f0 0x7f7770010450 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f77700110b0 msgr2=0x7f7770011530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.667+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f77700110b0 0x7f7770011530 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f77700172c0 msgr2=0x7f7770017700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f77700172c0 0x7f7770017700 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f77700183d0 msgr2=0x7f77700187f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f77700183d0 0x7f77700187f0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f778406c2b0 msgr2=0x7f778406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f778406c2b0 0x7f778406e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7788034000 tx=0x7f778802e000 comp rx=0 tx=0).stop 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd8c0 msgr2=0x7f7798153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f77977fe640 1 -- 172.21.15.2:0/3911031410 reap_dead start 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 --2- 172.21.15.2:0/3911031410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77980cd8c0 0x7f7798153000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c420 tx=0x7f778c00c8e0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f77977fe640 1 -- 172.21.15.2:0/3911031410 reap_dead start 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 shutdown_connections 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 >> 172.21.15.2:0/3911031410 conn(0x7f7798067b30 msgr2=0x7f77980cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 shutdown_connections 2024-09-19T05:44:18.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.668+0000 7f777d7fa640 1 -- 172.21.15.2:0/3911031410 wait complete. 2024-09-19T05:44:18.714 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:18.712+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:18.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.733+0000 7f909c36d640 1 Processor -- start 2024-09-19T05:44:18.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.733+0000 7f909c36d640 1 -- start start 2024-09-19T05:44:18.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909c36d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9094072e40 0x7f90940722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909c36d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9094158960 con 0x7f909415caa0 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909c36d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90940c7b40 con 0x7f9094073220 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909c36d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f90940c7e20 con 0x7f9094072e40 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909a109640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9094072e40 0x7f90940722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909a90a640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f909415caa0 0x7f9094155940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:59864/0 (socket says 172.21.15.2:59864) 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909a90a640 1 -- 172.21.15.2:0/2485845838 learned_addr learned my addr 172.21.15.2:0/2485845838 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909a109640 1 -- 172.21.15.2:0/2485845838 >> v1:172.21.15.123:6789/0 conn(0x7f9094073220 legacy=0x7f909406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909a109640 1 -- 172.21.15.2:0/2485845838 >> v1:172.21.15.2:6789/0 conn(0x7f909415caa0 legacy=0x7f9094155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f9099107640 1 -- 172.21.15.2:0/2485845838 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 959322906 0 0) 0x7f9094158960 con 0x7f909415caa0 2024-09-19T05:44:18.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.734+0000 7f909a109640 1 -- 172.21.15.2:0/2485845838 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9094158c20 con 0x7f9094072e40 2024-09-19T05:44:18.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.735+0000 7f909a109640 1 --2- 172.21.15.2:0/2485845838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90940722c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90900097a0 tx=0x7f909002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801cc541414396be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.735+0000 7f9099107640 1 -- 172.21.15.2:0/2485845838 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090037070 con 0x7f9094072e40 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.735+0000 7f9099107640 1 -- 172.21.15.2:0/2485845838 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9090032640 con 0x7f9094072e40 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.735+0000 7f9099107640 1 -- 172.21.15.2:0/2485845838 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090032a10 con 0x7f9094072e40 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.736+0000 7f909c36d640 1 -- 172.21.15.2:0/2485845838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 msgr2=0x7f90940722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.736+0000 7f909c36d640 1 --2- 172.21.15.2:0/2485845838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90940722c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90900097a0 tx=0x7f909002a2c0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.736+0000 7f909c36d640 1 -- 172.21.15.2:0/2485845838 shutdown_connections 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.736+0000 7f909c36d640 1 --2- 172.21.15.2:0/2485845838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90940722c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.736+0000 7f909c36d640 1 -- 172.21.15.2:0/2485845838 >> 172.21.15.2:0/2485845838 conn(0x7f909406e450 msgr2=0x7f909406e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.736+0000 7f909c36d640 1 -- 172.21.15.2:0/2485845838 shutdown_connections 2024-09-19T05:44:18.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.736+0000 7f909c36d640 1 -- 172.21.15.2:0/2485845838 wait complete. 2024-09-19T05:44:18.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.737+0000 7f909c36d640 1 Processor -- start 2024-09-19T05:44:18.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.737+0000 7f909c36d640 1 -- start start 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909c36d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90941e4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909c36d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9094073220 0x7f90941e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909c36d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f909415caa0 0x7f90941e5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909c36d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9094072800 con 0x7f9094072e40 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909c36d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90941591e0 con 0x7f9094073220 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909c36d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f90940c77e0 con 0x7f909415caa0 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90941e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90941e4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33340/0 (socket says 172.21.15.2:33340) 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 -- 172.21.15.2:0/2276193340 learned_addr learned my addr 172.21.15.2:0/2276193340 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f9099908640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9094073220 0x7f90941e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a90a640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f909415caa0 0x7f90941e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 -- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f909415caa0 msgr2=0x7f90941e5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f909415caa0 0x7f90941e5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 -- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9094073220 msgr2=0x7f90941e27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9094073220 0x7f90941e27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f909a109640 1 -- 172.21.15.2:0/2276193340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90941e1c00 con 0x7f9094072e40 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.738+0000 7f9099908640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9094073220 0x7f90941e27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.739+0000 7f909a109640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90941e4690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90900098c0 tx=0x7f90900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.739+0000 7f908affd640 1 -- 172.21.15.2:0/2276193340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090041020 con 0x7f9094072e40 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.739+0000 7f908affd640 1 -- 172.21.15.2:0/2276193340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9090030970 con 0x7f9094072e40 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.739+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90941f18f0 con 0x7f9094072e40 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.739+0000 7f908affd640 1 -- 172.21.15.2:0/2276193340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090037040 con 0x7f9094072e40 2024-09-19T05:44:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.739+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90941e3e90 con 0x7f9094072e40 2024-09-19T05:44:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.740+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90940c9700 con 0x7f9094072e40 2024-09-19T05:44:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.740+0000 7f908affd640 1 -- 172.21.15.2:0/2276193340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9090044020 con 0x7f9094072e40 2024-09-19T05:44:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.741+0000 7f908affd640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f907806c2b0 0x7f907806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.741+0000 7f908affd640 1 -- 172.21.15.2:0/2276193340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5770+0+0 (secure 0 0 0) 0x7f90900abd30 con 0x7f9094072e40 2024-09-19T05:44:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.741+0000 7f9099908640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f907806c2b0 0x7f907806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.742+0000 7f9099908640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f907806c2b0 0x7f907806e770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90941f28e0 tx=0x7f9084009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.745+0000 7f908affd640 1 -- 172.21.15.2:0/2276193340 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f90940c9700 con 0x7f9094072e40 2024-09-19T05:44:18.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:44:18.777 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:18.778 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:18.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:18.814 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:18.813+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:18.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.876+0000 7feb50c7b640 1 Processor -- start 2024-09-19T05:44:18.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.876+0000 7feb50c7b640 1 -- start start 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.877+0000 7feb50c7b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c155000 0x7feb4c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.877+0000 7feb50c7b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4c0ccbc0 con 0x7feb4c0caf40 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.877+0000 7feb50c7b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4c0cc600 con 0x7feb4c154c20 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.877+0000 7feb50c7b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c0cc8e0 con 0x7feb4c155000 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.877+0000 7feb4ad9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7feb4c0caf40 0x7feb4c15aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:59880/0 (socket says 172.21.15.2:59880) 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.877+0000 7feb4ad9d640 1 -- 172.21.15.2:0/1317297655 learned_addr learned my addr 172.21.15.2:0/1317297655 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.877+0000 7feb49d9b640 1 --2- 172.21.15.2:0/1317297655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c155000 0x7feb4c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.878+0000 7feb4959a640 1 -- 172.21.15.2:0/1317297655 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 794643174 0 0) 0x7feb4c0ccbc0 con 0x7feb4c0caf40 2024-09-19T05:44:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.878+0000 7feb4959a640 1 -- 172.21.15.2:0/1317297655 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb34003520 con 0x7feb4c0caf40 2024-09-19T05:44:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.878+0000 7feb4959a640 1 -- 172.21.15.2:0/1317297655 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125694164 0 0) 0x7feb4c0cc600 con 0x7feb4c154c20 2024-09-19T05:44:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.878+0000 7feb4959a640 1 -- 172.21.15.2:0/1317297655 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb4c0ccbc0 con 0x7feb4c154c20 2024-09-19T05:44:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.878+0000 7feb49d9b640 1 -- 172.21.15.2:0/1317297655 >> v1:172.21.15.123:6789/0 conn(0x7feb4c154c20 legacy=0x7feb4c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.878+0000 7feb49d9b640 1 -- 172.21.15.2:0/1317297655 >> v1:172.21.15.2:6789/0 conn(0x7feb4c0caf40 legacy=0x7feb4c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.878+0000 7feb49d9b640 1 -- 172.21.15.2:0/1317297655 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c0cbc60 con 0x7feb4c155000 2024-09-19T05:44:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.879+0000 7feb49d9b640 1 --2- 172.21.15.2:0/1317297655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c155000 0x7feb4c0cd9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb4002a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3030a18cd4dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.879+0000 7feb4959a640 1 -- 172.21.15.2:0/1317297655 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40037070 con 0x7feb4c155000 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.879+0000 7feb4959a640 1 -- 172.21.15.2:0/1317297655 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb40032440 con 0x7feb4c155000 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.879+0000 7feb4959a640 1 -- 172.21.15.2:0/1317297655 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40032810 con 0x7feb4c155000 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.880+0000 7feb50c7b640 1 -- 172.21.15.2:0/1317297655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c155000 msgr2=0x7feb4c0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.880+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1317297655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c155000 0x7feb4c0cd9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb4002a1f0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.880+0000 7feb50c7b640 1 -- 172.21.15.2:0/1317297655 shutdown_connections 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.880+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1317297655 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c155000 0x7feb4c0cd9e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.880+0000 7feb50c7b640 1 -- 172.21.15.2:0/1317297655 >> 172.21.15.2:0/1317297655 conn(0x7feb4c06d0e0 msgr2=0x7feb4c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.880+0000 7feb50c7b640 1 -- 172.21.15.2:0/1317297655 shutdown_connections 2024-09-19T05:44:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.880+0000 7feb50c7b640 1 -- 172.21.15.2:0/1317297655 wait complete. 2024-09-19T05:44:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.882+0000 7feb50c7b640 1 Processor -- start 2024-09-19T05:44:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.882+0000 7feb50c7b640 1 -- start start 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb50c7b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb4c0caf40 0x7feb4c0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb50c7b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c154c20 0x7feb4c1e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb50c7b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c155000 0x7feb4c1e4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb50c7b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c073a50 con 0x7feb4c0caf40 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb50c7b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c0cb430 con 0x7feb4c155000 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb50c7b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c073bd0 con 0x7feb4c154c20 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb4a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb4c0caf40 0x7feb4c0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb4a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb4c0caf40 0x7feb4c0b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33342/0 (socket says 172.21.15.2:33342) 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb4a59c640 1 -- 172.21.15.2:0/1100306674 learned_addr learned my addr 172.21.15.2:0/1100306674 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb49d9b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c154c20 0x7feb4c1e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.883+0000 7feb4ad9d640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c155000 0x7feb4c1e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb4a59c640 1 -- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c154c20 msgr2=0x7feb4c1e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb4a59c640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c154c20 0x7feb4c1e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb4a59c640 1 -- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c155000 msgr2=0x7feb4c1e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb4a59c640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c155000 0x7feb4c1e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb4a59c640 1 -- 172.21.15.2:0/1100306674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c155970 con 0x7feb4c0caf40 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb4a59c640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb4c0caf40 0x7feb4c0b5370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c530 tx=0x7feb3c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb3affd640 1 -- 172.21.15.2:0/1100306674 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0052d0 con 0x7feb4c0caf40 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb3affd640 1 -- 172.21.15.2:0/1100306674 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3c005490 con 0x7feb4c0caf40 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4c155c50 con 0x7feb4c0caf40 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb3affd640 1 -- 172.21.15.2:0/1100306674 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c012870 con 0x7feb4c0caf40 2024-09-19T05:44:18.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.884+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4c1e3c40 con 0x7feb4c0caf40 2024-09-19T05:44:18.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.885+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb4c0c5770 con 0x7feb4c0caf40 2024-09-19T05:44:18.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.886+0000 7feb3affd640 1 -- 172.21.15.2:0/1100306674 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feb3c012a10 con 0x7feb4c0caf40 2024-09-19T05:44:18.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.886+0000 7feb3affd640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb2c06c2e0 0x7feb2c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:18.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.886+0000 7feb3affd640 1 -- 172.21.15.2:0/1100306674 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5770+0+0 (secure 0 0 0) 0x7feb3c08b030 con 0x7feb4c0caf40 2024-09-19T05:44:18.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.887+0000 7feb49d9b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb2c06c2e0 0x7feb2c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:18.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.887+0000 7feb49d9b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb2c06c2e0 0x7feb2c06e7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb40034000 tx=0x7feb4002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:18.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.891+0000 7feb3affd640 1 -- 172.21.15.2:0/1100306674 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7feb3c059090 con 0x7feb4c0caf40 2024-09-19T05:44:18.915 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:18.914+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:18.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.924+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90941e1ec0 con 0x7f907806c2b0 2024-09-19T05:44:18.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.931+0000 7f908affd640 1 -- 172.21.15.2:0/2276193340 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253878 (secure 0 0 0) 0x7f90840013e0 con 0x7f907806c2b0 2024-09-19T05:44:18.935 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:44:18.937 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:44:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f907806c2b0 msgr2=0x7f907806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f907806c2b0 0x7f907806e770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90941f28e0 tx=0x7f9084009040 comp rx=0 tx=0).stop 2024-09-19T05:44:18.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 msgr2=0x7f90941e4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90941e4690 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90900098c0 tx=0x7f90900046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 shutdown_connections 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f907806c2b0 0x7f907806e770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9094073220 0x7f90941e27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f909415caa0 0x7f90941e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 --2- 172.21.15.2:0/2276193340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9094072e40 0x7f90941e4690 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 >> 172.21.15.2:0/2276193340 conn(0x7f909406e450 msgr2=0x7f909406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 shutdown_connections 2024-09-19T05:44:18.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:18.935+0000 7f909c36d640 1 -- 172.21.15.2:0/2276193340 wait complete. 2024-09-19T05:44:18.940 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:18.940 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:18.948 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-19T05:44:17.838821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1066048243,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":0,"num_read":117,"num_read_kb":101,"num_write":1495,"num_write_kb":4004369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":590508,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1695,"ondisk_log_size":1695,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":386996,"kb_used_data":136980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754587724,"statfs":{"total":773094113280,"available":772697829376,"internally_reserved":0,"allocated":140267520,"data_stored":2151372319,"data_compressed":103263713,"data_compressed_allocated":134307840,"data_compressed_original":2148646912,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":164,"apply_latency_ms":164,"commit_latency_ns":164000000,"apply_latency_ns":164000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":926948492,"num_objects":221,"num_object_clones":0,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":434,"num_write_kb":1769482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.129757"},"pg_stats":[{"pgid":"3.3f","version":"61'22","reported_seq":113,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.858041+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:44:14.858041+0000","last_peered":"2024-09-19T05:44:14.858041+0000","last_clean":"2024-09-19T05:44:14.858041+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:44:14.858041+0000","last_undegraded":"2024-09-19T05:44:14.858041+0000","last_fullsized":"2024-09-19T05:44:14.858041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"62'15","reported_seq":107,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767827+0000","last_change":"2024-09-19T05:42:32.941327+0000","last_active":"2024-09-19T05:44:14.767827+0000","last_peered":"2024-09-19T05:44:14.767827+0000","last_clean":"2024-09-19T05:44:14.767827+0000","last_became_active":"2024-09-19T05:42:32.941026+0000","last_became_peered":"2024-09-19T05:42:32.941026+0000","last_unstale":"2024-09-19T05:44:14.767827+0000","last_undegraded":"2024-09-19T05:44:14.767827+0000","last_fullsized":"2024-09-19T05:44:14.767827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:17.058604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"62'21","reported_seq":113,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767379+0000","last_change":"2024-09-19T05:42:32.938429+0000","last_active":"2024-09-19T05:44:14.767379+0000","last_peered":"2024-09-19T05:44:14.767379+0000","last_clean":"2024-09-19T05:44:14.767379+0000","last_became_active":"2024-09-19T05:42:32.938016+0000","last_became_peered":"2024-09-19T05:42:32.938016+0000","last_unstale":"2024-09-19T05:44:14.767379+0000","last_undegraded":"2024-09-19T05:44:14.767379+0000","last_fullsized":"2024-09-19T05:44:14.767379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.358319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"61'18","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754429+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:44:15.754429+0000","last_peered":"2024-09-19T05:44:15.754429+0000","last_clean":"2024-09-19T05:44:15.754429+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:44:15.754429+0000","last_undegraded":"2024-09-19T05:44:15.754429+0000","last_fullsized":"2024-09-19T05:44:15.754429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"62'28","reported_seq":119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.909698+0000","last_change":"2024-09-19T05:42:32.944765+0000","last_active":"2024-09-19T05:44:14.909698+0000","last_peered":"2024-09-19T05:44:14.909698+0000","last_clean":"2024-09-19T05:44:14.909698+0000","last_became_active":"2024-09-19T05:42:32.943607+0000","last_became_peered":"2024-09-19T05:42:32.943607+0000","last_unstale":"2024-09-19T05:44:14.909698+0000","last_undegraded":"2024-09-19T05:44:14.909698+0000","last_fullsized":"2024-09-19T05:44:14.909698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:22.131721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"61'19","reported_seq":113,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754333+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:44:15.754333+0000","last_peered":"2024-09-19T05:44:15.754333+0000","last_clean":"2024-09-19T05:44:15.754333+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:44:15.754333+0000","last_undegraded":"2024-09-19T05:44:15.754333+0000","last_fullsized":"2024-09-19T05:44:15.754333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"62'31","reported_seq":123,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.832888+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:44:14.832888+0000","last_peered":"2024-09-19T05:44:14.832888+0000","last_clean":"2024-09-19T05:44:14.832888+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:44:14.832888+0000","last_undegraded":"2024-09-19T05:44:14.832888+0000","last_fullsized":"2024-09-19T05:44:14.832888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"61'16","reported_seq":105,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.766658+0000","last_change":"2024-09-19T05:42:32.947992+0000","last_active":"2024-09-19T05:44:14.766658+0000","last_peered":"2024-09-19T05:44:14.766658+0000","last_clean":"2024-09-19T05:44:14.766658+0000","last_became_active":"2024-09-19T05:42:32.947633+0000","last_became_peered":"2024-09-19T05:42:32.947633+0000","last_unstale":"2024-09-19T05:44:14.766658+0000","last_undegraded":"2024-09-19T05:44:14.766658+0000","last_fullsized":"2024-09-19T05:44:14.766658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:41.840199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"61'27","reported_seq":82,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.903462+0000","last_change":"2024-09-19T05:44:07.706036+0000","last_active":"2024-09-19T05:44:14.903462+0000","last_peered":"2024-09-19T05:44:14.903462+0000","last_clean":"2024-09-19T05:44:14.903462+0000","last_became_active":"2024-09-19T05:44:07.705688+0000","last_became_peered":"2024-09-19T05:44:07.705688+0000","last_unstale":"2024-09-19T05:44:14.903462+0000","last_undegraded":"2024-09-19T05:44:14.903462+0000","last_fullsized":"2024-09-19T05:44:14.903462+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:02.308553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.36","version":"62'30","reported_seq":120,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.833808+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:44:14.833808+0000","last_peered":"2024-09-19T05:44:14.833808+0000","last_clean":"2024-09-19T05:44:14.833808+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:44:14.833808+0000","last_undegraded":"2024-09-19T05:44:14.833808+0000","last_fullsized":"2024-09-19T05:44:14.833808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"61'21","reported_seq":111,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.843713+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:44:14.843713+0000","last_peered":"2024-09-19T05:44:14.843713+0000","last_clean":"2024-09-19T05:44:14.843713+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:44:14.843713+0000","last_undegraded":"2024-09-19T05:44:14.843713+0000","last_fullsized":"2024-09-19T05:44:14.843713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"62'32","reported_seq":134,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.929131+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:44:14.929131+0000","last_peered":"2024-09-19T05:44:14.929131+0000","last_clean":"2024-09-19T05:44:14.929131+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:44:14.929131+0000","last_undegraded":"2024-09-19T05:44:14.929131+0000","last_fullsized":"2024-09-19T05:44:14.929131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:54.111389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"61'24","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.759007+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:44:14.759007+0000","last_peered":"2024-09-19T05:44:14.759007+0000","last_clean":"2024-09-19T05:44:14.759007+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:44:14.759007+0000","last_undegraded":"2024-09-19T05:44:14.759007+0000","last_fullsized":"2024-09-19T05:44:14.759007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"61'23","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.756815+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:44:14.756815+0000","last_peered":"2024-09-19T05:44:14.756815+0000","last_clean":"2024-09-19T05:44:14.756815+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:44:14.756815+0000","last_undegraded":"2024-09-19T05:44:14.756815+0000","last_fullsized":"2024-09-19T05:44:14.756815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"61'23","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767531+0000","last_change":"2024-09-19T05:42:32.947816+0000","last_active":"2024-09-19T05:44:14.767531+0000","last_peered":"2024-09-19T05:44:14.767531+0000","last_clean":"2024-09-19T05:44:14.767531+0000","last_became_active":"2024-09-19T05:42:32.947493+0000","last_became_peered":"2024-09-19T05:42:32.947493+0000","last_unstale":"2024-09-19T05:44:14.767531+0000","last_undegraded":"2024-09-19T05:44:14.767531+0000","last_fullsized":"2024-09-19T05:44:14.767531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.910270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"62'30","reported_seq":122,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.949111+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:44:14.949111+0000","last_peered":"2024-09-19T05:44:14.949111+0000","last_clean":"2024-09-19T05:44:14.949111+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:44:14.949111+0000","last_undegraded":"2024-09-19T05:44:14.949111+0000","last_fullsized":"2024-09-19T05:44:14.949111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"59'17","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.903663+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:44:14.903663+0000","last_peered":"2024-09-19T05:44:14.903663+0000","last_clean":"2024-09-19T05:44:14.903663+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:44:14.903663+0000","last_undegraded":"2024-09-19T05:44:14.903663+0000","last_fullsized":"2024-09-19T05:44:14.903663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"61'15","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.753729+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:44:14.753729+0000","last_peered":"2024-09-19T05:44:14.753729+0000","last_clean":"2024-09-19T05:44:14.753729+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:44:14.753729+0000","last_undegraded":"2024-09-19T05:44:14.753729+0000","last_fullsized":"2024-09-19T05:44:14.753729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"62'17","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.943699+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:44:14.943699+0000","last_peered":"2024-09-19T05:44:14.943699+0000","last_clean":"2024-09-19T05:44:14.943699+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:44:14.943699+0000","last_undegraded":"2024-09-19T05:44:14.943699+0000","last_fullsized":"2024-09-19T05:44:14.943699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"62'18","reported_seq":120,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.834686+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:44:14.834686+0000","last_peered":"2024-09-19T05:44:14.834686+0000","last_clean":"2024-09-19T05:44:14.834686+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:44:14.834686+0000","last_undegraded":"2024-09-19T05:44:14.834686+0000","last_fullsized":"2024-09-19T05:44:14.834686+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:17.766409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"61'21","reported_seq":111,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.750916+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:44:14.750916+0000","last_peered":"2024-09-19T05:44:14.750916+0000","last_clean":"2024-09-19T05:44:14.750916+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:44:14.750916+0000","last_undegraded":"2024-09-19T05:44:14.750916+0000","last_fullsized":"2024-09-19T05:44:14.750916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"61'29","reported_seq":82,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.833616+0000","last_change":"2024-09-19T05:44:07.704123+0000","last_active":"2024-09-19T05:44:14.833616+0000","last_peered":"2024-09-19T05:44:14.833616+0000","last_clean":"2024-09-19T05:44:14.833616+0000","last_became_active":"2024-09-19T05:44:07.703742+0000","last_became_peered":"2024-09-19T05:44:07.703742+0000","last_unstale":"2024-09-19T05:44:14.833616+0000","last_undegraded":"2024-09-19T05:44:14.833616+0000","last_fullsized":"2024-09-19T05:44:14.833616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:34:07.185021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.29","version":"61'11","reported_seq":103,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767549+0000","last_change":"2024-09-19T05:42:32.940647+0000","last_active":"2024-09-19T05:44:14.767549+0000","last_peered":"2024-09-19T05:44:14.767549+0000","last_clean":"2024-09-19T05:44:14.767549+0000","last_became_active":"2024-09-19T05:42:32.940252+0000","last_became_peered":"2024-09-19T05:42:32.940252+0000","last_unstale":"2024-09-19T05:44:14.767549+0000","last_undegraded":"2024-09-19T05:44:14.767549+0000","last_fullsized":"2024-09-19T05:44:14.767549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:40.321628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"61'27","reported_seq":119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.861243+0000","last_change":"2024-09-19T05:42:32.941862+0000","last_active":"2024-09-19T05:44:14.861243+0000","last_peered":"2024-09-19T05:44:14.861243+0000","last_clean":"2024-09-19T05:44:14.861243+0000","last_became_active":"2024-09-19T05:42:32.941290+0000","last_became_peered":"2024-09-19T05:42:32.941290+0000","last_unstale":"2024-09-19T05:44:14.861243+0000","last_undegraded":"2024-09-19T05:44:14.861243+0000","last_fullsized":"2024-09-19T05:44:14.861243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"61'16","reported_seq":118,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767793+0000","last_change":"2024-09-19T05:43:19.551649+0000","last_active":"2024-09-19T05:44:14.767793+0000","last_peered":"2024-09-19T05:44:14.767793+0000","last_clean":"2024-09-19T05:44:14.767793+0000","last_became_active":"2024-09-19T05:43:19.551457+0000","last_became_peered":"2024-09-19T05:43:19.551457+0000","last_unstale":"2024-09-19T05:44:14.767793+0000","last_undegraded":"2024-09-19T05:44:14.767793+0000","last_fullsized":"2024-09-19T05:44:14.767793+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:23.453548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"62'38","reported_seq":132,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754555+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:44:15.754555+0000","last_peered":"2024-09-19T05:44:15.754555+0000","last_clean":"2024-09-19T05:44:15.754555+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:44:15.754555+0000","last_undegraded":"2024-09-19T05:44:15.754555+0000","last_fullsized":"2024-09-19T05:44:15.754555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"61'15","reported_seq":106,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.758290+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:44:14.758290+0000","last_peered":"2024-09-19T05:44:14.758290+0000","last_clean":"2024-09-19T05:44:14.758290+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:44:14.758290+0000","last_undegraded":"2024-09-19T05:44:14.758290+0000","last_fullsized":"2024-09-19T05:44:14.758290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"61'23","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.802516+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:44:14.802516+0000","last_peered":"2024-09-19T05:44:14.802516+0000","last_clean":"2024-09-19T05:44:14.802516+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:44:14.802516+0000","last_undegraded":"2024-09-19T05:44:14.802516+0000","last_fullsized":"2024-09-19T05:44:14.802516+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"62'20","reported_seq":112,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.772410+0000","last_change":"2024-09-19T05:42:32.937218+0000","last_active":"2024-09-19T05:44:14.772410+0000","last_peered":"2024-09-19T05:44:14.772410+0000","last_clean":"2024-09-19T05:44:14.772410+0000","last_became_active":"2024-09-19T05:42:32.936912+0000","last_became_peered":"2024-09-19T05:42:32.936912+0000","last_unstale":"2024-09-19T05:44:14.772410+0000","last_undegraded":"2024-09-19T05:44:14.772410+0000","last_fullsized":"2024-09-19T05:44:14.772410+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:04.906758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"62'18","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.969280+0000","last_change":"2024-09-19T05:42:32.942631+0000","last_active":"2024-09-19T05:44:14.969280+0000","last_peered":"2024-09-19T05:44:14.969280+0000","last_clean":"2024-09-19T05:44:14.969280+0000","last_became_active":"2024-09-19T05:42:32.941858+0000","last_became_peered":"2024-09-19T05:42:32.941858+0000","last_unstale":"2024-09-19T05:44:14.969280+0000","last_undegraded":"2024-09-19T05:44:14.969280+0000","last_fullsized":"2024-09-19T05:44:14.969280+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:52.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"61'23","reported_seq":125,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.753140+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:44:14.753140+0000","last_peered":"2024-09-19T05:44:14.753140+0000","last_clean":"2024-09-19T05:44:14.753140+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:44:14.753140+0000","last_undegraded":"2024-09-19T05:44:14.753140+0000","last_fullsized":"2024-09-19T05:44:14.753140+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"61'33","reported_seq":127,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754187+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:44:15.754187+0000","last_peered":"2024-09-19T05:44:15.754187+0000","last_clean":"2024-09-19T05:44:15.754187+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:44:15.754187+0000","last_undegraded":"2024-09-19T05:44:15.754187+0000","last_fullsized":"2024-09-19T05:44:15.754187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"60'19","reported_seq":75,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767476+0000","last_change":"2024-09-19T05:44:07.704699+0000","last_active":"2024-09-19T05:44:14.767476+0000","last_peered":"2024-09-19T05:44:14.767476+0000","last_clean":"2024-09-19T05:44:14.767476+0000","last_became_active":"2024-09-19T05:44:07.704069+0000","last_became_peered":"2024-09-19T05:44:07.704069+0000","last_unstale":"2024-09-19T05:44:14.767476+0000","last_undegraded":"2024-09-19T05:44:14.767476+0000","last_fullsized":"2024-09-19T05:44:14.767476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:49:48.767176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":95,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.770227+0000","last_change":"2024-09-19T05:42:31.990782+0000","last_active":"2024-09-19T05:44:14.770227+0000","last_peered":"2024-09-19T05:44:14.770227+0000","last_clean":"2024-09-19T05:44:14.770227+0000","last_became_active":"2024-09-19T05:42:31.990329+0000","last_became_peered":"2024-09-19T05:42:31.990329+0000","last_unstale":"2024-09-19T05:44:14.770227+0000","last_undegraded":"2024-09-19T05:44:14.770227+0000","last_fullsized":"2024-09-19T05:44:14.770227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:39:50.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"61'22","reported_seq":113,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.847580+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:44:14.847580+0000","last_peered":"2024-09-19T05:44:14.847580+0000","last_clean":"2024-09-19T05:44:14.847580+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:44:14.847580+0000","last_undegraded":"2024-09-19T05:44:14.847580+0000","last_fullsized":"2024-09-19T05:44:14.847580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":95,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.913968+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:44:14.913968+0000","last_peered":"2024-09-19T05:44:14.913968+0000","last_clean":"2024-09-19T05:44:14.913968+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:44:14.913968+0000","last_undegraded":"2024-09-19T05:44:14.913968+0000","last_fullsized":"2024-09-19T05:44:14.913968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":93,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.756662+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:44:14.756662+0000","last_peered":"2024-09-19T05:44:14.756662+0000","last_clean":"2024-09-19T05:44:14.756662+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:44:14.756662+0000","last_undegraded":"2024-09-19T05:44:14.756662+0000","last_fullsized":"2024-09-19T05:44:14.756662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"62'21","reported_seq":113,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767380+0000","last_change":"2024-09-19T05:42:32.940658+0000","last_active":"2024-09-19T05:44:14.767380+0000","last_peered":"2024-09-19T05:44:14.767380+0000","last_clean":"2024-09-19T05:44:14.767380+0000","last_became_active":"2024-09-19T05:42:32.940321+0000","last_became_peered":"2024-09-19T05:42:32.940321+0000","last_unstale":"2024-09-19T05:44:14.767380+0000","last_undegraded":"2024-09-19T05:44:14.767380+0000","last_fullsized":"2024-09-19T05:44:14.767380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:48.122528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.860325+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:44:14.860325+0000","last_peered":"2024-09-19T05:44:14.860325+0000","last_clean":"2024-09-19T05:44:14.860325+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:44:14.860325+0000","last_undegraded":"2024-09-19T05:44:14.860325+0000","last_fullsized":"2024-09-19T05:44:14.860325+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"61'19","reported_seq":120,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.752247+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:44:14.752247+0000","last_peered":"2024-09-19T05:44:14.752247+0000","last_clean":"2024-09-19T05:44:14.752247+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:44:14.752247+0000","last_undegraded":"2024-09-19T05:44:14.752247+0000","last_fullsized":"2024-09-19T05:44:14.752247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.751294+0000","last_change":"2024-09-19T05:44:07.704658+0000","last_active":"2024-09-19T05:44:14.751294+0000","last_peered":"2024-09-19T05:44:14.751294+0000","last_clean":"2024-09-19T05:44:14.751294+0000","last_became_active":"2024-09-19T05:44:07.704389+0000","last_became_peered":"2024-09-19T05:44:07.704389+0000","last_unstale":"2024-09-19T05:44:14.751294+0000","last_undegraded":"2024-09-19T05:44:14.751294+0000","last_fullsized":"2024-09-19T05:44:14.751294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"62'28","reported_seq":122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754612+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:44:15.754612+0000","last_peered":"2024-09-19T05:44:15.754612+0000","last_clean":"2024-09-19T05:44:15.754612+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:44:15.754612+0000","last_undegraded":"2024-09-19T05:44:15.754612+0000","last_fullsized":"2024-09-19T05:44:15.754612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":104,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.833395+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:44:14.833395+0000","last_peered":"2024-09-19T05:44:14.833395+0000","last_clean":"2024-09-19T05:44:14.833395+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:44:14.833395+0000","last_undegraded":"2024-09-19T05:44:14.833395+0000","last_fullsized":"2024-09-19T05:44:14.833395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:39.063921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"61'20","reported_seq":111,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767884+0000","last_change":"2024-09-19T05:42:32.943510+0000","last_active":"2024-09-19T05:44:14.767884+0000","last_peered":"2024-09-19T05:44:14.767884+0000","last_clean":"2024-09-19T05:44:14.767884+0000","last_became_active":"2024-09-19T05:42:32.943203+0000","last_became_peered":"2024-09-19T05:42:32.943203+0000","last_unstale":"2024-09-19T05:44:14.767884+0000","last_undegraded":"2024-09-19T05:44:14.767884+0000","last_fullsized":"2024-09-19T05:44:14.767884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:31.670468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"59'5","reported_seq":99,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.753947+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:44:14.753947+0000","last_peered":"2024-09-19T05:44:14.753947+0000","last_clean":"2024-09-19T05:44:14.753947+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:44:14.753947+0000","last_undegraded":"2024-09-19T05:44:14.753947+0000","last_fullsized":"2024-09-19T05:44:14.753947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"61'24","reported_seq":116,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.872966+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:44:14.872966+0000","last_peered":"2024-09-19T05:44:14.872966+0000","last_clean":"2024-09-19T05:44:14.872966+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:44:14.872966+0000","last_undegraded":"2024-09-19T05:44:14.872966+0000","last_fullsized":"2024-09-19T05:44:14.872966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"62'52","reported_seq":148,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754521+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:44:15.754521+0000","last_peered":"2024-09-19T05:44:15.754521+0000","last_clean":"2024-09-19T05:44:15.754521+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:44:15.754521+0000","last_undegraded":"2024-09-19T05:44:15.754521+0000","last_fullsized":"2024-09-19T05:44:15.754521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103760,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"61'27","reported_seq":119,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.905956+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:44:14.905956+0000","last_peered":"2024-09-19T05:44:14.905956+0000","last_clean":"2024-09-19T05:44:14.905956+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:44:14.905956+0000","last_undegraded":"2024-09-19T05:44:14.905956+0000","last_fullsized":"2024-09-19T05:44:14.905956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":96,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.753365+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:44:14.753365+0000","last_peered":"2024-09-19T05:44:14.753365+0000","last_clean":"2024-09-19T05:44:14.753365+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:44:14.753365+0000","last_undegraded":"2024-09-19T05:44:14.753365+0000","last_fullsized":"2024-09-19T05:44:14.753365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"61'30","reported_seq":118,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.756878+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:44:14.756878+0000","last_peered":"2024-09-19T05:44:14.756878+0000","last_clean":"2024-09-19T05:44:14.756878+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:44:14.756878+0000","last_undegraded":"2024-09-19T05:44:14.756878+0000","last_fullsized":"2024-09-19T05:44:14.756878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.914057+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:44:14.914057+0000","last_peered":"2024-09-19T05:44:14.914057+0000","last_clean":"2024-09-19T05:44:14.914057+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:44:14.914057+0000","last_undegraded":"2024-09-19T05:44:14.914057+0000","last_fullsized":"2024-09-19T05:44:14.914057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"61'21","reported_seq":113,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.859829+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:44:14.859829+0000","last_peered":"2024-09-19T05:44:14.859829+0000","last_clean":"2024-09-19T05:44:14.859829+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:44:14.859829+0000","last_undegraded":"2024-09-19T05:44:14.859829+0000","last_fullsized":"2024-09-19T05:44:14.859829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.832944+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:44:14.832944+0000","last_peered":"2024-09-19T05:44:14.832944+0000","last_clean":"2024-09-19T05:44:14.832944+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:44:14.832944+0000","last_undegraded":"2024-09-19T05:44:14.832944+0000","last_fullsized":"2024-09-19T05:44:14.832944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"62'31","reported_seq":87,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.766701+0000","last_change":"2024-09-19T05:44:07.704404+0000","last_active":"2024-09-19T05:44:14.766701+0000","last_peered":"2024-09-19T05:44:14.766701+0000","last_clean":"2024-09-19T05:44:14.766701+0000","last_became_active":"2024-09-19T05:44:07.704102+0000","last_became_peered":"2024-09-19T05:44:07.704102+0000","last_unstale":"2024-09-19T05:44:14.766701+0000","last_undegraded":"2024-09-19T05:44:14.766701+0000","last_fullsized":"2024-09-19T05:44:14.766701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:19.628654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.750913+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:44:14.750913+0000","last_peered":"2024-09-19T05:44:14.750913+0000","last_clean":"2024-09-19T05:44:14.750913+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:44:14.750913+0000","last_undegraded":"2024-09-19T05:44:14.750913+0000","last_fullsized":"2024-09-19T05:44:14.750913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"61'43","reported_seq":130,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.900770+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:44:14.900770+0000","last_peered":"2024-09-19T05:44:14.900770+0000","last_clean":"2024-09-19T05:44:14.900770+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:44:14.900770+0000","last_undegraded":"2024-09-19T05:44:14.900770+0000","last_fullsized":"2024-09-19T05:44:14.900770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":95,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.858045+0000","last_change":"2024-09-19T05:42:31.993189+0000","last_active":"2024-09-19T05:44:14.858045+0000","last_peered":"2024-09-19T05:44:14.858045+0000","last_clean":"2024-09-19T05:44:14.858045+0000","last_became_active":"2024-09-19T05:42:31.992824+0000","last_became_peered":"2024-09-19T05:42:31.992824+0000","last_unstale":"2024-09-19T05:44:14.858045+0000","last_undegraded":"2024-09-19T05:44:14.858045+0000","last_fullsized":"2024-09-19T05:44:14.858045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"61'15","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754635+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:44:15.754635+0000","last_peered":"2024-09-19T05:44:15.754635+0000","last_clean":"2024-09-19T05:44:15.754635+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:44:15.754635+0000","last_undegraded":"2024-09-19T05:44:15.754635+0000","last_fullsized":"2024-09-19T05:44:15.754635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":95,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.751126+0000","last_change":"2024-09-19T05:42:31.992797+0000","last_active":"2024-09-19T05:44:14.751126+0000","last_peered":"2024-09-19T05:44:14.751126+0000","last_clean":"2024-09-19T05:44:14.751126+0000","last_became_active":"2024-09-19T05:42:31.992515+0000","last_became_peered":"2024-09-19T05:42:31.992515+0000","last_unstale":"2024-09-19T05:44:14.751126+0000","last_undegraded":"2024-09-19T05:44:14.751126+0000","last_fullsized":"2024-09-19T05:44:14.751126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:14.785384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-19T05:44:14.765415+0000","last_change":"2024-09-19T05:44:14.765415+0000","last_active":"0.000000","last_peered":"0.000000","last_clean":"0.000000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T05:44:14.765415+0000","last_undegraded":"2024-09-19T05:44:14.765415+0000","last_fullsized":"2024-09-19T05:44:14.765415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":113,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767622+0000","last_change":"2024-09-19T05:43:19.556467+0000","last_active":"2024-09-19T05:44:14.767622+0000","last_peered":"2024-09-19T05:44:14.767622+0000","last_clean":"2024-09-19T05:44:14.767622+0000","last_became_active":"2024-09-19T05:43:19.553271+0000","last_became_peered":"2024-09-19T05:43:19.553271+0000","last_unstale":"2024-09-19T05:44:14.767622+0000","last_undegraded":"2024-09-19T05:44:14.767622+0000","last_fullsized":"2024-09-19T05:44:14.767622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:05.953596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":140,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"61'17","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.834856+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:44:14.834856+0000","last_peered":"2024-09-19T05:44:14.834856+0000","last_clean":"2024-09-19T05:44:14.834856+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:44:14.834856+0000","last_undegraded":"2024-09-19T05:44:14.834856+0000","last_fullsized":"2024-09-19T05:44:14.834856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":93,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.757567+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:44:14.757567+0000","last_peered":"2024-09-19T05:44:14.757567+0000","last_clean":"2024-09-19T05:44:14.757567+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:44:14.757567+0000","last_undegraded":"2024-09-19T05:44:14.757567+0000","last_fullsized":"2024-09-19T05:44:14.757567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"62'23","reported_seq":125,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.943572+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:44:14.943572+0000","last_peered":"2024-09-19T05:44:14.943572+0000","last_clean":"2024-09-19T05:44:14.943572+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:44:14.943572+0000","last_undegraded":"2024-09-19T05:44:14.943572+0000","last_fullsized":"2024-09-19T05:44:14.943572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":203,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767431+0000","last_change":"2024-09-19T05:44:07.703859+0000","last_active":"2024-09-19T05:44:14.767431+0000","last_peered":"2024-09-19T05:44:14.767431+0000","last_clean":"2024-09-19T05:44:14.767431+0000","last_became_active":"2024-09-19T05:44:07.703552+0000","last_became_peered":"2024-09-19T05:44:07.703552+0000","last_unstale":"2024-09-19T05:44:14.767431+0000","last_undegraded":"2024-09-19T05:44:14.767431+0000","last_fullsized":"2024-09-19T05:44:14.767431+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:37.997161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.860085+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:44:14.860085+0000","last_peered":"2024-09-19T05:44:14.860085+0000","last_clean":"2024-09-19T05:44:14.860085+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:44:14.860085+0000","last_undegraded":"2024-09-19T05:44:14.860085+0000","last_fullsized":"2024-09-19T05:44:14.860085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"61'18","reported_seq":108,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.751055+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:44:14.751055+0000","last_peered":"2024-09-19T05:44:14.751055+0000","last_clean":"2024-09-19T05:44:14.751055+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:44:14.751055+0000","last_undegraded":"2024-09-19T05:44:14.751055+0000","last_fullsized":"2024-09-19T05:44:14.751055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.860416+0000","last_change":"2024-09-19T05:42:31.994287+0000","last_active":"2024-09-19T05:44:14.860416+0000","last_peered":"2024-09-19T05:44:14.860416+0000","last_clean":"2024-09-19T05:44:14.860416+0000","last_became_active":"2024-09-19T05:42:31.992733+0000","last_became_peered":"2024-09-19T05:42:31.992733+0000","last_unstale":"2024-09-19T05:44:14.860416+0000","last_undegraded":"2024-09-19T05:44:14.860416+0000","last_fullsized":"2024-09-19T05:44:14.860416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:28.303892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"61'39","reported_seq":127,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.756418+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:44:14.756418+0000","last_peered":"2024-09-19T05:44:14.756418+0000","last_clean":"2024-09-19T05:44:14.756418+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:44:14.756418+0000","last_undegraded":"2024-09-19T05:44:14.756418+0000","last_fullsized":"2024-09-19T05:44:14.756418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'2","reported_seq":97,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.753247+0000","last_change":"2024-09-19T05:42:31.991339+0000","last_active":"2024-09-19T05:44:14.753247+0000","last_peered":"2024-09-19T05:44:14.753247+0000","last_clean":"2024-09-19T05:44:14.753247+0000","last_became_active":"2024-09-19T05:42:31.991033+0000","last_became_peered":"2024-09-19T05:42:31.991033+0000","last_unstale":"2024-09-19T05:44:14.753247+0000","last_undegraded":"2024-09-19T05:44:14.753247+0000","last_fullsized":"2024-09-19T05:44:14.753247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"61'22","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767184+0000","last_change":"2024-09-19T05:42:32.937377+0000","last_active":"2024-09-19T05:44:14.767184+0000","last_peered":"2024-09-19T05:44:14.767184+0000","last_clean":"2024-09-19T05:44:14.767184+0000","last_became_active":"2024-09-19T05:42:32.937099+0000","last_became_peered":"2024-09-19T05:42:32.937099+0000","last_unstale":"2024-09-19T05:44:14.767184+0000","last_undegraded":"2024-09-19T05:44:14.767184+0000","last_fullsized":"2024-09-19T05:44:14.767184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:13.692802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.914134+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:44:14.914134+0000","last_peered":"2024-09-19T05:44:14.914134+0000","last_clean":"2024-09-19T05:44:14.914134+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:44:14.914134+0000","last_undegraded":"2024-09-19T05:44:14.914134+0000","last_fullsized":"2024-09-19T05:44:14.914134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"61'18","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.842739+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:44:14.842739+0000","last_peered":"2024-09-19T05:44:14.842739+0000","last_clean":"2024-09-19T05:44:14.842739+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:44:14.842739+0000","last_undegraded":"2024-09-19T05:44:14.842739+0000","last_fullsized":"2024-09-19T05:44:14.842739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767695+0000","last_change":"2024-09-19T05:42:31.990806+0000","last_active":"2024-09-19T05:44:14.767695+0000","last_peered":"2024-09-19T05:44:14.767695+0000","last_clean":"2024-09-19T05:44:14.767695+0000","last_became_active":"2024-09-19T05:42:31.990560+0000","last_became_peered":"2024-09-19T05:42:31.990560+0000","last_unstale":"2024-09-19T05:44:14.767695+0000","last_undegraded":"2024-09-19T05:44:14.767695+0000","last_fullsized":"2024-09-19T05:44:14.767695+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:52.609962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"62'24","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-19T05:44:15.754472+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:44:15.754472+0000","last_peered":"2024-09-19T05:44:15.754472+0000","last_clean":"2024-09-19T05:44:15.754472+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:44:15.754472+0000","last_undegraded":"2024-09-19T05:44:15.754472+0000","last_fullsized":"2024-09-19T05:44:15.754472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.911075+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:44:14.911075+0000","last_peered":"2024-09-19T05:44:14.911075+0000","last_clean":"2024-09-19T05:44:14.911075+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:44:14.911075+0000","last_undegraded":"2024-09-19T05:44:14.911075+0000","last_fullsized":"2024-09-19T05:44:14.911075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"62'19","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.781577+0000","last_change":"2024-09-19T05:42:32.941893+0000","last_active":"2024-09-19T05:44:14.781577+0000","last_peered":"2024-09-19T05:44:14.781577+0000","last_clean":"2024-09-19T05:44:14.781577+0000","last_became_active":"2024-09-19T05:42:32.941611+0000","last_became_peered":"2024-09-19T05:42:32.941611+0000","last_unstale":"2024-09-19T05:44:14.781577+0000","last_undegraded":"2024-09-19T05:44:14.781577+0000","last_fullsized":"2024-09-19T05:44:14.781577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:23.800542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"60'8","reported_seq":101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.751239+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:44:14.751239+0000","last_peered":"2024-09-19T05:44:14.751239+0000","last_clean":"2024-09-19T05:44:14.751239+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:44:14.751239+0000","last_undegraded":"2024-09-19T05:44:14.751239+0000","last_fullsized":"2024-09-19T05:44:14.751239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"61'36","reported_seq":127,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.751167+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:44:14.751167+0000","last_peered":"2024-09-19T05:44:14.751167+0000","last_clean":"2024-09-19T05:44:14.751167+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:44:14.751167+0000","last_undegraded":"2024-09-19T05:44:14.751167+0000","last_fullsized":"2024-09-19T05:44:14.751167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":96,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.904047+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:44:14.904047+0000","last_peered":"2024-09-19T05:44:14.904047+0000","last_clean":"2024-09-19T05:44:14.904047+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:44:14.904047+0000","last_undegraded":"2024-09-19T05:44:14.904047+0000","last_fullsized":"2024-09-19T05:44:14.904047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"61'19","reported_seq":110,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.750964+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:44:14.750964+0000","last_peered":"2024-09-19T05:44:14.750964+0000","last_clean":"2024-09-19T05:44:14.750964+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:44:14.750964+0000","last_undegraded":"2024-09-19T05:44:14.750964+0000","last_fullsized":"2024-09-19T05:44:14.750964+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":94,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.833701+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:44:14.833701+0000","last_peered":"2024-09-19T05:44:14.833701+0000","last_clean":"2024-09-19T05:44:14.833701+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:44:14.833701+0000","last_undegraded":"2024-09-19T05:44:14.833701+0000","last_fullsized":"2024-09-19T05:44:14.833701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"61'37","reported_seq":129,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.877164+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:44:14.877164+0000","last_peered":"2024-09-19T05:44:14.877164+0000","last_clean":"2024-09-19T05:44:14.877164+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:44:14.877164+0000","last_undegraded":"2024-09-19T05:44:14.877164+0000","last_fullsized":"2024-09-19T05:44:14.877164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":93,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.752253+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:44:14.752253+0000","last_peered":"2024-09-19T05:44:14.752253+0000","last_clean":"2024-09-19T05:44:14.752253+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:44:14.752253+0000","last_undegraded":"2024-09-19T05:44:14.752253+0000","last_fullsized":"2024-09-19T05:44:14.752253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"61'36","reported_seq":127,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.750793+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:44:14.750793+0000","last_peered":"2024-09-19T05:44:14.750793+0000","last_clean":"2024-09-19T05:44:14.750793+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:44:14.750793+0000","last_undegraded":"2024-09-19T05:44:14.750793+0000","last_fullsized":"2024-09-19T05:44:14.750793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":93,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.756913+0000","last_change":"2024-09-19T05:42:31.993052+0000","last_active":"2024-09-19T05:44:14.756913+0000","last_peered":"2024-09-19T05:44:14.756913+0000","last_clean":"2024-09-19T05:44:14.756913+0000","last_became_active":"2024-09-19T05:42:31.992662+0000","last_became_peered":"2024-09-19T05:42:31.992662+0000","last_unstale":"2024-09-19T05:44:14.756913+0000","last_undegraded":"2024-09-19T05:44:14.756913+0000","last_fullsized":"2024-09-19T05:44:14.756913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:24.103973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"61'26","reported_seq":118,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.753266+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:44:14.753266+0000","last_peered":"2024-09-19T05:44:14.753266+0000","last_clean":"2024-09-19T05:44:14.753266+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:44:14.753266+0000","last_undegraded":"2024-09-19T05:44:14.753266+0000","last_fullsized":"2024-09-19T05:44:14.753266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"59'3","reported_seq":97,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.859984+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:44:14.859984+0000","last_peered":"2024-09-19T05:44:14.859984+0000","last_clean":"2024-09-19T05:44:14.859984+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:44:14.859984+0000","last_undegraded":"2024-09-19T05:44:14.859984+0000","last_fullsized":"2024-09-19T05:44:14.859984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"62'13","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.913916+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:44:14.913916+0000","last_peered":"2024-09-19T05:44:14.913916+0000","last_clean":"2024-09-19T05:44:14.913916+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:44:14.913916+0000","last_undegraded":"2024-09-19T05:44:14.913916+0000","last_fullsized":"2024-09-19T05:44:14.913916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"62'34","reported_seq":126,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.944121+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:44:14.944121+0000","last_peered":"2024-09-19T05:44:14.944121+0000","last_clean":"2024-09-19T05:44:14.944121+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:44:14.944121+0000","last_undegraded":"2024-09-19T05:44:14.944121+0000","last_fullsized":"2024-09-19T05:44:14.944121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":114,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.834899+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:44:14.834899+0000","last_peered":"2024-09-19T05:44:14.834899+0000","last_clean":"2024-09-19T05:44:14.834899+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:44:14.834899+0000","last_undegraded":"2024-09-19T05:44:14.834899+0000","last_fullsized":"2024-09-19T05:44:14.834899+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:59.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"61'27","reported_seq":78,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.834737+0000","last_change":"2024-09-19T05:44:07.705198+0000","last_active":"2024-09-19T05:44:14.834737+0000","last_peered":"2024-09-19T05:44:14.834737+0000","last_clean":"2024-09-19T05:44:14.834737+0000","last_became_active":"2024-09-19T05:44:07.704821+0000","last_became_peered":"2024-09-19T05:44:07.704821+0000","last_unstale":"2024-09-19T05:44:14.834737+0000","last_undegraded":"2024-09-19T05:44:14.834737+0000","last_fullsized":"2024-09-19T05:44:14.834737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:59.539763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"59'3","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.906022+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:44:14.906022+0000","last_peered":"2024-09-19T05:44:14.906022+0000","last_clean":"2024-09-19T05:44:14.906022+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:44:14.906022+0000","last_undegraded":"2024-09-19T05:44:14.906022+0000","last_fullsized":"2024-09-19T05:44:14.906022+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:52.607339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"60'13","reported_seq":115,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767867+0000","last_change":"2024-09-19T05:43:19.550136+0000","last_active":"2024-09-19T05:44:14.767867+0000","last_peered":"2024-09-19T05:44:14.767867+0000","last_clean":"2024-09-19T05:44:14.767867+0000","last_became_active":"2024-09-19T05:43:19.549913+0000","last_became_peered":"2024-09-19T05:43:19.549913+0000","last_unstale":"2024-09-19T05:44:14.767867+0000","last_undegraded":"2024-09-19T05:44:14.767867+0000","last_fullsized":"2024-09-19T05:44:14.767867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:58.288341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":95,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.768022+0000","last_change":"2024-09-19T05:42:31.990572+0000","last_active":"2024-09-19T05:44:14.768022+0000","last_peered":"2024-09-19T05:44:14.768022+0000","last_clean":"2024-09-19T05:44:14.768022+0000","last_became_active":"2024-09-19T05:42:31.990149+0000","last_became_peered":"2024-09-19T05:42:31.990149+0000","last_unstale":"2024-09-19T05:44:14.768022+0000","last_undegraded":"2024-09-19T05:44:14.768022+0000","last_fullsized":"2024-09-19T05:44:14.768022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:38.136362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"60'17","reported_seq":109,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.767882+0000","last_change":"2024-09-19T05:42:32.943958+0000","last_active":"2024-09-19T05:44:14.767882+0000","last_peered":"2024-09-19T05:44:14.767882+0000","last_clean":"2024-09-19T05:44:14.767882+0000","last_became_active":"2024-09-19T05:42:32.943720+0000","last_became_peered":"2024-09-19T05:42:32.943720+0000","last_unstale":"2024-09-19T05:44:14.767882+0000","last_undegraded":"2024-09-19T05:44:14.767882+0000","last_fullsized":"2024-09-19T05:44:14.767882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:32.951716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":97,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-19T05:44:14.751440+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:44:14.751440+0000","last_peered":"2024-09-19T05:44:14.751440+0000","last_clean":"2024-09-19T05:44:14.751440+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:44:14.751440+0000","last_undegraded":"2024-09-19T05:44:14.751440+0000","last_fullsized":"2024-09-19T05:44:14.751440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":1065353216,"num_objects":254,"num_object_clones":0,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1229,"num_write_kb":4001792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134217728,"data_stored":2147483648,"data_compressed":106539904,"data_compressed_allocated":134217728,"data_compressed_original":2147483648,"omap_allocated":0,"internal_metadata":0},"log_size":1479,"ondisk_log_size":1479,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":104659,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":107,"num_write_kb":161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":140,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":207238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54352,"kb_used_data":19000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94317488,"statfs":{"total":96636764160,"available":96581107712,"internally_reserved":0,"allocated":19456000,"data_stored":298222411,"data_compressed":14895232,"data_compressed_allocated":18612224,"data_compressed_original":297795584,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":10172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":10416128,"data_stored":155500563,"data_compressed":7331328,"data_compressed_allocated":9699328,"data_compressed_original":155189248,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47492,"kb_used_data":16304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324348,"statfs":{"total":96636764160,"available":96588132352,"internally_reserved":0,"allocated":16695296,"data_stored":256163634,"data_compressed":12331392,"data_compressed_allocated":15990784,"data_compressed_original":255852544,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51664,"kb_used_data":20476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320176,"statfs":{"total":96636764160,"available":96583860224,"internally_reserved":0,"allocated":20967424,"data_stored":323863059,"data_compressed":15569215,"data_compressed_allocated":20230144,"data_compressed_original":323534848,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44788,"kb_used_data":13348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327052,"statfs":{"total":96636764160,"available":96590901248,"internally_reserved":0,"allocated":13668352,"data_stored":205934338,"data_compressed":9930368,"data_compressed_allocated":12845056,"data_compressed_original":205520896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542169,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52872,"kb_used_data":21684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318968,"statfs":{"total":96636764160,"available":96582623232,"internally_reserved":0,"allocated":22204416,"data_stored":344244600,"data_compressed":16247808,"data_compressed_allocated":21495808,"data_compressed_original":343932928,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542169,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45752,"kb_used_data":14308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326088,"statfs":{"total":96636764160,"available":96589914112,"internally_reserved":0,"allocated":14651392,"data_stored":223199640,"data_compressed":10710562,"data_compressed_allocated":13938688,"data_compressed_original":222887936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542170,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52876,"kb_used_data":21688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318964,"statfs":{"total":96636764160,"available":96582619136,"internally_reserved":0,"allocated":22208512,"data_stored":344244074,"data_compressed":16247808,"data_compressed_allocated":21495808,"data_compressed_original":343932928,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":103760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":102352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21495808,"data_stored":343932928,"data_compressed":17121408,"data_compressed_allocated":21495808,"data_compressed_original":343932928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13893632,"data_stored":222298112,"data_compressed":10944256,"data_compressed_allocated":13893632,"data_compressed_original":222298112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18612224,"data_stored":297795584,"data_compressed":14895232,"data_compressed_allocated":18612224,"data_compressed_original":297795584,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21495808,"data_stored":343932928,"data_compressed":17168000,"data_compressed_allocated":21495808,"data_compressed_original":343932928,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12845056,"data_stored":205520896,"data_compressed":10151680,"data_compressed_allocated":12845056,"data_compressed_original":205520896,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20185088,"data_stored":322961408,"data_compressed":15804544,"data_compressed_allocated":20185088,"data_compressed_original":322961408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":15990784,"data_stored":255852544,"data_compressed":12739072,"data_compressed_allocated":15990784,"data_compressed_original":255852544,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9699328,"data_stored":155189248,"data_compressed":7715712,"data_compressed_allocated":9699328,"data_compressed_original":155189248,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:44:18.952 INFO:tasks.thrashosds.thrasher:Setting 3.2a to [5, 0] 2024-09-19T05:44:18.952 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2a', '5', '0'] 2024-09-19T05:44:18.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2a 5 0 2024-09-19T05:44:18.953 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:19.032 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:19.031+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.081+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7feb4c0b32c0 con 0x7feb4c0caf40 2024-09-19T05:44:19.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.095+0000 7f558fa7b640 1 Processor -- start 2024-09-19T05:44:19.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.095+0000 7f558fa7b640 1 -- start start 2024-09-19T05:44:19.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558fa7b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5588154d60 0x7f55880c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558fa7b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f55880c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558fa7b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55880739e0 con 0x7f558806c040 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558fa7b640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f558815ed70 con 0x7f5588154d60 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558fa7b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f558815e160 con 0x7f558815b280 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558d817640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5588154d60 0x7f55880c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558d817640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5588154d60 0x7f55880c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33354/0 (socket says 172.21.15.2:33354) 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558d817640 1 -- 172.21.15.2:0/2327825554 learned_addr learned my addr 172.21.15.2:0/2327825554 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558e018640 1 --2- 172.21.15.2:0/2327825554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f55880c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558d817640 1 -- 172.21.15.2:0/2327825554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 msgr2=0x7f55880c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558d817640 1 --2- 172.21.15.2:0/2327825554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f55880c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558d817640 1 -- 172.21.15.2:0/2327825554 >> v1:172.21.15.123:6789/0 conn(0x7f558806c040 legacy=0x7f558815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.096+0000 7f558d817640 1 -- 172.21.15.2:0/2327825554 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558815e440 con 0x7f5588154d60 2024-09-19T05:44:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.097+0000 7f558d817640 1 --2- 172.21.15.2:0/2327825554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5588154d60 0x7f55880c8060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5584009870 tx=0x7f558402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1da1695e2dde5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.097+0000 7f558c815640 1 -- 172.21.15.2:0/2327825554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584037070 con 0x7f5588154d60 2024-09-19T05:44:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.097+0000 7f558c815640 1 -- 172.21.15.2:0/2327825554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5584032440 con 0x7f5588154d60 2024-09-19T05:44:19.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.097+0000 7f558c815640 1 -- 172.21.15.2:0/2327825554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584032810 con 0x7f5588154d60 2024-09-19T05:44:19.100 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:19.100 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:19.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 -- 172.21.15.2:0/2327825554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5588154d60 msgr2=0x7f55880c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 --2- 172.21.15.2:0/2327825554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5588154d60 0x7f55880c8060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5584009870 tx=0x7f558402a270 comp rx=0 tx=0).stop 2024-09-19T05:44:19.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 -- 172.21.15.2:0/2327825554 shutdown_connections 2024-09-19T05:44:19.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 --2- 172.21.15.2:0/2327825554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f55880c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 --2- 172.21.15.2:0/2327825554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5588154d60 0x7f55880c8060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 -- 172.21.15.2:0/2327825554 >> 172.21.15.2:0/2327825554 conn(0x7f558806f3d0 msgr2=0x7f55880707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 -- 172.21.15.2:0/2327825554 shutdown_connections 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.098+0000 7f558fa7b640 1 -- 172.21.15.2:0/2327825554 wait complete. 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.099+0000 7f558fa7b640 1 Processor -- start 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.099+0000 7f558fa7b640 1 -- start start 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558fa7b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f558806c040 0x7f5588159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558fa7b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5588154d60 0x7f55881442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558fa7b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f558815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558fa7b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f55880c8b80 con 0x7f558806c040 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558fa7b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f558815ea00 con 0x7f5588154d60 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558fa7b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f55880c8d00 con 0x7f558815b280 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f558806c040 0x7f5588159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f558806c040 0x7f5588159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33368/0 (socket says 172.21.15.2:33368) 2024-09-19T05:44:19.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 -- 172.21.15.2:0/3172232248 learned_addr learned my addr 172.21.15.2:0/3172232248 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558e018640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f558815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d016640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5588154d60 0x7f55881442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 -- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 msgr2=0x7f558815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f558815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 -- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5588154d60 msgr2=0x7f55881442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5588154d60 0x7f55881442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d817640 1 -- 172.21.15.2:0/3172232248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5588158b60 con 0x7f558806c040 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.100+0000 7f558d016640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5588154d60 0x7f55881442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.101+0000 7f558d817640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f558806c040 0x7f5588159bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f558402a2b0 tx=0x7f55840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.101+0000 7f557e7fc640 1 -- 172.21.15.2:0/3172232248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584041020 con 0x7f558806c040 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.101+0000 7f557e7fc640 1 -- 172.21.15.2:0/3172232248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55840306c0 con 0x7f558806c040 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.101+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55881f40f0 con 0x7f558806c040 2024-09-19T05:44:19.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.101+0000 7f557e7fc640 1 -- 172.21.15.2:0/3172232248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584037040 con 0x7f558806c040 2024-09-19T05:44:19.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.101+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55881f46b0 con 0x7f558806c040 2024-09-19T05:44:19.105 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.102+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55880735c0 con 0x7f558806c040 2024-09-19T05:44:19.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.102+0000 7f557e7fc640 1 -- 172.21.15.2:0/3172232248 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5584044020 con 0x7f558806c040 2024-09-19T05:44:19.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.103+0000 7f557e7fc640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f556c06c2b0 0x7f556c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:19.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.103+0000 7f557e7fc640 1 -- 172.21.15.2:0/3172232248 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5770+0+0 (secure 0 0 0) 0x7f55840abb80 con 0x7f558806c040 2024-09-19T05:44:19.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.103+0000 7f558d016640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f556c06c2b0 0x7f556c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:19.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.104+0000 7f558d016640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f556c06c2b0 0x7f556c06e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55881e8200 tx=0x7f5578009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:19.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:19.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.110+0000 7f557e7fc640 1 -- 172.21.15.2:0/3172232248 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f558404b030 con 0x7f558806c040 2024-09-19T05:44:19.154 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:19.153+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:19.218 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:19.217+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:19.227 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:19.227 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:19.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:19.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.296+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "3.2a", "id": [5, 0]} v 0) -- 0x7f558806bc10 con 0x7f558806c040 2024-09-19T05:44:19.319 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:19.318+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:19.379 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:19.379 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:19.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:19.432 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:19.431+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:19.492 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:19.492 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:19.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:19.520 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:19.519+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:19.620 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:19.619+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:19.644 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:19.644 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:19.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:19.722 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:19.720+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:19.757 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:19.757 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:19.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.761+0000 7feb3affd640 1 -- 172.21.15.2:0/1100306674 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v65) ==== 91+0+0 (secure 0 0 0) 0x7feb3c05d390 con 0x7feb4c0caf40 2024-09-19T05:44:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.761+0000 7f557e7fc640 1 -- 172.21.15.2:0/3172232248 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "3.2a", "id": [5, 0]}]=0 set 3.2a pg_upmap mapping to [5,0] v65) ==== 124+0+0 (secure 0 0 0) 0x7f558407a340 con 0x7f558806c040 2024-09-19T05:44:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:44:19.763 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.2a pg_upmap mapping to [5,0] 2024-09-19T05:44:19.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:19.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb2c06c2e0 msgr2=0x7feb2c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb2c06c2e0 0x7feb2c06e7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb40034000 tx=0x7feb4002e000 comp rx=0 tx=0).stop 2024-09-19T05:44:19.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb4c0caf40 msgr2=0x7feb4c0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb4c0caf40 0x7feb4c0b5370 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00c530 tx=0x7feb3c00c9f0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 shutdown_connections 2024-09-19T05:44:19.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7feb2c06c2e0 0x7feb2c06e7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb4c155000 0x7feb4c1e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb4c154c20 0x7feb4c1e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 --2- 172.21.15.2:0/1100306674 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feb4c0caf40 0x7feb4c0b5370 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 >> 172.21.15.2:0/1100306674 conn(0x7feb4c06d0e0 msgr2=0x7feb4c150e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:19.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f556c06c2b0 msgr2=0x7f556c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f556c06c2b0 0x7f556c06e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55881e8200 tx=0x7f5578009040 comp rx=0 tx=0).stop 2024-09-19T05:44:19.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f558806c040 msgr2=0x7f5588159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:19.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f558806c040 0x7f5588159bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f558402a2b0 tx=0x7f55840046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 shutdown_connections 2024-09-19T05:44:19.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f556c06c2b0 0x7f556c06e770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5588154d60 0x7f55881442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 shutdown_connections 2024-09-19T05:44:19.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f558815b280 0x7f558815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 --2- 172.21.15.2:0/3172232248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f558806c040 0x7f5588159bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:19.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 >> 172.21.15.2:0/3172232248 conn(0x7f558806f3d0 msgr2=0x7f55880c8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:19.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7feb50c7b640 1 -- 172.21.15.2:0/1100306674 wait complete. 2024-09-19T05:44:19.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 shutdown_connections 2024-09-19T05:44:19.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:19.766+0000 7f558fa7b640 1 -- 172.21.15.2:0/3172232248 wait complete. 2024-09-19T05:44:19.850 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:19.849+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:19.888 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:19.888 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 re-readers = 8166992.50 kB/sec 2024-09-19T05:44:19.888 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 re-readers = 8151250.74 kB/sec 2024-09-19T05:44:19.888 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 8166992.50 kB/sec 2024-09-19T05:44:19.888 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 8166992.50 kB/sec 2024-09-19T05:44:19.888 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 8166992.50 kB/sec 2024-09-19T05:44:19.888 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 1048576.00 kB 2024-09-19T05:44:19.909 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:19.910 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:19.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:19.960 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:19.958+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:20.025 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:20.024+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:20.062 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:20.063 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:20.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:20.125 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:20.124+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:20.173 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:20.174 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:20.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:20.227 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:20.226+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:20.285 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:20.285 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:20.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:20.339 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:20.337+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:20.397 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:20.397 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:20.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:20.447 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:20.445+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:20.530 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:20.529+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:20.550 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:20.550 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:20.558 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:20.631 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:20.631+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:20.682 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:20.702 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:20.702 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:20.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:20.732 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:20.731+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:20.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.799+0000 7efe81903640 1 Processor -- start 2024-09-19T05:44:20.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.799+0000 7efe81903640 1 -- start start 2024-09-19T05:44:20.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.800+0000 7efe81903640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c0cd4e0 0x7efe7c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.800+0000 7efe81903640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe7c15aa90 0x7efe7c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.800+0000 7efe81903640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe7c15ebe0 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.800+0000 7efe81903640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efe7c15eec0 con 0x7efe7c15aa90 2024-09-19T05:44:20.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.800+0000 7efe81903640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efe7c0b2e00 con 0x7efe7c0cd4e0 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7affd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c0cd4e0 0x7efe7c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7affd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c0cd4e0 0x7efe7c0b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:47786/0 (socket says 172.21.15.2:47786) 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7affd640 1 -- 172.21.15.2:0/4028637732 learned_addr learned my addr 172.21.15.2:0/4028637732 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7b7fe640 1 --1- 172.21.15.2:0/4028637732 >> v1:172.21.15.123:6789/0 conn(0x7efe7c0cd8c0 0x7efe7c15df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:59376/0 (socket says 172.21.15.2:59376) 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/4028637732 >> v2:172.21.15.123:3300/0 conn(0x7efe7c15aa90 0x7efe7c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7affd640 1 -- 172.21.15.2:0/4028637732 >> v2:172.21.15.123:3300/0 conn(0x7efe7c15aa90 msgr2=0x7efe7c0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7affd640 1 --2- 172.21.15.2:0/4028637732 >> v2:172.21.15.123:3300/0 conn(0x7efe7c15aa90 0x7efe7c0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7affd640 1 -- 172.21.15.2:0/4028637732 >> v1:172.21.15.123:6789/0 conn(0x7efe7c0cd8c0 legacy=0x7efe7c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe79ffb640 1 -- 172.21.15.2:0/4028637732 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1198746187 0 0) 0x7efe7c15ebe0 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.801+0000 7efe7affd640 1 -- 172.21.15.2:0/4028637732 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c0b30e0 con 0x7efe7c0cd4e0 2024-09-19T05:44:20.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.802+0000 7efe7affd640 1 --2- 172.21.15.2:0/4028637732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c0cd4e0 0x7efe7c0b3bc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009870 tx=0x7efe6c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a91ad8439ff6db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.802+0000 7efe79ffb640 1 -- 172.21.15.2:0/4028637732 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c037070 con 0x7efe7c0cd4e0 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.802+0000 7efe79ffb640 1 -- 172.21.15.2:0/4028637732 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c032440 con 0x7efe7c0cd4e0 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.802+0000 7efe79ffb640 1 -- 172.21.15.2:0/4028637732 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c032810 con 0x7efe7c0cd4e0 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 -- 172.21.15.2:0/4028637732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c0cd4e0 msgr2=0x7efe7c0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 --2- 172.21.15.2:0/4028637732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c0cd4e0 0x7efe7c0b3bc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009870 tx=0x7efe6c02a270 comp rx=0 tx=0).stop 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 -- 172.21.15.2:0/4028637732 shutdown_connections 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 --2- 172.21.15.2:0/4028637732 >> v2:172.21.15.123:3300/0 conn(0x7efe7c15aa90 0x7efe7c0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 --2- 172.21.15.2:0/4028637732 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c0cd4e0 0x7efe7c0b3bc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 -- 172.21.15.2:0/4028637732 >> 172.21.15.2:0/4028637732 conn(0x7efe7c067b30 msgr2=0x7efe7c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 -- 172.21.15.2:0/4028637732 shutdown_connections 2024-09-19T05:44:20.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.803+0000 7efe81903640 1 -- 172.21.15.2:0/4028637732 wait complete. 2024-09-19T05:44:20.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.804+0000 7efe81903640 1 Processor -- start 2024-09-19T05:44:20.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.805+0000 7efe81903640 1 -- start start 2024-09-19T05:44:20.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.805+0000 7efe81903640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c0cd4e0 0x7efe7c14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.805+0000 7efe81903640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c0cd8c0 0x7efe7c1f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe81903640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c15aa90 0x7efe7c158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe81903640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c15e640 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe81903640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c0b36a0 con 0x7efe7c0cd4e0 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe81903640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efe7c15e7c0 con 0x7efe7c15aa90 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c0cd8c0 0x7efe7c1f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c0cd8c0 0x7efe7c1f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33378/0 (socket says 172.21.15.2:33378) 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 -- 172.21.15.2:0/2492902294 learned_addr learned my addr 172.21.15.2:0/2492902294 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c0cd4e0 0x7efe7c14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c15aa90 msgr2=0x7efe7c158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c15aa90 0x7efe7c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c15aa90 0x7efe7c158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c0cd4e0 msgr2=0x7efe7c14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c0cd4e0 0x7efe7c14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c151d30 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe7c0cd4e0 0x7efe7c14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe7c15aa90 0x7efe7c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.806+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c0cd8c0 0x7efe7c1f3890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efe7000d5f0 tx=0x7efe7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.807+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe700125f0 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.807+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe70019070 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.807+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe700129d0 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.807+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c1f4250 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.807+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c150610 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.808+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7efe7c1515a0 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.808+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe70012b70 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.809+0000 7efe63fff640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe6806c2b0 0x7efe6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.809+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5799+0+0 (secure 0 0 0) 0x7efe7008b910 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.809+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v66) ==== 54+0+16 (secure 0 0 0) 0x7efe70090240 con 0x7efe7c0cd8c0 2024-09-19T05:44:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.809+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe6806c2b0 0x7efe6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.810+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe6806c2b0 0x7efe6806e770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe6c034000 tx=0x7efe6c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.810+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7efe7c150c80 0x7efe7c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.810+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe7c158de0 con 0x7efe7c150c80 2024-09-19T05:44:20.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.810+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7efe7c150c80 0x7efe7c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.811+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7efe7c150c80 0x7efe7c1588a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.812+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c158de0 con 0x7efe7c150c80 2024-09-19T05:44:20.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.828+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c0681f0 con 0x7efe7c150c80 2024-09-19T05:44:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.829+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c0681f0 con 0x7efe7c150c80 2024-09-19T05:44:20.831 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:20.831 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:20.831 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.830+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7efe7c1f3dd0 0x7efe7c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.830+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7efe7c15a1e0 con 0x7efe7c1f3dd0 2024-09-19T05:44:20.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.830+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7efe7c1f3dd0 0x7efe7c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.830+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7efe7c1f3dd0 0x7efe7c1529c0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.832 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.831+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c15a1e0 con 0x7efe7c1f3dd0 2024-09-19T05:44:20.833 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:20.832+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:20.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.848+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c14f3c0 con 0x7efe7c1f3dd0 2024-09-19T05:44:20.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.851+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c14f3c0 con 0x7efe7c1f3dd0 2024-09-19T05:44:20.852 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:20.852 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:20.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.851+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efe7c153410 0x7efe7c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.851+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7efe7c062b70 con 0x7efe7c153410 2024-09-19T05:44:20.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.851+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efe7c153410 0x7efe7c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.851+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efe7c153410 0x7efe7c062630 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.853 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:20.853 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:20.853 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:20.853 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:20.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.852+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c062b70 con 0x7efe7c153410 2024-09-19T05:44:20.862 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:20.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.866+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c062fa0 con 0x7efe7c153410 2024-09-19T05:44:20.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.868+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c062fa0 con 0x7efe7c153410 2024-09-19T05:44:20.869 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:20.870 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:20.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.868+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efe7c0c7e20 0x7efe7c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.868+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7efe7c063570 con 0x7efe7c0c7e20 2024-09-19T05:44:20.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.868+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efe7c0c7e20 0x7efe7c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.868+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efe7c0c7e20 0x7efe7c0c8240 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.869+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c063570 con 0x7efe7c0c7e20 2024-09-19T05:44:20.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.885+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c02c640 con 0x7efe7c0c7e20 2024-09-19T05:44:20.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.886+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c02c640 con 0x7efe7c0c7e20 2024-09-19T05:44:20.888 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.886+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7efe7c02d120 0x7efe7c02d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.886+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7efe7c02c640 con 0x7efe7c02d120 2024-09-19T05:44:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.887+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7efe7c02d120 0x7efe7c02d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.887+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7efe7c02d120 0x7efe7c02d5a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.888+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c02c640 con 0x7efe7c02d120 2024-09-19T05:44:20.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.902+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c02c640 con 0x7efe7c02d120 2024-09-19T05:44:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.904+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c02c640 con 0x7efe7c02d120 2024-09-19T05:44:20.905 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.904+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7efe7c02c640 0x7efe7c206ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.904+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7efe7c209c90 con 0x7efe7c02c640 2024-09-19T05:44:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.904+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7efe7c02c640 0x7efe7c206ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.905+0000 7efe7a7fc640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7efe7c02c640 0x7efe7c206ec0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.905+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c209c90 con 0x7efe7c02c640 2024-09-19T05:44:20.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.919+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c207400 con 0x7efe7c02c640 2024-09-19T05:44:20.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.921+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c207400 con 0x7efe7c02c640 2024-09-19T05:44:20.922 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.921+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7efe7c207910 0x7efe7c209d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.921+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7efe7c20cd60 con 0x7efe7c207910 2024-09-19T05:44:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.921+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7efe7c207910 0x7efe7c209d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.922+0000 7efe7b7fe640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7efe7c207910 0x7efe7c209d00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.922+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c20cd60 con 0x7efe7c207910 2024-09-19T05:44:20.934 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:20.932+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:20.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.936+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c20baa0 con 0x7efe7c207910 2024-09-19T05:44:20.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.938+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c20baa0 con 0x7efe7c207910 2024-09-19T05:44:20.939 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.938+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efe7c20c1c0 0x7efe7c20c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.938+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7efe7c2101b0 con 0x7efe7c20c1c0 2024-09-19T05:44:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.938+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efe7c20c1c0 0x7efe7c20c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:20.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.939+0000 7efe7affd640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efe7c20c1c0 0x7efe7c20c620 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:20.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.940+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c2101b0 con 0x7efe7c20c1c0 2024-09-19T05:44:20.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.953+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efe7c20efb0 con 0x7efe7c20c1c0 2024-09-19T05:44:20.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe63fff640 1 -- 172.21.15.2:0/2492902294 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efe7c20efb0 con 0x7efe7c20c1c0 2024-09-19T05:44:20.956 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:20.957 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:20.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7efe7c150c80 msgr2=0x7efe7c1588a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7efe7c150c80 0x7efe7c1588a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7efe7c1f3dd0 msgr2=0x7efe7c1529c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7efe7c1f3dd0 0x7efe7c1529c0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efe7c153410 msgr2=0x7efe7c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7efe7c153410 0x7efe7c062630 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efe7c0c7e20 msgr2=0x7efe7c0c8240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7efe7c0c7e20 0x7efe7c0c8240 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7efe7c02d120 msgr2=0x7efe7c02d5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7efe7c02d120 0x7efe7c02d5a0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe7b7fe640 1 -- 172.21.15.2:0/2492902294 reap_dead start 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe7b7fe640 1 -- 172.21.15.2:0/2492902294 reap_dead start 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7efe7c02c640 msgr2=0x7efe7c206ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7efe7c02c640 0x7efe7c206ec0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7efe7c207910 msgr2=0x7efe7c209d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7efe7c207910 0x7efe7c209d00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efe7c20c1c0 msgr2=0x7efe7c20c620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7efe7c20c1c0 0x7efe7c20c620 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe6806c2b0 msgr2=0x7efe6806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7efe6806c2b0 0x7efe6806e770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe6c034000 tx=0x7efe6c02e000 comp rx=0 tx=0).stop 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c0cd8c0 msgr2=0x7efe7c1f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 --2- 172.21.15.2:0/2492902294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe7c0cd8c0 0x7efe7c1f3890 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efe7000d5f0 tx=0x7efe7000dab0 comp rx=0 tx=0).stop 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe7b7fe640 1 -- 172.21.15.2:0/2492902294 reap_dead start 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 shutdown_connections 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.955+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 >> 172.21.15.2:0/2492902294 conn(0x7efe7c067b30 msgr2=0x7efe7c0cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.956+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 shutdown_connections 2024-09-19T05:44:20.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:20.956+0000 7efe81903640 1 -- 172.21.15.2:0/2492902294 wait complete. 2024-09-19T05:44:20.977 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:20.977 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:20.978 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:20.978 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:20.978 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:20.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:21.072 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:21.071+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:12 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.849195225999999, 2024-09-19T05:44:21.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.301330818, 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 12 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285078+0000", 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8979999999999998e-06 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288881+0000", 2024-09-19T05:44:21.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038028900000000002 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288908+0000", 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6831000000000001e-05 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288922+0000", 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4647999999999999e-05 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289034+0000", 2024-09-19T05:44:21.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011129 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289158+0000", 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012398199999999999 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289283+0000", 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012596100000000001 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302442+0000", 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01315887 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586307+0000", 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28386498300000002 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586357+0000", 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9406000000000001e-05 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586405+0000", 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8059000000000002e-05 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:16 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.835071486, 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28733763699999998, 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 16 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:21.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299338+0000", 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013993499999999999 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305742+0000", 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064047180000000002 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305795+0000", 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2988000000000002e-05 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305808+0000", 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.307e-05 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305883+0000", 2024-09-19T05:44:21.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4265999999999997e-05 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306078+0000", 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195281 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306298+0000", 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000219662 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.319662+0000", 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0133641 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586451+0000", 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26678922700000002 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586494+0000", 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3102999999999997e-05 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586535+0000", 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1287e-05 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:36 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.817733099000002, 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54259264799999996, 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.170 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316539+0000", 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1310000000000002e-06 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326313+0000", 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0097740320000000002 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326341+0000", 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8178e-05 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326354+0000", 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2767e-05 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326454+0000", 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9741999999999995e-05 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326598+0000", 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000144454 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326740+0000", 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014189899999999999 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.381404+0000", 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054663549999999998 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859031+0000", 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47762663900000002 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859074+0000", 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3578999999999997e-05 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859129+0000", 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4676999999999998e-05 2024-09-19T05:44:21.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:40 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.807843752, 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29103886899999998, 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 40 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326428+0000", 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9160000000000001e-06 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343603+0000", 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017174545999999999 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343619+0000", 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6260999999999999e-05 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343630+0000", 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0774999999999999e-05 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343729+0000", 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8694000000000001e-05 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343846+0000", 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117588 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343969+0000", 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012271600000000001 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.376370+0000", 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032401093999999998 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617348+0000", 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24097808100000001 2024-09-19T05:44:21.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617405+0000", 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6577999999999999e-05 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617464+0000", 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9620000000000002e-05 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:101 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.667158050000001, 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.392178684, 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 101 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467114+0000", 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.269e-06 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.477820+0000", 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010705696000000001 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.500766+0000", 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022946191000000001 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508150+0000", 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073836930000000002 2024-09-19T05:44:21.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508253+0000", 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103222 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508541+0000", 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028815600000000002 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508725+0000", 2024-09-19T05:44:21.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183766 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.543910+0000", 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035184897 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859196+0000", 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31528614500000002 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859243+0000", 2024-09-19T05:44:21.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6727000000000002e-05 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859290+0000", 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6922000000000002e-05 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:268 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.534416916000001, 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37283424700000001, 2024-09-19T05:44:21.187 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 268 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599857+0000", 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.228e-06 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604581+0000", 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047240240000000003 2024-09-19T05:44:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604601+0000", 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0738e-05 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604613+0000", 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621794+0000", 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017181279000000001 2024-09-19T05:44:21.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621899+0000", 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104218 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.622059+0000", 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015997200000000001 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629055+0000", 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069966560000000004 2024-09-19T05:44:21.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972534+0000", 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34347880400000003 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972610+0000", 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6398999999999998e-05 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972687+0000", 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6038999999999995e-05 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:272 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.517864964000001, 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40633524799999998, 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 272 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616407+0000", 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.993e-06 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631509+0000", 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015101787 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631525+0000", 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5829999999999999e-05 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631535+0000", 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0224000000000001e-05 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631631+0000", 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5696e-05 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631752+0000", 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000121605 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.632659+0000", 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090671499999999999 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.708071+0000", 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075412347000000005 2024-09-19T05:44:21.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022627+0000", 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31455541599999998 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022684+0000", 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7543000000000001e-05 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022739+0000", 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5092e-05 2024-09-19T05:44:21.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:292 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.486602214000001, 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53039040900000001, 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 292 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647670+0000", 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657999+0000", 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010329481 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667975+0000", 2024-09-19T05:44:21.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0099757179999999997 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667984+0000", 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8470000000000002e-06 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668049+0000", 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4999000000000006e-05 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668145+0000", 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6514000000000005e-05 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668295+0000", 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015011399999999999 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.729044+0000", 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060748914000000001 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.177943+0000", 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44889912500000001 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178002+0000", 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8829000000000002e-05 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178057+0000", 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5284999999999997e-05 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:296 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.462422103000002, 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43521636699999999, 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 296 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671851+0000", 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.659e-06 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688220+0000", 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016369595000000001 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688250+0000", 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9499e-05 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688262+0000", 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2031999999999999e-05 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688539+0000", 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027760399999999998 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688662+0000", 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122031 2024-09-19T05:44:21.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688813+0000", 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015106399999999999 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.765112+0000", 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076299661000000005 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106912+0000", 2024-09-19T05:44:21.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34180015600000002 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106973+0000", 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0927999999999997e-05 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.107063+0000", 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0137999999999997e-05 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:288 3.31 e54/20 3:8d4d39bf:::10000000002.0000001d:head v 54'5, mlcod=53'3)", 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.436351304999999, 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48371873900000001, 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697920+0000", 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7719999999999999e-06 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099856+0000", 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40193617399999998 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099878+0000", 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2306999999999998e-05 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099891+0000", 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132365+0000", 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032473872000000001 2024-09-19T05:44:21.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132403+0000", 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8396999999999998e-05 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181593+0000", 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049190325 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181637+0000", 2024-09-19T05:44:21.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3368000000000002e-05 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:337 3.34 3:2c8ffcca:::10000000002.0000004e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.356274605999999, 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24486184699999999, 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 337 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777997+0000", 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.17e-06 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781851+0000", 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038539429999999999 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781870+0000", 2024-09-19T05:44:21.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9012e-05 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781882+0000", 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1864e-05 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781987+0000", 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105691 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782084+0000", 2024-09-19T05:44:21.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7072999999999997e-05 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782457+0000", 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037225700000000002 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799269+0000", 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016812247999999998 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022778+0000", 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22350931499999999 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022823+0000", 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4502999999999997e-05 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022856+0000", 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3770999999999999e-05 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:357 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.29418961, 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33810015799999998, 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 357 2024-09-19T05:44:21.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840082+0000", 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2029999999999999e-06 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842172+0000", 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020904890000000001 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842193+0000", 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1058999999999999e-05 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842207+0000", 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3444e-05 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848376+0000", 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061690909999999998 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848486+0000", 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109979 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848644+0000", 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015821699999999999 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.894080+0000", 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045435709999999997 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178104+0000", 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28402411700000002 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178148+0000", 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4492e-05 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178180+0000", 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1356999999999998e-05 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.226 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:363 3.3f 3:ff5df6dc:::10000000002.00000068:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.283888257000001, 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32356367800000002, 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 363 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850383+0000", 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0600000000000002e-06 2024-09-19T05:44:21.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862283+0000", 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011899873 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862330+0000", 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862352+0000", 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1141e-05 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.877929+0000", 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015577153 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878045+0000", 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011612000000000001 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878204+0000", 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158832 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896926+0000", 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018722714000000001 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173814+0000", 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27688780099999999 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173871+0000", 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6907000000000003e-05 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173945+0000", 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3530999999999998e-05 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.232 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:365 3.c 3:3214729d:::10000000002.0000006a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.245867857, 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27874868600000002, 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 365 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888405+0000", 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7620000000000001e-06 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892569+0000", 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041637540000000004 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892661+0000", 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2134999999999998e-05 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892675+0000", 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4173e-05 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908596+0000", 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015920815000000001 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908712+0000", 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011617599999999999 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908878+0000", 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165935 2024-09-19T05:44:21.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972488+0000", 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063609938000000005 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167039+0000", 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19455101699999999 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167097+0000", 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7776999999999999e-05 2024-09-19T05:44:21.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167150+0000", 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3204000000000001e-05 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:372 3.34 3:2ebe227a:::10000000002.00000071:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.232164953000002, 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.264918877, 2024-09-19T05:44:21.238 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 372 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902109+0000", 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6990000000000002e-06 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910547+0000", 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084382120000000005 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910566+0000", 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9111e-05 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910579+0000", 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2330999999999999e-05 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923326+0000", 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012747596 2024-09-19T05:44:21.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923501+0000", 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000174802 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923775+0000", 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027407199999999999 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023005+0000", 2024-09-19T05:44:21.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099229554999999997 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.157725+0000", 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13472034199999999 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.158104+0000", 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037887499999999998 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167023+0000", 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089192820000000006 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:315 3.1c e54/38 3:394f8a26:::10000000002.00000038:head v 54'8, mlcod=53'5)", 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.228744548000002, 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28226446599999999, 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.244 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905527+0000", 2024-09-19T05:44:21.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5739999999999998e-06 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939034+0000", 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033507185000000002 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939067+0000", 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2186e-05 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939084+0000", 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6982000000000001e-05 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972466+0000", 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033381985000000003 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972491+0000", 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5789e-05 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187219+0000", 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21472724700000001 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187789+0000", 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000570518 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:373 3.36 3:6d44e697:::10000000002.00000072:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.248 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.223661391, 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29574129399999999, 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 373 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910611+0000", 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5779999999999999e-06 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.919588+0000", 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089768220000000006 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938096+0000", 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018507920000000001 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938107+0000", 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0488e-05 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944322+0000", 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062149900000000001 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944546+0000", 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022402300000000001 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944884+0000", 2024-09-19T05:44:21.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033806299999999999 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.074757+0000", 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.129873715 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206239+0000", 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13148132000000001 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206302+0000", 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3046000000000006e-05 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206349+0000", 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7329e-05 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:348 3.1c e54/38 3:3aa14ed1:::10000000002.00000059:head v 54'9, mlcod=53'5)", 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:21.254 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.18959813, 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24318706500000001, 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:21.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944672+0000", 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1790000000000001e-06 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947488+0000", 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028159130000000002 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947504+0000", 2024-09-19T05:44:21.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5367e-05 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947511+0000", 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3300000000000001e-06 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006421+0000", 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058910000999999997 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006451+0000", 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0576e-05 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187796+0000", 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18134434799999999 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187858+0000", 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2351e-05 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:395 3.1c e54/38 3:384ad1d9:::10000000002.00000088:head v 54'10, mlcod=53'5)", 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.186741242, 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24038785900000001, 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947528+0000", 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947529+0000", 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7400000000000002e-07 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959658+0000", 2024-09-19T05:44:21.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012129493999999999 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959675+0000", 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6996000000000001e-05 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.959687+0000", 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2143999999999999e-05 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023041+0000", 2024-09-19T05:44:21.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063353484000000002 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023062+0000", 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1171000000000001e-05 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187862+0000", 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.164799521 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187916+0000", 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4175000000000002e-05 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:393 3.36 3:6ccb00e9:::10000000002.00000086:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.147276384000001, 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24664645599999999, 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 393 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986998+0000", 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9899999999999997e-06 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995844+0000", 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0088464340000000002 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995859+0000", 2024-09-19T05:44:21.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4329e-05 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995866+0000", 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037368+0000", 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041501563999999998 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037440+0000", 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1773999999999994e-05 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037555+0000", 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011548399999999999 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117549+0000", 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079993623 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233527+0000", 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11597800699999999 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233591+0000", 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3789999999999997e-05 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233639+0000", 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8606999999999997e-05 2024-09-19T05:44:21.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.269 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.269 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.269 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:21.269 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:21.271 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:21.156+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:21.271 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:21.269+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:21.371 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:21.370+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:21.377 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:21.377 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:21.377 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:21.378 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:21.386 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:21.473 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:21.471+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:21.535 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:21.535 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:21.535 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:21.536 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:21.536 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:21.544 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:21.573 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:21.572+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:21.654 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:21.654 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:21.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:21.654 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:21.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.772390347, 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:44:21.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:44:21.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:44:21.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:44:21.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:7 3.2a 3:557f550e:::10000000002.00000004:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.368942042, 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27494626900000002, 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 7 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282495+0000", 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4609999999999998e-06 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286920+0000", 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044248229999999996 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286972+0000", 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1659000000000001e-05 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286986+0000", 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287117+0000", 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131328 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287358+0000", 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024141700000000001 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287543+0000", 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018436700000000001 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299025+0000", 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011482327000000001 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557305+0000", 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25828001099999998 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557360+0000", 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5074000000000002e-05 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557439+0000", 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9018999999999997e-05 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.666 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:92 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.235794024, 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37051118500000002, 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415644+0000", 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.422983+0000", 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073390779999999997 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423020+0000", 2024-09-19T05:44:21.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7660000000000002e-05 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423036+0000", 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5359999999999999e-05 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423261+0000", 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225217 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423458+0000", 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196555 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423737+0000", 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027922100000000002 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.435447+0000", 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011709855999999999 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.785961+0000", 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35051446400000003 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786086+0000", 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012537599999999999 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786152+0000", 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691000000000005e-05 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:104 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.221567702000002, 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28556600799999998, 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104 2024-09-19T05:44:21.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429870+0000", 2024-09-19T05:44:21.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-06 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446619+0000", 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016749447000000001 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446650+0000", 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0218000000000001e-05 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446666+0000", 2024-09-19T05:44:21.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6920999999999999e-05 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448857+0000", 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021902699999999998 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449011+0000", 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015441700000000001 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449168+0000", 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156527 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489828+0000", 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040659908000000002 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715285+0000", 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22545759500000001 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715345+0000", 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0269999999999997e-05 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715433+0000", 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7814999999999999e-05 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:108 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.678 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.204716685000001, 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26890963299999998, 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446721+0000", 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8250000000000001e-06 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451943+0000", 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052222140000000002 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451961+0000", 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8117e-05 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451970+0000", 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7420000000000003e-06 2024-09-19T05:44:21.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473034+0000", 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021064282 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473242+0000", 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020716699999999999 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473446+0000", 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204396 2024-09-19T05:44:21.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489832+0000", 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016385994000000001 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715506+0000", 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22567427700000001 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715577+0000", 2024-09-19T05:44:21.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0548000000000003e-05 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715628+0000", 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070999999999999e-05 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:139 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.187905278999999, 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.328123891, 2024-09-19T05:44:21.684 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463532+0000", 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0339999999999999e-06 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468331+0000", 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047995099999999999 2024-09-19T05:44:21.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468365+0000", 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3667000000000001e-05 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468382+0000", 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6651e-05 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468915+0000", 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053293299999999997 2024-09-19T05:44:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469199+0000", 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028406900000000002 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469482+0000", 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028390000000000002 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.485026+0000", 2024-09-19T05:44:21.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015543509 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791538+0000", 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30651197400000002 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791602+0000", 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3780000000000003e-05 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791654+0000", 2024-09-19T05:44:21.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1863999999999997e-05 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:148 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.169455245000002, 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34695000399999998, 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 148 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481982+0000", 2024-09-19T05:44:21.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6309999999999999e-06 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489765+0000", 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077821310000000003 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489778+0000", 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3604999999999999e-05 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489787+0000", 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489875+0000", 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8039000000000002e-05 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490101+0000", 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022636299999999999 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490426+0000", 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032453799999999997 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.506325+0000", 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015899149000000001 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828797+0000", 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32247239900000002 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828865+0000", 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7521999999999995e-05 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828930+0000", 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4795999999999994e-05 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:156 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.695 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.161600038, 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26935146300000001, 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:21.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489837+0000", 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2689999999999998e-06 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495152+0000", 2024-09-19T05:44:21.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053150209999999996 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495356+0000", 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203537 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495373+0000", 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496071+0000", 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069810200000000003 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496246+0000", 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017547299999999999 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496469+0000", 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022283200000000001 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.540840+0000", 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044371318999999999 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759082+0000", 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21824185500000001 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759137+0000", 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645000000000003e-05 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759186+0000", 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9499000000000002e-05 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:185 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.148920193999999, 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32688410400000001, 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 185 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502518+0000", 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.509973+0000", 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074553919999999999 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510031+0000", 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8356999999999997e-05 2024-09-19T05:44:21.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510046+0000", 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932e-05 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510135+0000", 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8548000000000007e-05 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510886+0000", 2024-09-19T05:44:21.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075159700000000001 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511079+0000", 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019261500000000001 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532794+0000", 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021714989000000001 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829312+0000", 2024-09-19T05:44:21.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29651747699999997 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829360+0000", 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8396000000000002e-05 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829399+0000", 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8961999999999997e-05 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:264 3.1c 3:3ae0713d:::10000000002.00000005:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.059001483999999, 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29060550499999999, 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 264 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592436+0000", 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5629999999999999e-06 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595027+0000", 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025912600000000002 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595055+0000", 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8164999999999999e-05 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595069+0000", 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3953e-05 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595156+0000", 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6981e-05 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595260+0000", 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010363 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595417+0000", 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015732100000000001 2024-09-19T05:44:21.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.605722+0000", 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010305099999999999 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882929+0000", 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27720663499999998 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882988+0000", 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9237000000000002e-05 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.883039+0000", 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0659999999999999e-05 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:315 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.020041163999998, 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55644120600000002, 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 315 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631396+0000", 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5490000000000001e-06 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636257+0000", 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048607859999999998 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636331+0000", 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3992000000000006e-05 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636355+0000", 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3813999999999999e-05 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636445+0000", 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0180999999999996e-05 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636569+0000", 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123991 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636849+0000", 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000279664 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659414+0000", 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022565053000000002 2024-09-19T05:44:21.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187715+0000", 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52830067000000003 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187783+0000", 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8509000000000004e-05 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187835+0000", 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1996999999999997e-05 2024-09-19T05:44:21.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:347 3.2a 3:5747858d:::10000000002.00000058:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.011968337999999, 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48789592399999998, 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 347 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639470+0000", 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.145e-06 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644022+0000", 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045525940000000001 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646323+0000", 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023007879999999998 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646342+0000", 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8766999999999998e-05 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646435+0000", 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3394999999999996e-05 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646564+0000", 2024-09-19T05:44:21.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012848800000000001 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646738+0000", 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017418499999999999 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.741685+0000", 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094946825999999998 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127231+0000", 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.385546526 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127301+0000", 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9741999999999997e-05 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127363+0000", 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1468000000000002e-05 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:348 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.005065848000001, 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54159870300000001, 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 348 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646372+0000", 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9110000000000002e-06 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.678772+0000", 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032399869999999997 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681572+0000", 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002799714 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681592+0000", 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0298000000000001e-05 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682777+0000", 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011847190000000001 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682909+0000", 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000132146 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.683102+0000", 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019280499999999999 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.702631+0000", 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019529271000000001 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187923+0000", 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48529252 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187948+0000", 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5004999999999999e-05 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187968+0000", 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9443999999999999e-05 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.729 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:360 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.961689433, 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43778335400000001, 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 360 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689748+0000", 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7690000000000001e-06 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692563+0000", 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028147329999999998 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692593+0000", 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9476999999999999e-05 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692616+0000", 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3340999999999999e-05 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723279+0000", 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030662759000000001 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723430+0000", 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151247 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723629+0000", 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199426 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793213+0000", 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069583435999999999 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127443+0000", 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33423067000000001 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127491+0000", 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8000999999999997e-05 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127529+0000", 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7494999999999999e-05 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:364 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.958785716000001, 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43506039200000002, 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 364 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692652+0000", 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9280000000000002e-06 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697100+0000", 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004447678 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701619+0000", 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045192829999999998 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701638+0000", 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8365000000000001e-05 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742561+0000", 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040923560999999997 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742686+0000", 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012482899999999999 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742839+0000", 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015348 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808887+0000", 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066047260999999996 2024-09-19T05:44:21.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127632+0000", 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31874497099999999 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127672+0000", 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0244999999999998e-05 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127710+0000", 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7790999999999998e-05 2024-09-19T05:44:21.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:395 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.946043872000001, 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48275294299999999, 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 395 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705394+0000", 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7800000000000001e-06 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720169+0000", 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014775346 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720185+0000", 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5721000000000001e-05 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720194+0000", 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4609999999999997e-06 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720294+0000", 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100005 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720392+0000", 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7547999999999995e-05 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720531+0000", 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000139099 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.767968+0000", 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047436846999999997 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188062+0000", 2024-09-19T05:44:21.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42009412899999998 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188107+0000", 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5269000000000003e-05 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188144+0000", 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6738e-05 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:404 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.930606705999999, 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46750329699999998, 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 404 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720832+0000", 2024-09-19T05:44:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8469999999999998e-06 2024-09-19T05:44:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726318+0000", 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005485775 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726490+0000", 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172339 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726505+0000", 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4467999999999999e-05 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743219+0000", 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016714046 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743320+0000", 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101548 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743459+0000", 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013899499999999999 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.775025+0000", 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031565658000000003 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188247+0000", 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.413221583 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188297+0000", 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0586e-05 2024-09-19T05:44:21.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188332+0000", 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4452000000000002e-05 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:412 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.912987902000001, 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40023217100000003, 2024-09-19T05:44:21.753 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 412 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738450+0000", 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7e-06 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.758682+0000", 2024-09-19T05:44:21.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020232357999999999 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772625+0000", 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013943232 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.774996+0000", 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023703890000000001 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793240+0000", 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018244646999999999 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793347+0000", 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106276 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793511+0000", 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016406099999999999 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808895+0000", 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015384296 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138570+0000", 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32967460500000001 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138630+0000", 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0087999999999998e-05 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138679+0000", 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9518999999999998e-05 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:441 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.759 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.857279353999999, 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39435722000000001, 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 441 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794159+0000", 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.579e-06 2024-09-19T05:44:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.802843+0000", 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0086840579999999997 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.807986+0000", 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051424890000000001 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808000+0000", 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808113+0000", 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011294900000000001 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808200+0000", 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6792999999999994e-05 2024-09-19T05:44:21.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808612+0000", 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041192799999999999 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827676+0000", 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019063875000000001 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188432+0000", 2024-09-19T05:44:21.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36075675699999998 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188479+0000", 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6149000000000001e-05 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188513+0000", 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4310999999999997e-05 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:454 3.37 3:ec162ecd:::10000000002.000000c3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.818515882, 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32323945999999998, 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 454 2024-09-19T05:44:21.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832922+0000", 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6280000000000001e-06 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839384+0000", 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064625180000000004 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850436+0000", 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011051365 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850464+0000", 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8719000000000001e-05 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:21.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850532+0000", 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7212000000000006e-05 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850663+0000", 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013160799999999999 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850877+0000", 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000213716 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871083+0000", 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020205864 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156086+0000", 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.285003532 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156125+0000", 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8569999999999998e-05 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:21.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156159+0000", 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3728e-05 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:21.772 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:21.773 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:21.774 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:21.772+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:21.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:44:21.875 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:21.874+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:21.883 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:21.883 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:21.883 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:21.883 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:21.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:21.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.920+0000 7fada0a77640 1 Processor -- start 2024-09-19T05:44:21.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.920+0000 7fada0a77640 1 -- start start 2024-09-19T05:44:21.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.921+0000 7fada0a77640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fad9c15b2a0 0x7fad9c0718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.921+0000 7fada0a77640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad9c0ca910 con 0x7fad9c0cb870 2024-09-19T05:44:21.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.921+0000 7fada0a77640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad9c0cabf0 con 0x7fad9c0cbc50 2024-09-19T05:44:21.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.921+0000 7fada0a77640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fad9c06d930 con 0x7fad9c15b2a0 2024-09-19T05:44:21.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.921+0000 7fad9a59c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fad9c0cb870 0x7fad9c15ab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:38534/0 (socket says 172.21.15.2:38534) 2024-09-19T05:44:21.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.921+0000 7fad9a59c640 1 -- 172.21.15.2:0/2765039316 learned_addr learned my addr 172.21.15.2:0/2765039316 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:21.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.921+0000 7fad99d9b640 1 --2- 172.21.15.2:0/2765039316 >> v2:172.21.15.123:3300/0 conn(0x7fad9c15b2a0 0x7fad9c0718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 154367263 0 0) 0x7fad9c0ca910 con 0x7fad9c0cb870 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad88003520 con 0x7fad9c0cb870 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3951005844 0 0) 0x7fad88003520 con 0x7fad9c0cb870 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 >> v2:172.21.15.123:3300/0 conn(0x7fad9c15b2a0 msgr2=0x7fad9c0718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 --2- 172.21.15.2:0/2765039316 >> v2:172.21.15.123:3300/0 conn(0x7fad9c15b2a0 0x7fad9c0718a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 >> v1:172.21.15.123:6789/0 conn(0x7fad9c0cbc50 legacy=0x7fad9c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c06ce50 con 0x7fad9c0cb870 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7fad94001670 con 0x7fad9c0cb870 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 >> v1:172.21.15.2:6789/0 conn(0x7fad9c0cb870 legacy=0x7fad9c15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880039b0 0x7fad88003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad880043b0 0x7fad88004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad88005080 0x7fad8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c06d930 con 0x7fad880043b0 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad8800bfe0 con 0x7fad880039b0 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fad8800c1e0 con 0x7fad88005080 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad99d9b640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad880043b0 0x7fad88004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9a59c640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880039b0 0x7fad88003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9ad9d640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad88005080 0x7fad8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad99d9b640 1 -- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad88005080 msgr2=0x7fad8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad99d9b640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad88005080 0x7fad8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad99d9b640 1 -- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880039b0 msgr2=0x7fad88003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad99d9b640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880039b0 0x7fad88003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:21.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad9ad9d640 1 -- 172.21.15.2:0/2765039316 reap_dead start 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.922+0000 7fad99d9b640 1 -- 172.21.15.2:0/2765039316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad88003520 con 0x7fad880043b0 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.923+0000 7fad99d9b640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad880043b0 0x7fad88004880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad84009ac0 tx=0x7fad88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4595b965bbf852 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.923+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84036070 con 0x7fad880043b0 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.923+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84031400 con 0x7fad880043b0 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.923+0000 7fad9959a640 1 -- 172.21.15.2:0/2765039316 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84009690 con 0x7fad880043b0 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.924+0000 7fada0a77640 1 -- 172.21.15.2:0/2765039316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad880043b0 msgr2=0x7fad88004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.924+0000 7fada0a77640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad880043b0 0x7fad88004880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad84009ac0 tx=0x7fad88005830 comp rx=0 tx=0).stop 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.924+0000 7fada0a77640 1 -- 172.21.15.2:0/2765039316 shutdown_connections 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.924+0000 7fada0a77640 1 --2- 172.21.15.2:0/2765039316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad880043b0 0x7fad88004880 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.924+0000 7fada0a77640 1 -- 172.21.15.2:0/2765039316 >> 172.21.15.2:0/2765039316 conn(0x7fad9c06bdf0 msgr2=0x7fad9c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.924+0000 7fada0a77640 1 -- 172.21.15.2:0/2765039316 shutdown_connections 2024-09-19T05:44:21.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.924+0000 7fada0a77640 1 -- 172.21.15.2:0/2765039316 wait complete. 2024-09-19T05:44:21.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.925+0000 7fada0a77640 1 Processor -- start 2024-09-19T05:44:21.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.925+0000 7fada0a77640 1 -- start start 2024-09-19T05:44:21.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fada0a77640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880043b0 0x7fad9c1470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fada0a77640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad9c155b10 0x7fad9c1f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fada0a77640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad9c154010 0x7fad9c1f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fada0a77640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c06d4a0 con 0x7fad9c154010 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fada0a77640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c0caed0 con 0x7fad880043b0 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fada0a77640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fad9c0ca420 con 0x7fad9c155b10 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad9c154010 0x7fad9c1f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fad9ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad9c154010 0x7fad9c1f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58598/0 (socket says 172.21.15.2:58598) 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fad9ad9d640 1 -- 172.21.15.2:0/2011023413 learned_addr learned my addr 172.21.15.2:0/2011023413 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fad99d9b640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad9c155b10 0x7fad9c1f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.926+0000 7fad9a59c640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880043b0 0x7fad9c1470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad9ad9d640 1 -- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad9c155b10 msgr2=0x7fad9c1f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad9ad9d640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad9c155b10 0x7fad9c1f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad9ad9d640 1 -- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880043b0 msgr2=0x7fad9c1470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad9ad9d640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880043b0 0x7fad9c1470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad9ad9d640 1 -- 172.21.15.2:0/2011023413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c0c7c60 con 0x7fad9c154010 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad9a59c640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880043b0 0x7fad9c1470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:21.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad9ad9d640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad9c154010 0x7fad9c1f2630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00fae0 tx=0x7fad8c0020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad92ffd640 1 -- 172.21.15.2:0/2011023413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c01a070 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad92ffd640 1 -- 172.21.15.2:0/2011023413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8c006e20 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c0c7f40 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fad92ffd640 1 -- 172.21.15.2:0/2011023413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c003bb0 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.927+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c1f18f0 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.928+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9c15ab90 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.928+0000 7fad92ffd640 1 -- 172.21.15.2:0/2011023413 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fad8c01d070 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.929+0000 7fad92ffd640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fad7c06c2b0 0x7fad7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.929+0000 7fad92ffd640 1 -- 172.21.15.2:0/2011023413 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5799+0+0 (secure 0 0 0) 0x7fad8c08ab20 con 0x7fad9c154010 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.929+0000 7fad9a59c640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fad7c06c2b0 0x7fad7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:21.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.929+0000 7fad9a59c640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fad7c06c2b0 0x7fad7c06e770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fad9c06a650 tx=0x7fad94009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:21.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:21.933+0000 7fad92ffd640 1 -- 172.21.15.2:0/2011023413 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fad8c059230 con 0x7fad9c154010 2024-09-19T05:44:21.975 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:21.974+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:22.030 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:22.030 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:22.030 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:22.030 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:22.030 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:22.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:22.076 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:22.075+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.121+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fad9c0ce3d0 con 0x7fad9c154010 2024-09-19T05:44:22.177 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:22.176+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:77 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.839991234999999, 2024-09-19T05:44:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40221933999999998, 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 77 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348160+0000", 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0459999999999999e-06 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362035+0000", 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013874391999999999 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362063+0000", 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7804e-05 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362076+0000", 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3200000000000001e-05 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362162+0000", 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6469000000000002e-05 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362317+0000", 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015507100000000001 2024-09-19T05:44:22.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362454+0000", 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013636799999999999 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.372327+0000", 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0098729219999999993 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750183+0000", 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37785643600000002 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750233+0000", 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9960999999999998e-05 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750376+0000", 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142671 2024-09-19T05:44:22.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:87 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.805526309000001, 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39257614200000002, 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382623+0000", 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2529999999999999e-06 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390766+0000", 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081425790000000005 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390808+0000", 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2311999999999997e-05 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390827+0000", 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390955+0000", 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012825900000000001 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391148+0000", 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000192673 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391432+0000", 2024-09-19T05:44:22.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028362599999999998 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.455303+0000", 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063870983000000006 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775090+0000", 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31978768600000002 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775146+0000", 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5393000000000003e-05 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775197+0000", 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1699000000000001e-05 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:98 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.204 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.750316997999999, 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42340282099999998, 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 98 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437834+0000", 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.442377+0000", 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045437719999999997 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448982+0000", 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066041210000000001 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449009+0000", 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7898999999999998e-05 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449078+0000", 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8301999999999998e-05 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449208+0000", 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013036300000000001 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449349+0000", 2024-09-19T05:44:22.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000140791 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515702+0000", 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066353102999999997 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861115+0000", 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34541303600000001 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861183+0000", 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8238999999999995e-05 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861233+0000", 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0081999999999999e-05 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.210 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:143 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.715185094999999, 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40938193499999997, 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 143 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472965+0000", 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7879999999999998e-06 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486591+0000", 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013625417000000001 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486611+0000", 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0285e-05 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486624+0000", 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2763e-05 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501834+0000", 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01521081 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504117+0000", 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022829479999999999 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504306+0000", 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018812599999999999 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515709+0000", 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403529000000001 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882232+0000", 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36652316899999998 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882300+0000", 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7539000000000005e-05 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882344+0000", 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4561000000000002e-05 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:145 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.701497371999999, 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39312144199999999, 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486653+0000", 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0089999999999999e-06 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491674+0000", 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050208450000000003 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491698+0000", 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4457999999999999e-05 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491721+0000", 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2294999999999999e-05 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510737+0000", 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019016000000000002 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510880+0000", 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014287999999999999 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511053+0000", 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017286299999999999 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555682+0000", 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044629755 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879645+0000", 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.323962272 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879715+0000", 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0400000000000004e-05 2024-09-19T05:44:22.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879772+0000", 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6665e-05 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:175 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.660931989000002, 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35525569800000001, 2024-09-19T05:44:22.222 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 175 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527221+0000", 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2299999999999999e-06 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531495+0000", 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004274211 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556121+0000", 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024626329999999998 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556132+0000", 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0740000000000001e-05 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556187+0000", 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5096000000000002e-05 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556372+0000", 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185059 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556560+0000", 2024-09-19T05:44:22.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000187803 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.562115+0000", 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055550449999999998 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882386+0000", 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32027064199999999 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882435+0000", 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8990999999999999e-05 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882471+0000", 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6551000000000003e-05 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:298 3.26 3:64018a55:::10000000002.00000027:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.575840195000001, 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59287720300000002, 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 298 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612310+0000", 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.757e-06 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619160+0000", 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068500490000000004 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619231+0000", 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0642000000000006e-05 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619249+0000", 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8544000000000001e-05 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619441+0000", 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019125500000000001 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619600+0000", 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015895800000000001 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619776+0000", 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017654300000000001 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.630552+0000", 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010775523 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205074+0000", 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57452220799999998 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205132+0000", 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8038999999999998e-05 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205185+0000", 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2685000000000001e-05 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.234 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:323 3.16 3:6a386151:::10000000002.00000040:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.543611299999998, 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.567029063, 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 323 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644539+0000", 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5040000000000001e-06 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649094+0000", 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045552429999999996 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649120+0000", 2024-09-19T05:44:22.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6176000000000001e-05 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649138+0000", 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8046e-05 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649227+0000", 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8505999999999996e-05 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649330+0000", 2024-09-19T05:44:22.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010301700000000001 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649482+0000", 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015256300000000001 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.719941+0000", 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.070459037000000002 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211483+0000", 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49154185500000003 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211531+0000", 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8026999999999999e-05 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211565+0000", 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4088999999999998e-05 2024-09-19T05:44:22.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:269 3.2 e54/20 3:438a76aa:::10000000002.0000000a:head v 54'6, mlcod=53'4)", 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.543508608, 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37357711900000001, 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644640+0000", 2024-09-19T05:44:22.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.192e-06 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976810+0000", 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33216997700000001 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976833+0000", 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2878e-05 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976847+0000", 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3644e-05 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976972+0000", 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125487 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.977007+0000", 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5027000000000002e-05 2024-09-19T05:44:22.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.017830+0000", 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040822590999999998 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.018216+0000", 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038632299999999999 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:333 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.53738113, 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39671826999999998, 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 333 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650769+0000", 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0110000000000001e-06 2024-09-19T05:44:22.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657005+0000", 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062355090000000002 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657046+0000", 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.074e-05 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657054+0000", 2024-09-19T05:44:22.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3310000000000004e-06 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657132+0000", 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7696999999999998e-05 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657256+0000", 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012425000000000001 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:22.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657471+0000", 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021535099999999999 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.715589+0000", 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058117380000000003 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047372+0000", 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33178333599999998 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047417+0000", 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5423000000000002e-05 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047485+0000", 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7242000000000004e-05 2024-09-19T05:44:22.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:343 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.523932050999999, 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39137022999999999, 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 343 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664218+0000", 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666946+0000", 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027284710000000001 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666962+0000", 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5620999999999998e-05 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666973+0000", 2024-09-19T05:44:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1199000000000001e-05 2024-09-19T05:44:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667053+0000", 2024-09-19T05:44:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9689000000000002e-05 2024-09-19T05:44:22.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667166+0000", 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011285699999999999 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667509+0000", 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00034300599999999997 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.675954+0000", 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084449580000000007 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055481+0000", 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37952754500000002 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055535+0000", 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055586+0000", 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0358000000000001e-05 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:354 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.444740904, 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.468021299, 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 354 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743409+0000", 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7369999999999998e-06 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.747060+0000", 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036510430000000001 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768120+0000", 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021059484 2024-09-19T05:44:22.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768139+0000", 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9085000000000001e-05 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768289+0000", 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150252 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768387+0000", 2024-09-19T05:44:22.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8169000000000001e-05 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768573+0000", 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185973 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.776933+0000", 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083599629999999998 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211285+0000", 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43435194300000002 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211363+0000", 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7985000000000006e-05 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211428+0000", 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4665000000000005e-05 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:399 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.403155028, 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.455193454, 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 399 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.265 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784997+0000", 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.791076+0000", 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00607888 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793492+0000", 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024155940000000001 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793539+0000", 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7246999999999997e-05 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793694+0000", 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000155589 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793818+0000", 2024-09-19T05:44:22.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012335499999999999 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794028+0000", 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000209822 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839319+0000", 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045291168999999999 2024-09-19T05:44:22.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240115+0000", 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40079620599999999 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240147+0000", 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1782000000000002e-05 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240186+0000", 2024-09-19T05:44:22.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9168999999999997e-05 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:401 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.390827217999998, 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41436097300000002, 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 401 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797324+0000", 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6239999999999999e-06 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803827+0000", 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065027219999999998 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803845+0000", 2024-09-19T05:44:22.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8717999999999999e-05 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803858+0000", 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2599000000000001e-05 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803963+0000", 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010488 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.804102+0000", 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013866900000000001 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.804299+0000", 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000197624 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.861232+0000", 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056932356000000003 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211606+0000", 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35037407999999998 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211650+0000", 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4629000000000002e-05 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211681+0000", 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1072000000000003e-05 2024-09-19T05:44:22.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:431 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.334434480999999, 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38660652099999998, 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 431 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853715+0000", 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.95e-06 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862660+0000", 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008945138 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862684+0000", 2024-09-19T05:44:22.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4102999999999999e-05 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862697+0000", 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2899e-05 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862819+0000", 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012190500000000001 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863007+0000", 2024-09-19T05:44:22.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018809600000000001 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.865417+0000", 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002410316 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967698+0000", 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102280148 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240225+0000", 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27252755200000001 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240282+0000", 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6935999999999998e-05 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240320+0000", 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7478000000000003e-05 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:432 3.26 3:66aef6e9:::10000000002.000000ad:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.325415576000001, 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37769895199999998, 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 432 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.283 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862736+0000", 2024-09-19T05:44:22.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1749999999999998e-06 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868431+0000", 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005694569 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868467+0000", 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5809000000000002e-05 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868482+0000", 2024-09-19T05:44:22.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.52e-05 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896790+0000", 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028308262000000001 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896919+0000", 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012855399999999999 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.897240+0000", 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032181899999999999 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037309+0000", 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.140068843 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240357+0000", 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20304760699999999 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240400+0000", 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2849000000000003e-05 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240431+0000", 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1265e-05 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:457 3.26 3:64ff7f1d:::10000000002.000000c6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.302117707000001, 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35451085300000001, 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 457 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886033+0000", 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889815+0000", 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037826800000000001 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889868+0000", 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2592000000000001e-05 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889881+0000", 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3251000000000001e-05 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912234+0000", 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022353101 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912416+0000", 2024-09-19T05:44:22.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000181695 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912603+0000", 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186536 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037342+0000", 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.124739117 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240463+0000", 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.203121157 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240505+0000", 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2406e-05 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240541+0000", 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5488999999999999e-05 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:460 3.26 3:6430d4f3:::10000000002.000000c9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.292273081000001, 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34478341699999998, 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 460 2024-09-19T05:44:22.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895877+0000", 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.779e-06 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898920+0000", 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030423379999999999 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898937+0000", 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7782999999999999e-05 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898948+0000", 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665e-05 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953289+0000", 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054340672999999999 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953400+0000", 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111527 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953557+0000", 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015719599999999999 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.130946+0000", 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17738901100000001 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240578+0000", 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10963127 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240625+0000", 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7429000000000003e-05 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240658+0000", 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2746000000000001e-05 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:463 3.26 3:65689812:::10000000002.000000cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.28917216, 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34245139299999999, 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 463 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.301 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898979+0000", 2024-09-19T05:44:22.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2610000000000002e-06 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902351+0000", 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033725270000000002 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902372+0000", 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0699e-05 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902388+0000", 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6461e-05 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967734+0000", 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065346107 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967874+0000", 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013911199999999999 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.968092+0000", 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218438 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.204745+0000", 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23665298800000001 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241370+0000", 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036625433999999998 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241402+0000", 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1523999999999998e-05 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241427+0000", 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4842e-05 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.306 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:467 3.26 3:65bb0d93:::10000000002.000000d0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.285727736999998, 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34422604099999998, 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 467 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902423+0000", 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.117e-06 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908646+0000", 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062233660000000001 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908664+0000", 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7880000000000002e-05 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908679+0000", 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5129e-05 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211711+0000", 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30303182099999998 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211794+0000", 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2541999999999997e-05 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211940+0000", 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014651900000000001 2024-09-19T05:44:22.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241451+0000", 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029510593000000002 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246560+0000", 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051091019999999999 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246605+0000", 2024-09-19T05:44:22.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123999999999997e-05 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246646+0000", 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0847999999999999e-05 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.312 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:22.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:22.314 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:22.313+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:22.414 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:22.413+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:22.462 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:22.463 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:22.463 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:22.463 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:22.471 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:22.515 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:22.513+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:22.578 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:22.578 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:22.578 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:22.578 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:22.578 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:22.587 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:22.628 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:22.626+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:22.716 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:22.715+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:11 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.447282940000001, 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53368345399999995, 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287043+0000", 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294617+0000", 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075741280000000003 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294641+0000", 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3424000000000001e-05 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294656+0000", 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4679e-05 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294841+0000", 2024-09-19T05:44:22.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185176 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295155+0000", 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031388199999999997 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295811+0000", 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065616700000000003 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.308016+0000", 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012205472 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820590+0000", 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51257333400000005 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820658+0000", 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8836000000000003e-05 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820724+0000", 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5426999999999995e-05 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:18 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.396571937000001, 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48181817700000001, 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 18 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337754+0000", 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346234+0000", 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084795919999999993 2024-09-19T05:44:22.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346309+0000", 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4950999999999994e-05 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346324+0000", 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5199e-05 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346456+0000", 2024-09-19T05:44:22.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013213799999999999 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346703+0000", 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024689500000000001 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346893+0000", 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019002400000000001 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382206+0000", 2024-09-19T05:44:22.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035313485999999998 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819443+0000", 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43723664899999998 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819516+0000", 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3021000000000004e-05 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819570+0000", 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3696999999999998e-05 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:37 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.317020535000001, 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.412251706, 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417306+0000", 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3790000000000002e-06 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421246+0000", 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039399869999999998 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421273+0000", 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6412999999999999e-05 2024-09-19T05:44:22.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421284+0000", 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1856e-05 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421374+0000", 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9411000000000002e-05 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429518+0000", 2024-09-19T05:44:22.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008143908 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429741+0000", 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022306100000000001 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.460095+0000", 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030354269999999999 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829462+0000", 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36936722900000002 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829510+0000", 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7698999999999999e-05 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829555+0000", 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4493000000000002e-05 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:53 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.303853144000001, 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45610759200000001, 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 53 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.755 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430472+0000", 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1320000000000001e-06 2024-09-19T05:44:22.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437581+0000", 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071082300000000001 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444289+0000", 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067080070000000002 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444322+0000", 2024-09-19T05:44:22.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3034000000000003e-05 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445189+0000", 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086775100000000005 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445402+0000", 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021237099999999999 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445599+0000", 2024-09-19T05:44:22.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019734099999999999 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501230+0000", 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055630863000000003 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886462+0000", 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38523158200000002 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886525+0000", 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3138000000000004e-05 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886578+0000", 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3143000000000003e-05 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:61 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:22.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.276918811000002, 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.424715008, 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 61 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:22.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457409+0000", 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461813+0000", 2024-09-19T05:44:22.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044040060000000002 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461834+0000", 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0789e-05 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461876+0000", 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2268000000000003e-05 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461987+0000", 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110593 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467170+0000", 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051828849999999999 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467417+0000", 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024757199999999999 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490516+0000", 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023098802000000002 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.881965+0000", 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39144917200000001 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882043+0000", 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7830000000000005e-05 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882120+0000", 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6360999999999996e-05 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:267 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.137772282, 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52119119999999997, 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.766 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 267 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596555+0000", 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.596e-06 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597947+0000", 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001392517 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597975+0000", 2024-09-19T05:44:22.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7993000000000001e-05 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597989+0000", 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3927999999999999e-05 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598090+0000", 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010057500000000001 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598219+0000", 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012952999999999999 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598408+0000", 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018851600000000001 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604394+0000", 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059861159999999997 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117572+0000", 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51317804499999997 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117624+0000", 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2126000000000002e-05 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117742+0000", 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011825800000000001 2024-09-19T05:44:22.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:274 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.12213023, 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63191469899999997, 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.772 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 274 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612198+0000", 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.476e-06 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619545+0000", 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073473399999999999 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619574+0000", 2024-09-19T05:44:22.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8994000000000001e-05 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619585+0000", 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1103e-05 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619846+0000", 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026047199999999998 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619979+0000", 2024-09-19T05:44:22.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013353099999999999 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.620203+0000", 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000223945 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629556+0000", 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093532560000000008 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244028+0000", 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61447147300000005 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244065+0000", 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6968999999999998e-05 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244108+0000", 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3139999999999997e-05 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:293 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.080631543999999, 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59051000200000003, 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 293 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.778 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653696+0000", 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.33e-06 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658892+0000", 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051954360000000003 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658913+0000", 2024-09-19T05:44:22.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1705999999999999e-05 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658928+0000", 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4361e-05 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659326+0000", 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000398253 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659446+0000", 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012050200000000001 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659655+0000", 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020868500000000001 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.734308+0000", 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074653061000000007 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244147+0000", 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.50983838800000003 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244175+0000", 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8901999999999999e-05 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244202+0000", 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6378000000000001e-05 2024-09-19T05:44:22.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:309 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.064897612999999, 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.505676974, 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.784 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 309 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669428+0000", 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.593e-06 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.676971+0000", 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075425839999999997 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677009+0000", 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8154000000000001e-05 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677039+0000", 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9788999999999999e-05 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677223+0000", 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000184372 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677347+0000", 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123979 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677516+0000", 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016852499999999999 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732124+0000", 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054607912000000002 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.174998+0000", 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44287454799999998 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175054+0000", 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5476999999999998e-05 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175103+0000", 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9041e-05 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:313 3.2f 3:f7c72662:::10000000002.00000036:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.789 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.057196914999999, 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47726204500000002, 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 313 2024-09-19T05:44:22.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677129+0000", 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3479999999999998e-06 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685213+0000", 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080837730000000007 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685237+0000", 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4311000000000002e-05 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685254+0000", 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7428999999999999e-05 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863622+0000", 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.178367422 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863724+0000", 2024-09-19T05:44:22.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101938 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863861+0000", 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013715 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.152292+0000", 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.288431614 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154270+0000", 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019771250000000001 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154330+0000", 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154388+0000", 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8717e-05 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:317 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.048974547, 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56821626300000005, 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 317 2024-09-19T05:44:22.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685351+0000", 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3549999999999999e-06 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692638+0000", 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072869909999999996 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692666+0000", 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8010000000000001e-05 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692677+0000", 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0776e-05 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692800+0000", 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012327200000000001 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692936+0000", 2024-09-19T05:44:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013544100000000001 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.693426+0000", 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00049003600000000001 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732131+0000", 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038705212000000003 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253472+0000", 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52134096299999999 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253518+0000", 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5731e-05 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253565+0000", 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7475999999999997e-05 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.801 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:331 3.30 3:0d2dbd17:::10000000002.00000048:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.985310251000001, 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.495291814, 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 331 2024-09-19T05:44:22.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749016+0000", 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.754846+0000", 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058302459999999999 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756282+0000", 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001436295 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756297+0000", 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4921e-05 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756370+0000", 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2856999999999997e-05 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756495+0000", 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012511999999999999 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756711+0000", 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216233 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828851+0000", 2024-09-19T05:44:22.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072139743000000006 2024-09-19T05:44:22.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244223+0000", 2024-09-19T05:44:22.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41537228599999998 2024-09-19T05:44:22.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244270+0000", 2024-09-19T05:44:22.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-19T05:44:22.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244305+0000", 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4527999999999999e-05 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:346 3.11 3:8815798b:::10000000002.00000057:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.977996101999999, 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49739494099999998, 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 346 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756330+0000", 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759712+0000", 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033821889999999999 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759737+0000", 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5069e-05 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759752+0000", 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4372999999999999e-05 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759847+0000", 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5291999999999995e-05 2024-09-19T05:44:22.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760098+0000", 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025112099999999998 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760422+0000", 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032433199999999998 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855328+0000", 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094905274999999997 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253622+0000", 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39829475600000003 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253679+0000", 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6047000000000001e-05 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253722+0000", 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3702999999999998e-05 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:361 3.30 3:0eedefab:::10000000002.00000066:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.961860583, 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47196730199999998, 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 361 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772466+0000", 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8279999999999999e-06 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777394+0000", 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049281790000000004 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777413+0000", 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8913999999999999e-05 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777424+0000", 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1192999999999999e-05 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799507+0000", 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022083230999999998 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799656+0000", 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014921300000000001 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799858+0000", 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020186999999999999 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830773+0000", 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030915148999999999 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244343+0000", 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41356947900000002 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244392+0000", 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9489e-05 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244430+0000", 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7756999999999998e-05 2024-09-19T05:44:22.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:366 3.11 3:8b4c71c3:::10000000002.0000006b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.956870761000001, 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47662371999999997, 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 366 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777455+0000", 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7640000000000001e-06 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783373+0000", 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005917533 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783420+0000", 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6709999999999998e-05 2024-09-19T05:44:22.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783443+0000", 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3836999999999999e-05 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808013+0000", 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024569607 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808128+0000", 2024-09-19T05:44:22.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011490299999999999 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808375+0000", 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024700299999999998 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937889+0000", 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951412500000001 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253987+0000", 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31609763600000002 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254037+0000", 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0383999999999997e-05 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254076+0000", 2024-09-19T05:44:22.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9218000000000003e-05 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:368 3.14 3:280f29a3:::10000000002.0000006d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.950825384000002, 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45471794900000001, 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 368 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:22.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783504+0000", 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7749999999999998e-06 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785589+0000", 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020852409999999998 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785606+0000", 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6609999999999999e-05 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785614+0000", 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4940000000000003e-06 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789369+0000", 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037552250000000001 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789485+0000", 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116019 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789654+0000", 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016819299999999999 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828885+0000", 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039231901999999999 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238106+0000", 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40922085699999999 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238164+0000", 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7373000000000001e-05 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238216+0000", 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2259999999999998e-05 2024-09-19T05:44:22.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:382 3.11 3:8818af48:::10000000002.0000007b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.945417462999998, 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46533816500000003, 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.833 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 382 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788911+0000", 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6609999999999999e-06 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796253+0000", 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073421060000000002 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796284+0000", 2024-09-19T05:44:22.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1383000000000001e-05 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796299+0000", 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4793e-05 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832493+0000", 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036194021 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832612+0000", 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011929 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832778+0000", 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.066630+0000", 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.233852226 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254144+0000", 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18751385100000001 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254200+0000", 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6028e-05 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254244+0000", 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4373999999999999e-05 2024-09-19T05:44:22.838 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:403 3.9 3:91a8729a:::10000000002.00000090:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.917752415999999, 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41822062799999998, 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 403 2024-09-19T05:44:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816573+0000", 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.345e-06 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.820472+0000", 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038989060000000002 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827487+0000", 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070152189999999996 2024-09-19T05:44:22.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827507+0000", 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9434000000000001e-05 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858059+0000", 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030552598 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858252+0000", 2024-09-19T05:44:22.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019281999999999999 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871029+0000", 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012776648999999999 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175256+0000", 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30422741199999997 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234683+0000", 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059426432000000001 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234738+0000", 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4923999999999998e-05 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234792+0000", 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3888999999999999e-05 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.844 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:405 3.11 3:8834f03f:::10000000002.00000092:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.906789098000001, 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42686835699999998, 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 405 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827537+0000", 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6620000000000001e-06 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830490+0000", 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029526209999999999 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830506+0000", 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6891000000000001e-05 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830535+0000", 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8377e-05 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855366+0000", 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024831576000000001 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855488+0000", 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012119499999999999 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855676+0000", 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018832899999999999 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233600+0000", 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37792366900000002 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254308+0000", 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020708684000000002 2024-09-19T05:44:22.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254362+0000", 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3430000000000002e-05 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254403+0000", 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0923000000000001e-05 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.850 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:410 3.11 3:8988b152:::10000000002.00000097:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.903766926999999, 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42399440900000002, 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 410 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:22.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830559+0000", 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-19T05:44:22.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835124+0000", 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045654520000000002 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835145+0000", 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0750999999999998e-05 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835179+0000", 2024-09-19T05:44:22.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4013000000000001e-05 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937947+0000", 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102767884 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938050+0000", 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010270199999999999 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:22.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938225+0000", 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000175494 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234859+0000", 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.296633759 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254459+0000", 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019599979999999999 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254510+0000", 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1431000000000003e-05 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254551+0000", 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0391e-05 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:22.856 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:22.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.780+0000 7fad92ffd640 1 -- 172.21.15.2:0/2011023413 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v67) ==== 101+0+0 (secure 0 0 0) 0x7fad8c058b30 con 0x7fad9c154010 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fad7c06c2b0 msgr2=0x7fad7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fad7c06c2b0 0x7fad7c06e770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fad9c06a650 tx=0x7fad94009040 comp rx=0 tx=0).stop 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad9c154010 msgr2=0x7fad9c1f2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad9c154010 0x7fad9c1f2630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00fae0 tx=0x7fad8c0020a0 comp rx=0 tx=0).stop 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 shutdown_connections 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fad7c06c2b0 0x7fad7c06e770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad9c154010 0x7fad9c1f2630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fad9c155b10 0x7fad9c1f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 --2- 172.21.15.2:0/2011023413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad880043b0 0x7fad9c1470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 >> 172.21.15.2:0/2011023413 conn(0x7fad9c06bdf0 msgr2=0x7fad9c0ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 shutdown_connections 2024-09-19T05:44:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:22.783+0000 7fada0a77640 1 -- 172.21.15.2:0/2011023413 wait complete. 2024-09-19T05:44:22.861 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:22.856+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:22.959 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:22.957+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:22.968 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:22.983 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:22.983 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:22.983 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:22.983 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:22.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:23.061 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:23.059+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:23.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.069+0000 7f435741e640 1 Processor -- start 2024-09-19T05:44:23.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.069+0000 7f435741e640 1 -- start start 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f435741e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd4e0 0x7f43500b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f435741e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f435015ebe0 con 0x7f43500cd8c0 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f435741e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43500b2e00 con 0x7f435015aa90 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f435741e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f435015eec0 con 0x7f43500cd4e0 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f43559bb640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f43500cd8c0 0x7f43500c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:38544/0 (socket says 172.21.15.2:38544) 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f43559bb640 1 -- 172.21.15.2:0/1412344631 learned_addr learned my addr 172.21.15.2:0/1412344631 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f43551ba640 1 --2- 172.21.15.2:0/1412344631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd4e0 0x7f43500b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f4347fff640 1 -- 172.21.15.2:0/1412344631 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1504180201 0 0) 0x7f435015ebe0 con 0x7f43500cd8c0 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.070+0000 7f4347fff640 1 -- 172.21.15.2:0/1412344631 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f433c003520 con 0x7f43500cd8c0 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.071+0000 7f43551ba640 1 -- 172.21.15.2:0/1412344631 >> v1:172.21.15.123:6789/0 conn(0x7f435015aa90 legacy=0x7f435015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.071+0000 7f43551ba640 1 -- 172.21.15.2:0/1412344631 >> v1:172.21.15.2:6789/0 conn(0x7f43500cd8c0 legacy=0x7f43500c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.071+0000 7f4347fff640 1 -- 172.21.15.2:0/1412344631 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279565232 0 0) 0x7f43500b2e00 con 0x7f435015aa90 2024-09-19T05:44:23.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.071+0000 7f43551ba640 1 -- 172.21.15.2:0/1412344631 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43500b30e0 con 0x7f43500cd4e0 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.071+0000 7f43551ba640 1 --2- 172.21.15.2:0/1412344631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd4e0 0x7f43500b3bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f434c0097d0 tx=0x7f434c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd8a528fd251fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f4347fff640 1 -- 172.21.15.2:0/1412344631 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c037070 con 0x7f43500cd4e0 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f4347fff640 1 -- 172.21.15.2:0/1412344631 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434c032440 con 0x7f43500cd4e0 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f4347fff640 1 -- 172.21.15.2:0/1412344631 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0327f0 con 0x7f43500cd4e0 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f435741e640 1 -- 172.21.15.2:0/1412344631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd4e0 msgr2=0x7f43500b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f435741e640 1 --2- 172.21.15.2:0/1412344631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd4e0 0x7f43500b3bc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f434c0097d0 tx=0x7f434c02a210 comp rx=0 tx=0).stop 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f435741e640 1 -- 172.21.15.2:0/1412344631 shutdown_connections 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f435741e640 1 --2- 172.21.15.2:0/1412344631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd4e0 0x7f43500b3bc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f435741e640 1 -- 172.21.15.2:0/1412344631 >> 172.21.15.2:0/1412344631 conn(0x7f4350067b30 msgr2=0x7f4350067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f435741e640 1 -- 172.21.15.2:0/1412344631 shutdown_connections 2024-09-19T05:44:23.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.072+0000 7f435741e640 1 -- 172.21.15.2:0/1412344631 wait complete. 2024-09-19T05:44:23.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.073+0000 7f435741e640 1 Processor -- start 2024-09-19T05:44:23.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.073+0000 7f435741e640 1 -- start start 2024-09-19T05:44:23.077 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f435741e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43500cd4e0 0x7f435014f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f435741e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd8c0 0x7f4350152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f435741e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f435015aa90 0x7f4350152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f435741e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f435015e640 con 0x7f43500cd4e0 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f435741e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43500b36a0 con 0x7f435015aa90 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f435741e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f435015e7c0 con 0x7f43500cd8c0 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43551ba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43500cd4e0 0x7f435014f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43551ba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43500cd4e0 0x7f435014f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58604/0 (socket says 172.21.15.2:58604) 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43551ba640 1 -- 172.21.15.2:0/3581880456 learned_addr learned my addr 172.21.15.2:0/3581880456 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43559bb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f435015aa90 0x7f4350152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43549b9640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd8c0 0x7f4350152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43551ba640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd8c0 msgr2=0x7f4350152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43500cd8c0 0x7f4350152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.074+0000 7f43551ba640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f435015aa90 msgr2=0x7f4350152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f435015aa90 0x7f4350152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f43551ba640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43501f45e0 con 0x7f43500cd4e0 2024-09-19T05:44:23.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43500cd4e0 0x7f435014f250 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f434c000c00 tx=0x7f434c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c041070 con 0x7f43500cd4e0 2024-09-19T05:44:23.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434c03c870 con 0x7f43500cd4e0 2024-09-19T05:44:23.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c037040 con 0x7f43500cd4e0 2024-09-19T05:44:23.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43501f0600 con 0x7f43500cd4e0 2024-09-19T05:44:23.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.075+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501f0bc0 con 0x7f43500cd4e0 2024-09-19T05:44:23.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.076+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f43500caaf0 con 0x7f43500cd4e0 2024-09-19T05:44:23.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.077+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f434c044070 con 0x7f43500cd4e0 2024-09-19T05:44:23.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.077+0000 7f4345ffb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f433406c2b0 0x7f433406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.077+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5799+0+0 (secure 0 0 0) 0x7f434c0abfe0 con 0x7f43500cd4e0 2024-09-19T05:44:23.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.077+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v67) ==== 54+0+16 (secure 0 0 0) 0x7f43500caaf0 con 0x7f43500cd4e0 2024-09-19T05:44:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.077+0000 7f43549b9640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f433406c2b0 0x7f433406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.078+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f435014f790 0x7f435015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.078+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f435015e3f0 con 0x7f435014f790 2024-09-19T05:44:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.078+0000 7f43559bb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f435014f790 0x7f435015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.078+0000 7f43549b9640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f433406c2b0 0x7f433406e770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f435015bcd0 tx=0x7f4340009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.078+0000 7f43559bb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f435014f790 0x7f435015bd70 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.079+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f435015e3f0 con 0x7f435014f790 2024-09-19T05:44:23.095 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.093+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f43500b50e0 con 0x7f435014f790 2024-09-19T05:44:23.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.094+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f43500b50e0 con 0x7f435014f790 2024-09-19T05:44:23.097 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:23.097 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:23.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.095+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4350150f40 0x7f43500cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.095+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f435006ce80 con 0x7f4350150f40 2024-09-19T05:44:23.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.095+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4350150f40 0x7f43500cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.095+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4350150f40 0x7f43500cf2d0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.096+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f435006ce80 con 0x7f4350150f40 2024-09-19T05:44:23.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.110+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f43501f4230 con 0x7f4350150f40 2024-09-19T05:44:23.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.112+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f43501f4230 con 0x7f4350150f40 2024-09-19T05:44:23.114 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.112+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f43500cfd20 0x7f4350062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.112+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f43501f0f00 con 0x7f43500cfd20 2024-09-19T05:44:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.112+0000 7f43549b9640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f43500cfd20 0x7f4350062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.112+0000 7f43549b9640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f43500cfd20 0x7f4350062630 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.113+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f43501f0f00 con 0x7f43500cfd20 2024-09-19T05:44:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.127+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4350062e40 con 0x7f43500cfd20 2024-09-19T05:44:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.128+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4350062e40 con 0x7f43500cfd20 2024-09-19T05:44:23.131 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:23.131 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:23.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.129+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4350151dd0 0x7f43501521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.129+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f43501f10e0 con 0x7f4350151dd0 2024-09-19T05:44:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.129+0000 7f43559bb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4350151dd0 0x7f43501521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.129+0000 7f43559bb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4350151dd0 0x7f43501521f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.130+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4348002030 con 0x7f4350151dd0 2024-09-19T05:44:23.139 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:23.140 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:23.140 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:23.140 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:23.140 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:23.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.143+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4350062dc0 con 0x7f4350151dd0 2024-09-19T05:44:23.146 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.144+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4350062dc0 con 0x7f4350151dd0 2024-09-19T05:44:23.147 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.145+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f435002cb50 0x7f435002cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.145+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4350062dc0 con 0x7f435002cb50 2024-09-19T05:44:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.145+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f435002cb50 0x7f435002cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.145+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f435002cb50 0x7f435002cfb0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.146+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4350062dc0 con 0x7f435002cb50 2024-09-19T05:44:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.159+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4350062dc0 con 0x7f435002cb50 2024-09-19T05:44:23.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.162+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4350062dc0 con 0x7f435002cb50 2024-09-19T05:44:23.164 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:23.164 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:23.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.162+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4350204ba0 0x7f4350206f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.162+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f435002da00 con 0x7f4350204ba0 2024-09-19T05:44:23.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.162+0000 7f43549b9640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4350204ba0 0x7f4350206f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.162+0000 7f43549b9640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4350204ba0 0x7f4350206f90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.163+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.5 v2:172.21.15.123:6800/3690741328 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f435002da00 con 0x7f4350204ba0 2024-09-19T05:44:23.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.177+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f435002da00 con 0x7f4350204ba0 2024-09-19T05:44:23.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.179+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.5 v2:172.21.15.123:6800/3690741328 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f435002da00 con 0x7f4350204ba0 2024-09-19T05:44:23.182 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:23.182 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:23.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.180+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f43502079e0 0x7f4350209dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.180+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4350063490 con 0x7f43502079e0 2024-09-19T05:44:23.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.180+0000 7f43559bb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f43502079e0 0x7f4350209dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.180+0000 7f43559bb640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f43502079e0 0x7f4350209dd0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.181+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4350063490 con 0x7f43502079e0 2024-09-19T05:44:23.192 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:23.191+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:23.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.194+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4350075510 con 0x7f43502079e0 2024-09-19T05:44:23.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.196+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4350075510 con 0x7f43502079e0 2024-09-19T05:44:23.199 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:23.199 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:23.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.197+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f435020a820 0x7f435020cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:23.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.197+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f435020fb20 con 0x7f435020a820 2024-09-19T05:44:23.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.197+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f435020a820 0x7f435020cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:23.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.197+0000 7f43551ba640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f435020a820 0x7f435020cc10 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:23.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.198+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f435020fb20 con 0x7f435020a820 2024-09-19T05:44:23.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.211+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f435020d150 con 0x7f435020a820 2024-09-19T05:44:23.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.213+0000 7f4345ffb640 1 -- 172.21.15.2:0/3581880456 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f435020d150 con 0x7f435020a820 2024-09-19T05:44:23.216 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:23.217 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:23.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f435014f790 msgr2=0x7f435015bd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f435014f790 0x7f435015bd70 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4350150f40 msgr2=0x7f43500cf2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f4350150f40 0x7f43500cf2d0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f43500cfd20 msgr2=0x7f4350062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f43500cfd20 0x7f4350062630 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4350151dd0 msgr2=0x7f43501521f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f4350151dd0 0x7f43501521f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f435002cb50 msgr2=0x7f435002cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f435002cb50 0x7f435002cfb0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f43559bb640 1 -- 172.21.15.2:0/3581880456 reap_dead start 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4350204ba0 msgr2=0x7f4350206f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7f4350204ba0 0x7f4350206f90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f43559bb640 1 -- 172.21.15.2:0/3581880456 reap_dead start 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f43559bb640 1 -- 172.21.15.2:0/3581880456 reap_dead start 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f43502079e0 msgr2=0x7f4350209dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f43502079e0 0x7f4350209dd0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f435020a820 msgr2=0x7f435020cc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f435020a820 0x7f435020cc10 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f433406c2b0 msgr2=0x7f433406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f433406c2b0 0x7f433406e770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f435015bcd0 tx=0x7f4340009040 comp rx=0 tx=0).stop 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43500cd4e0 msgr2=0x7f435014f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43500cd4e0 0x7f435014f250 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f434c000c00 tx=0x7f434c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 shutdown_connections 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f435020a820 0x7f435020cc10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f43502079e0 0x7f4350209dd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f433406c2b0 0x7f433406e770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 --2- 172.21.15.2:0/3581880456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43500cd4e0 0x7f435014f250 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 >> 172.21.15.2:0/3581880456 conn(0x7f4350067b30 msgr2=0x7f43500cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 shutdown_connections 2024-09-19T05:44:23.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:23.214+0000 7f435741e640 1 -- 172.21.15.2:0/3581880456 wait complete. 2024-09-19T05:44:23.253 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:23.254 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:23.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:23.263 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:23.260+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:23.363 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:23.361+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:23.406 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:23.407 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:23.415 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:23.463 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:23.462+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:23.559 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:23.559 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:23.565 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:23.563+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:23.568 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:23.665 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:23.664+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:23.672 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:23.672 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:23.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:23.766 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:23.764+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:23.825 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:23.826 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:23.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:23.877 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:23.876+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:23.937 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:23.937 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:23.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:23.987 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:23.985+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:24.069 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:24.067+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:24.090 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:24.090 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:24.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:24.182 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:24.181+0000 7f273cba7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37598) UID: 0 2024-09-19T05:44:24.241 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:24.241 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:24.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:24.271 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:24.269+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:24.372 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:24.370+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:24.392 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:24.393 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:24.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:24.473 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:24.471+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:24.504 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:24.505 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:24.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:24.574 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:24.572+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:24.634 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:24.634 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:24.643 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:24.675 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:24.674+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:24.776 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:24.774+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:24.787 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:44:24.788 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:44:24.788 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-19T05:44:24.788 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-19T05:44:24.788 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:44:24.793 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:24.793 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:24.802 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:24.822 INFO:tasks.ceph.osd.5:Stopped 2024-09-19T05:44:24.822 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-19T05:44:24.823 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:44:24.823 DEBUG:teuthology.orchestra.run.smithi123:> type ceph-objectstore-tool 2024-09-19T05:44:24.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:44:24.877 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-19T05:44:24.877 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-19T05:44:24.905 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:24.903+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:24.911 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:24.911 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:24.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:24.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.965+0000 7ff2c554b640 1 Processor -- start 2024-09-19T05:44:24.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.965+0000 7ff2c554b640 1 -- start start 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2c554b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c7f40 0x7ff2c0152410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2c554b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c72a0 0x7ff2c0152950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2c554b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c015cd90 con 0x7ff2c00c6ec0 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2c554b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c00c86d0 con 0x7ff2c00c72a0 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2c554b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c00c8410 con 0x7ff2c00c7f40 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2be7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c7f40 0x7ff2c0152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2bf7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c72a0 0x7ff2c0152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2be7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c7f40 0x7ff2c0152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:34576/0 (socket says 172.21.15.2:34576) 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.967+0000 7ff2bf7fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c72a0 0x7ff2c0152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:56346/0 (socket says 172.21.15.2:56346) 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.966+0000 7ff2beffd640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff2c00c6ec0 0x7ff2c0158840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:38772/0 (socket says 172.21.15.2:38772) 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.967+0000 7ff2be7fc640 1 -- 172.21.15.2:0/949682503 learned_addr learned my addr 172.21.15.2:0/949682503 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.967+0000 7ff2be7fc640 1 -- 172.21.15.2:0/949682503 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c72a0 msgr2=0x7ff2c0152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.967+0000 7ff2be7fc640 1 --2- 172.21.15.2:0/949682503 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c72a0 0x7ff2c0152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.967+0000 7ff2be7fc640 1 -- 172.21.15.2:0/949682503 >> v1:172.21.15.123:6789/0 conn(0x7ff2c00c6ec0 legacy=0x7ff2c0158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.967+0000 7ff2be7fc640 1 -- 172.21.15.2:0/949682503 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c015bfd0 con 0x7ff2c00c7f40 2024-09-19T05:44:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.967+0000 7ff2bf7fe640 1 --2- 172.21.15.2:0/949682503 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c72a0 0x7ff2c0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2be7fc640 1 --2- 172.21.15.2:0/949682503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c7f40 0x7ff2c0152410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009870 tx=0x7ff2a802a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3381d9a9d5eba146 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2bdffb640 1 -- 172.21.15.2:0/949682503 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8037070 con 0x7ff2c00c7f40 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2bdffb640 1 -- 172.21.15.2:0/949682503 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a802ae50 con 0x7ff2c00c7f40 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2bdffb640 1 -- 172.21.15.2:0/949682503 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8032860 con 0x7ff2c00c7f40 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 -- 172.21.15.2:0/949682503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c7f40 msgr2=0x7ff2c0152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 --2- 172.21.15.2:0/949682503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c7f40 0x7ff2c0152410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009870 tx=0x7ff2a802a300 comp rx=0 tx=0).stop 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 -- 172.21.15.2:0/949682503 shutdown_connections 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 --2- 172.21.15.2:0/949682503 >> v2:172.21.15.123:3300/0 conn(0x7ff2c00c72a0 0x7ff2c0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 --2- 172.21.15.2:0/949682503 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c7f40 0x7ff2c0152410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 -- 172.21.15.2:0/949682503 >> 172.21.15.2:0/949682503 conn(0x7ff2c006cd20 msgr2=0x7ff2c006d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 -- 172.21.15.2:0/949682503 shutdown_connections 2024-09-19T05:44:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.968+0000 7ff2c554b640 1 -- 172.21.15.2:0/949682503 wait complete. 2024-09-19T05:44:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.970+0000 7ff2c554b640 1 Processor -- start 2024-09-19T05:44:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.970+0000 7ff2c554b640 1 -- start start 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2c554b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c00c6ec0 0x7ff2c00cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2c554b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c72a0 0x7ff2c01591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2c554b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c00c7f40 0x7ff2c01483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2c554b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c015c870 con 0x7ff2c00c6ec0 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2c554b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c015c590 con 0x7ff2c00c7f40 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2c554b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c0152e90 con 0x7ff2c00c72a0 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2beffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c00c6ec0 0x7ff2c00cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2beffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c00c6ec0 0x7ff2c00cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58606/0 (socket says 172.21.15.2:58606) 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2beffd640 1 -- 172.21.15.2:0/178762520 learned_addr learned my addr 172.21.15.2:0/178762520 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2be7fc640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c72a0 0x7ff2c01591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2bf7fe640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c00c7f40 0x7ff2c01483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2be7fc640 1 -- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c00c7f40 msgr2=0x7ff2c01483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2be7fc640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c00c7f40 0x7ff2c01483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2be7fc640 1 -- 172.21.15.2:0/178762520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c00c6ec0 msgr2=0x7ff2c00cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2be7fc640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c00c6ec0 0x7ff2c00cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2be7fc640 1 -- 172.21.15.2:0/178762520 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c00c4240 con 0x7ff2c00c72a0 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2bf7fe640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c00c7f40 0x7ff2c01483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.971+0000 7ff2beffd640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c00c6ec0 0x7ff2c00cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.972+0000 7ff2be7fc640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c72a0 0x7ff2c01591d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00c8590 tx=0x7ff2a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.972+0000 7ff2b77fe640 1 -- 172.21.15.2:0/178762520 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8041020 con 0x7ff2c00c72a0 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.972+0000 7ff2b77fe640 1 -- 172.21.15.2:0/178762520 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a803c870 con 0x7ff2c00c72a0 2024-09-19T05:44:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.972+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c01585b0 con 0x7ff2c00c72a0 2024-09-19T05:44:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.972+0000 7ff2b77fe640 1 -- 172.21.15.2:0/178762520 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8037040 con 0x7ff2c00c72a0 2024-09-19T05:44:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.972+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c01589d0 con 0x7ff2c00c72a0 2024-09-19T05:44:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.973+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c0071500 con 0x7ff2c00c72a0 2024-09-19T05:44:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.973+0000 7ff2b77fe640 1 -- 172.21.15.2:0/178762520 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff2a8044020 con 0x7ff2c00c72a0 2024-09-19T05:44:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.974+0000 7ff2b77fe640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a006c2b0 0x7ff2a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.974+0000 7ff2b77fe640 1 -- 172.21.15.2:0/178762520 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(67..67 src has 1..67) ==== 5799+0+0 (secure 0 0 0) 0x7ff2a80abab0 con 0x7ff2c00c72a0 2024-09-19T05:44:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.974+0000 7ff2beffd640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a006c2b0 0x7ff2a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.974+0000 7ff2beffd640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a006c2b0 0x7ff2a006e770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0000f60 tx=0x7ff2b0007f00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:24.978 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:24.976+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:24.978+0000 7ff2b77fe640 1 -- 172.21.15.2:0/178762520 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff2a804b030 con 0x7ff2c00c72a0 2024-09-19T05:44:25.072 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:25.072 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:25.079 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:25.077+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:25.081 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:25.170 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.168+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7ff2c014ba70 con 0x7ff2c00c72a0 2024-09-19T05:44:25.179 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:25.178+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:25.228 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:25.229 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:25.229 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:25.238 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:25.281 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:25.279+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:25.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.327+0000 7fef8b5b0640 1 Processor -- start 2024-09-19T05:44:25.329 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.327+0000 7fef8b5b0640 1 -- start start 2024-09-19T05:44:25.330 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8b5b0640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fef840cd4e0 0x7fef840b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8b5b0640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef8415aa90 0x7fef840b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8b5b0640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef8415ebe0 con 0x7fef840cd8c0 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8b5b0640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fef8415eec0 con 0x7fef840cd4e0 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8b5b0640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fef840b2e00 con 0x7fef8415aa90 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8934c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fef840cd4e0 0x7fef840b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8934c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fef840cd4e0 0x7fef840b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58612/0 (socket says 172.21.15.2:58612) 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef89b4d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fef840cd8c0 0x7fef8415df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:38556/0 (socket says 172.21.15.2:38556) 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8934c640 1 -- 172.21.15.2:0/3618734574 learned_addr learned my addr 172.21.15.2:0/3618734574 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef88b4b640 1 --2- 172.21.15.2:0/3618734574 >> v2:172.21.15.123:3300/0 conn(0x7fef8415aa90 0x7fef840b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8934c640 1 -- 172.21.15.2:0/3618734574 >> v2:172.21.15.123:3300/0 conn(0x7fef8415aa90 msgr2=0x7fef840b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8934c640 1 --2- 172.21.15.2:0/3618734574 >> v2:172.21.15.123:3300/0 conn(0x7fef8415aa90 0x7fef840b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.331 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8934c640 1 -- 172.21.15.2:0/3618734574 >> v1:172.21.15.2:6789/0 conn(0x7fef840cd8c0 legacy=0x7fef8415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:25.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.328+0000 7fef8934c640 1 -- 172.21.15.2:0/3618734574 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef840b30e0 con 0x7fef840cd4e0 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.329+0000 7fef8934c640 1 --2- 172.21.15.2:0/3618734574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef840cd4e0 0x7fef840b3bc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fef8415a940 tx=0x7fef8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6737c1165dedc47a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.329+0000 7fef7bfff640 1 -- 172.21.15.2:0/3618734574 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80037070 con 0x7fef840cd4e0 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.329+0000 7fef7bfff640 1 -- 172.21.15.2:0/3618734574 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef80032440 con 0x7fef840cd4e0 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.329+0000 7fef7bfff640 1 -- 172.21.15.2:0/3618734574 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80032810 con 0x7fef840cd4e0 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 -- 172.21.15.2:0/3618734574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef840cd4e0 msgr2=0x7fef840b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/3618734574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef840cd4e0 0x7fef840b3bc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fef8415a940 tx=0x7fef8002a270 comp rx=0 tx=0).stop 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 -- 172.21.15.2:0/3618734574 shutdown_connections 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/3618734574 >> v2:172.21.15.123:3300/0 conn(0x7fef8415aa90 0x7fef840b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/3618734574 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef840cd4e0 0x7fef840b3bc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 -- 172.21.15.2:0/3618734574 >> 172.21.15.2:0/3618734574 conn(0x7fef84067b30 msgr2=0x7fef84067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:25.333 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 -- 172.21.15.2:0/3618734574 shutdown_connections 2024-09-19T05:44:25.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.330+0000 7fef8b5b0640 1 -- 172.21.15.2:0/3618734574 wait complete. 2024-09-19T05:44:25.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.331+0000 7fef8b5b0640 1 Processor -- start 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.331+0000 7fef8b5b0640 1 -- start start 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef8b5b0640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef840cd4e0 0x7fef841e6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef8b5b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef840cd8c0 0x7fef8415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef8b5b0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef8415aa90 0x7fef841594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef8b5b0640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fef8415e640 con 0x7fef8415aa90 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef8b5b0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef840b36a0 con 0x7fef840cd8c0 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef8b5b0640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fef8415e7c0 con 0x7fef840cd4e0 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef8415aa90 0x7fef841594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef8415aa90 0x7fef841594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58616/0 (socket says 172.21.15.2:58616) 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 -- 172.21.15.2:0/460346772 learned_addr learned my addr 172.21.15.2:0/460346772 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef840cd8c0 0x7fef8415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef8934c640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef840cd4e0 0x7fef841e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef840cd4e0 msgr2=0x7fef841e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef840cd4e0 0x7fef841e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef840cd8c0 msgr2=0x7fef8415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef840cd8c0 0x7fef8415d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.332+0000 7fef89b4d640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef841554e0 con 0x7fef8415aa90 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.333+0000 7fef89b4d640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef8415aa90 0x7fef841594d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00c7d0 tx=0x7fef7c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.333+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c017070 con 0x7fef8415aa90 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.333+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef7c00a5e0 con 0x7fef8415aa90 2024-09-19T05:44:25.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.333+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef84158710 con 0x7fef8415aa90 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.333+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00a990 con 0x7fef8415aa90 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.333+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef84158cd0 con 0x7fef8415aa90 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.334+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fef8415df30 con 0x7fef8415aa90 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.334+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fef7c01a070 con 0x7fef8415aa90 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.335+0000 7fef79ffb640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef6806c2b0 0x7fef6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.335+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5799+0+0 (secure 0 0 0) 0x7fef7c08a950 con 0x7fef8415aa90 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.335+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v67) ==== 54+0+16 (secure 0 0 0) 0x7fef7c08f240 con 0x7fef8415aa90 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.335+0000 7fef8934c640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef6806c2b0 0x7fef6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.335+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fef8415cca0 0x7fef840cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.335+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef8406d490 con 0x7fef8415cca0 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.336+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fef8415cca0 0x7fef840cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.336+0000 7fef8934c640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef6806c2b0 0x7fef6806e770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fef80034000 tx=0x7fef8002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.336+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fef8415cca0 0x7fef840cbc60 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.337+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef8406d490 con 0x7fef8415cca0 2024-09-19T05:44:25.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.351+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fef840cc1a0 con 0x7fef8415cca0 2024-09-19T05:44:25.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.353+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fef840cc1a0 con 0x7fef8415cca0 2024-09-19T05:44:25.355 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:25.356 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:25.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.353+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fef841f3e70 0x7fef841f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.353+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fef841f4790 con 0x7fef841f3e70 2024-09-19T05:44:25.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.353+0000 7fef8934c640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fef841f3e70 0x7fef841f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.356 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.354+0000 7fef8934c640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fef841f3e70 0x7fef841f4250 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.355+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef841f4790 con 0x7fef841f3e70 2024-09-19T05:44:25.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.369+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fef8415da40 con 0x7fef841f3e70 2024-09-19T05:44:25.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.372+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fef8415da40 con 0x7fef841f3e70 2024-09-19T05:44:25.374 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:25.375 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:25.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.372+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fef84154840 0x7fef84154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.372+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fef84155160 con 0x7fef84154840 2024-09-19T05:44:25.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.372+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fef84154840 0x7fef84154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.373+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fef84154840 0x7fef84154c20 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.374+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef84155160 con 0x7fef84154840 2024-09-19T05:44:25.382 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:25.380+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:25.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.387+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fef84062900 con 0x7fef84154840 2024-09-19T05:44:25.391 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:25.391 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:25.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.509779849, 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:44:25.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:44:25.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:44:25.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:7 3.2a 3:557f550e:::10000000002.00000004:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.106331544, 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27494626900000002, 2024-09-19T05:44:25.398 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 7 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282495+0000", 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4609999999999998e-06 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286920+0000", 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044248229999999996 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286972+0000", 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1659000000000001e-05 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286986+0000", 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-19T05:44:25.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287117+0000", 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131328 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287358+0000", 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024141700000000001 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287543+0000", 2024-09-19T05:44:25.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018436700000000001 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299025+0000", 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011482327000000001 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557305+0000", 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25828001099999998 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557360+0000", 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5074000000000002e-05 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557439+0000", 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9018999999999997e-05 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:92 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.973183526, 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37051118500000002, 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:25.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415644+0000", 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.422983+0000", 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073390779999999997 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423020+0000", 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7660000000000002e-05 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423036+0000", 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5359999999999999e-05 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423261+0000", 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225217 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423458+0000", 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196555 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423737+0000", 2024-09-19T05:44:25.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027922100000000002 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.435447+0000", 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011709855999999999 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.785961+0000", 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35051446400000003 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786086+0000", 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012537599999999999 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786152+0000", 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691000000000005e-05 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:104 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.412 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.958957204000001, 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28556600799999998, 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429870+0000", 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-06 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446619+0000", 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016749447000000001 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446650+0000", 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0218000000000001e-05 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446666+0000", 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6920999999999999e-05 2024-09-19T05:44:25.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448857+0000", 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021902699999999998 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449011+0000", 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015441700000000001 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449168+0000", 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156527 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489828+0000", 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040659908000000002 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715285+0000", 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22545759500000001 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715345+0000", 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0269999999999997e-05 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715433+0000", 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7814999999999999e-05 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.418 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:108 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.942106187, 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26890963299999998, 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446721+0000", 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8250000000000001e-06 2024-09-19T05:44:25.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451943+0000", 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052222140000000002 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451961+0000", 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8117e-05 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451970+0000", 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7420000000000003e-06 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473034+0000", 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021064282 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473242+0000", 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020716699999999999 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473446+0000", 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204396 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489832+0000", 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016385994000000001 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715506+0000", 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22567427700000001 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715577+0000", 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0548000000000003e-05 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715628+0000", 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070999999999999e-05 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:139 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.925294781000002, 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.328123891, 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.425 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463532+0000", 2024-09-19T05:44:25.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0339999999999999e-06 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468331+0000", 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047995099999999999 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468365+0000", 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3667000000000001e-05 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468382+0000", 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6651e-05 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468915+0000", 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053293299999999997 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469199+0000", 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028406900000000002 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469482+0000", 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028390000000000002 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.485026+0000", 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015543509 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791538+0000", 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30651197400000002 2024-09-19T05:44:25.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791602+0000", 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3780000000000003e-05 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791654+0000", 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1863999999999997e-05 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:148 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.906844747000001, 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34695000399999998, 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 148 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481982+0000", 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6309999999999999e-06 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489765+0000", 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077821310000000003 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489778+0000", 2024-09-19T05:44:25.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3604999999999999e-05 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489787+0000", 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489875+0000", 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8039000000000002e-05 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490101+0000", 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022636299999999999 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490426+0000", 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032453799999999997 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.506325+0000", 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015899149000000001 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828797+0000", 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32247239900000002 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828865+0000", 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7521999999999995e-05 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828930+0000", 2024-09-19T05:44:25.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4795999999999994e-05 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:156 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.898989539999999, 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26935146300000001, 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:25.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489837+0000", 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2689999999999998e-06 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495152+0000", 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053150209999999996 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495356+0000", 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203537 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495373+0000", 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496071+0000", 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069810200000000003 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496246+0000", 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017547299999999999 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496469+0000", 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022283200000000001 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.540840+0000", 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044371318999999999 2024-09-19T05:44:25.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759082+0000", 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21824185500000001 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759137+0000", 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645000000000003e-05 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759186+0000", 2024-09-19T05:44:25.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9499000000000002e-05 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:185 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.886309696000001, 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32688410400000001, 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 185 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502518+0000", 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.509973+0000", 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074553919999999999 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510031+0000", 2024-09-19T05:44:25.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8356999999999997e-05 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510046+0000", 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932e-05 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510135+0000", 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8548000000000007e-05 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510886+0000", 2024-09-19T05:44:25.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075159700000000001 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511079+0000", 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019261500000000001 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532794+0000", 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021714989000000001 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829312+0000", 2024-09-19T05:44:25.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29651747699999997 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829360+0000", 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8396000000000002e-05 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829399+0000", 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8961999999999997e-05 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:264 3.1c 3:3ae0713d:::10000000002.00000005:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.796390985999999, 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29060550499999999, 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 264 2024-09-19T05:44:25.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592436+0000", 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5629999999999999e-06 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595027+0000", 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025912600000000002 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595055+0000", 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8164999999999999e-05 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595069+0000", 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3953e-05 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595156+0000", 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6981e-05 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595260+0000", 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010363 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595417+0000", 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015732100000000001 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.605722+0000", 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010305099999999999 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882929+0000", 2024-09-19T05:44:25.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27720663499999998 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882988+0000", 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9237000000000002e-05 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.883039+0000", 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0659999999999999e-05 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:315 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.757430666000001, 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55644120600000002, 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 315 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631396+0000", 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5490000000000001e-06 2024-09-19T05:44:25.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636257+0000", 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048607859999999998 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636331+0000", 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3992000000000006e-05 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636355+0000", 2024-09-19T05:44:25.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3813999999999999e-05 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636445+0000", 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0180999999999996e-05 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636569+0000", 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123991 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636849+0000", 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000279664 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659414+0000", 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022565053000000002 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187715+0000", 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52830067000000003 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187783+0000", 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8509000000000004e-05 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187835+0000", 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1996999999999997e-05 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:347 3.2a 3:5747858d:::10000000002.00000058:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:25.460 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.749357840000002, 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48789592399999998, 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 347 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:25.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639470+0000", 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.145e-06 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644022+0000", 2024-09-19T05:44:25.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045525940000000001 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646323+0000", 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023007879999999998 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646342+0000", 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8766999999999998e-05 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646435+0000", 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3394999999999996e-05 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646564+0000", 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012848800000000001 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646738+0000", 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017418499999999999 2024-09-19T05:44:25.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.741685+0000", 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094946825999999998 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127231+0000", 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.385546526 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127301+0000", 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9741999999999997e-05 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127363+0000", 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1468000000000002e-05 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:348 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.466 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.74245535, 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54159870300000001, 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 348 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646372+0000", 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9110000000000002e-06 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.678772+0000", 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032399869999999997 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681572+0000", 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002799714 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681592+0000", 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0298000000000001e-05 2024-09-19T05:44:25.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682777+0000", 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011847190000000001 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682909+0000", 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000132146 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.683102+0000", 2024-09-19T05:44:25.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019280499999999999 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.702631+0000", 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019529271000000001 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187923+0000", 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48529252 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187948+0000", 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5004999999999999e-05 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187968+0000", 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9443999999999999e-05 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:360 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.699078934999999, 2024-09-19T05:44:25.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43778335400000001, 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 360 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689748+0000", 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7690000000000001e-06 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692563+0000", 2024-09-19T05:44:25.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028147329999999998 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692593+0000", 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9476999999999999e-05 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692616+0000", 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3340999999999999e-05 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723279+0000", 2024-09-19T05:44:25.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030662759000000001 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723430+0000", 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151247 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723629+0000", 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199426 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793213+0000", 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069583435999999999 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127443+0000", 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33423067000000001 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127491+0000", 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8000999999999997e-05 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127529+0000", 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7494999999999999e-05 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:364 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.696175218, 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43506039200000002, 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 364 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692652+0000", 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9280000000000002e-06 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697100+0000", 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004447678 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701619+0000", 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045192829999999998 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701638+0000", 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8365000000000001e-05 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742561+0000", 2024-09-19T05:44:25.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040923560999999997 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742686+0000", 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012482899999999999 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742839+0000", 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015348 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808887+0000", 2024-09-19T05:44:25.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066047260999999996 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127632+0000", 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31874497099999999 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127672+0000", 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0244999999999998e-05 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127710+0000", 2024-09-19T05:44:25.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7790999999999998e-05 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:395 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.683433374, 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48275294299999999, 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.484 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 395 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705394+0000", 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7800000000000001e-06 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720169+0000", 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014775346 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720185+0000", 2024-09-19T05:44:25.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5721000000000001e-05 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720194+0000", 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4609999999999997e-06 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720294+0000", 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100005 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720392+0000", 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7547999999999995e-05 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720531+0000", 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000139099 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.767968+0000", 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047436846999999997 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188062+0000", 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42009412899999998 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188107+0000", 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5269000000000003e-05 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188144+0000", 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6738e-05 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:404 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.667996208000002, 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46750329699999998, 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 404 2024-09-19T05:44:25.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720832+0000", 2024-09-19T05:44:25.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8469999999999998e-06 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726318+0000", 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005485775 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726490+0000", 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172339 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726505+0000", 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4467999999999999e-05 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743219+0000", 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016714046 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743320+0000", 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101548 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743459+0000", 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013899499999999999 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.775025+0000", 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031565658000000003 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188247+0000", 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.413221583 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188297+0000", 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0586e-05 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188332+0000", 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4452000000000002e-05 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.495 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:412 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.650377404, 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40023217100000003, 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 412 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738450+0000", 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7e-06 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.758682+0000", 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020232357999999999 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772625+0000", 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013943232 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.774996+0000", 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023703890000000001 2024-09-19T05:44:25.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793240+0000", 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018244646999999999 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793347+0000", 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106276 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793511+0000", 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016406099999999999 2024-09-19T05:44:25.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808895+0000", 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015384296 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138570+0000", 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32967460500000001 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138630+0000", 2024-09-19T05:44:25.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0087999999999998e-05 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138679+0000", 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9518999999999998e-05 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:441 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:25.501 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.594668855999998, 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39435722000000001, 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 441 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:25.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794159+0000", 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.579e-06 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.802843+0000", 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0086840579999999997 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.807986+0000", 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051424890000000001 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808000+0000", 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808113+0000", 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011294900000000001 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808200+0000", 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6792999999999994e-05 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808612+0000", 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041192799999999999 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827676+0000", 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019063875000000001 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188432+0000", 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36075675699999998 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188479+0000", 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6149000000000001e-05 2024-09-19T05:44:25.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188513+0000", 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4310999999999997e-05 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:454 3.37 3:ec162ecd:::10000000002.000000c3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.555905383999999, 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32323945999999998, 2024-09-19T05:44:25.507 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 454 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832922+0000", 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6280000000000001e-06 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839384+0000", 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064625180000000004 2024-09-19T05:44:25.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850436+0000", 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011051365 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850464+0000", 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8719000000000001e-05 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850532+0000", 2024-09-19T05:44:25.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7212000000000006e-05 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850663+0000", 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013160799999999999 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850877+0000", 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000213716 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871083+0000", 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020205864 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156086+0000", 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.285003532 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156125+0000", 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8569999999999998e-05 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156159+0000", 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3728e-05 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:25.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.389+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fef84062900 con 0x7fef84154840 2024-09-19T05:44:25.514 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:25.514 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.390+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fef841e6f40 0x7fef841e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.390+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fef840c71f0 con 0x7fef841e6f40 2024-09-19T05:44:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.390+0000 7fef89b4d640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fef841e6f40 0x7fef841e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.390+0000 7fef89b4d640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fef841e6f40 0x7fef841e73a0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.391+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef840c71f0 con 0x7fef841e6f40 2024-09-19T05:44:25.516 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:25.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.404+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fef840c71f0 con 0x7fef841e6f40 2024-09-19T05:44:25.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.406+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fef840c71f0 con 0x7fef841e6f40 2024-09-19T05:44:25.517 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:25.518 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:25.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.406+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fef841e7ca0 0x7fef8402cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.406+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fef840c71f0 con 0x7fef841e7ca0 2024-09-19T05:44:25.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.407+0000 7fef8934c640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fef841e7ca0 0x7fef8402cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:25.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.407+0000 7fef8934c640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fef841e7ca0 0x7fef8402cb50 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:25.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.408+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef840c71f0 con 0x7fef841e7ca0 2024-09-19T05:44:25.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.422+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fef840008d0 con 0x7fef841e7ca0 2024-09-19T05:44:25.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.424+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fef840008d0 con 0x7fef841e7ca0 2024-09-19T05:44:25.519 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:25.519 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:25.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.424+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 0x7fef84206f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:25.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.424+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fef8402da20 con 0x7fef8402d5a0 2024-09-19T05:44:25.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.424+0000 7fef88b4b640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 msgr2=0x7fef84206f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3690741328 2024-09-19T05:44:25.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.424+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 0x7fef84206f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T05:44:25.557 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:25.555+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:25.619 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:25.617+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.625+0000 7fef88b4b640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 msgr2=0x7fef84206f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3690741328 2024-09-19T05:44:25.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.625+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 0x7fef84206f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T05:44:25.664 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:25.664 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:25.664 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:25.664 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:25.673 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:25.720 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:25.719+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:25.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.791+0000 7ff2b77fe640 1 -- 172.21.15.2:0/178762520 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v68) ==== 87+0+0 (secure 0 0 0) 0x7ff2a807a1b0 con 0x7ff2c00c72a0 2024-09-19T05:44:25.793 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a006c2b0 msgr2=0x7ff2a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a006c2b0 0x7ff2a006e770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0000f60 tx=0x7ff2b0007f00 comp rx=0 tx=0).stop 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c72a0 msgr2=0x7ff2c01591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c72a0 0x7ff2c01591d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00c8590 tx=0x7ff2a80046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 shutdown_connections 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff2a006c2b0 0x7ff2a006e770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c00c7f40 0x7ff2c01483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff2c00c72a0 0x7ff2c01591d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 --2- 172.21.15.2:0/178762520 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff2c00c6ec0 0x7ff2c00cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 >> 172.21.15.2:0/178762520 conn(0x7ff2c006cd20 msgr2=0x7ff2c0151120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:25.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 shutdown_connections 2024-09-19T05:44:25.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:25.793+0000 7ff2c554b640 1 -- 172.21.15.2:0/178762520 wait complete. 2024-09-19T05:44:25.821 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:25.820+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:25.824 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:25.824 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:25.824 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:25.824 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:25.824 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:25.833 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:25.921 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:25.920+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:25.982 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:77 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.631890274, 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40221933999999998, 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 77 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348156+0000", 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.348160+0000", 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0459999999999999e-06 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362035+0000", 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013874391999999999 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362063+0000", 2024-09-19T05:44:25.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7804e-05 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362076+0000", 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3200000000000001e-05 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362162+0000", 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6469000000000002e-05 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362317+0000", 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015507100000000001 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.362454+0000", 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013636799999999999 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.372327+0000", 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0098729219999999993 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750183+0000", 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37785643600000002 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750233+0000", 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9960999999999998e-05 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.750376+0000", 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142671 2024-09-19T05:44:25.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:87 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.597425348000002, 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39257614200000002, 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 87 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382621+0000", 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382623+0000", 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2529999999999999e-06 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390766+0000", 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081425790000000005 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390808+0000", 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2311999999999997e-05 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390827+0000", 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8678999999999999e-05 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.390955+0000", 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012825900000000001 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391148+0000", 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000192673 2024-09-19T05:44:25.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.391432+0000", 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028362599999999998 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.455303+0000", 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063870983000000006 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775090+0000", 2024-09-19T05:44:25.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31978768600000002 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775146+0000", 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5393000000000003e-05 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.775197+0000", 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1699000000000001e-05 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:98 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.542216037000003, 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42340282099999998, 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 98 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:25.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437831+0000", 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437834+0000", 2024-09-19T05:44:25.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.442377+0000", 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045437719999999997 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448982+0000", 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066041210000000001 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:25.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449009+0000", 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7898999999999998e-05 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449078+0000", 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8301999999999998e-05 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449208+0000", 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013036300000000001 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449349+0000", 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000140791 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515702+0000", 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066353102999999997 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861115+0000", 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34541303600000001 2024-09-19T05:44:25.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861183+0000", 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8238999999999995e-05 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.861233+0000", 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0081999999999999e-05 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:143 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.507084134000003, 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40938193499999997, 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 143 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.001 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472962+0000", 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.472965+0000", 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7879999999999998e-06 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486591+0000", 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013625417000000001 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486611+0000", 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0285e-05 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486624+0000", 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2763e-05 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501834+0000", 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01521081 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504117+0000", 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022829479999999999 2024-09-19T05:44:26.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.504306+0000", 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018812599999999999 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.515709+0000", 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011403529000000001 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882232+0000", 2024-09-19T05:44:26.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36652316899999998 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882300+0000", 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7539000000000005e-05 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882344+0000", 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4561000000000002e-05 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:145 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.493396410999999, 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39312144199999999, 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486650+0000", 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.486653+0000", 2024-09-19T05:44:26.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0089999999999999e-06 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491674+0000", 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050208450000000003 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491698+0000", 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4457999999999999e-05 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.491721+0000", 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2294999999999999e-05 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510737+0000", 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019016000000000002 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510880+0000", 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014287999999999999 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511053+0000", 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017286299999999999 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.555682+0000", 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044629755 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879645+0000", 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.323962272 2024-09-19T05:44:26.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879715+0000", 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0400000000000004e-05 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.879772+0000", 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6665e-05 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:175 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 32.452831027999999, 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35525569800000001, 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 175 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.013 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527216+0000", 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.527221+0000", 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2299999999999999e-06 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.531495+0000", 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004274211 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556121+0000", 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024626329999999998 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556132+0000", 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0740000000000001e-05 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556187+0000", 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5096000000000002e-05 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556372+0000", 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185059 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.556560+0000", 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000187803 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.562115+0000", 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0055550449999999998 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882386+0000", 2024-09-19T05:44:26.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32027064199999999 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882435+0000", 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8990999999999999e-05 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882471+0000", 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6551000000000003e-05 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:298 3.26 3:64018a55:::10000000002.00000027:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.367739233999998, 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59287720300000002, 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 298 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612307+0000", 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612310+0000", 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.757e-06 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619160+0000", 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068500490000000004 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619231+0000", 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0642000000000006e-05 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619249+0000", 2024-09-19T05:44:26.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8544000000000001e-05 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619441+0000", 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019125500000000001 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619600+0000", 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015895800000000001 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619776+0000", 2024-09-19T05:44:26.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017654300000000001 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.630552+0000", 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010775523 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205074+0000", 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57452220799999998 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205132+0000", 2024-09-19T05:44:26.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8038999999999998e-05 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.205185+0000", 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2685000000000001e-05 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:323 3.16 3:6a386151:::10000000002.00000040:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.335510338999999, 2024-09-19T05:44:26.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.567029063, 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 323 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:26.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644536+0000", 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644539+0000", 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5040000000000001e-06 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649094+0000", 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045552429999999996 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649120+0000", 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6176000000000001e-05 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649138+0000", 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8046e-05 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649227+0000", 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8505999999999996e-05 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649330+0000", 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010301700000000001 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.649482+0000", 2024-09-19T05:44:26.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015256300000000001 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.719941+0000", 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.070459037000000002 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211483+0000", 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49154185500000003 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211531+0000", 2024-09-19T05:44:26.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8026999999999999e-05 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211565+0000", 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4088999999999998e-05 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:269 3.2 e54/20 3:438a76aa:::10000000002.0000000a:head v 54'6, mlcod=53'4)", 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:26.030 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.335407647, 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37357711900000001, 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644639+0000", 2024-09-19T05:44:26.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644640+0000", 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.192e-06 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976810+0000", 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33216997700000001 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976833+0000", 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2878e-05 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976847+0000", 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3644e-05 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.976972+0000", 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125487 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.977007+0000", 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5027000000000002e-05 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.017830+0000", 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040822590999999998 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.018216+0000", 2024-09-19T05:44:26.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038632299999999999 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:333 3.3c 3:3db0268f:::10000000002.0000004a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.329280169, 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39671826999999998, 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 333 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650766+0000", 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.650769+0000", 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0110000000000001e-06 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657005+0000", 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062355090000000002 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657046+0000", 2024-09-19T05:44:26.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.074e-05 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657054+0000", 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3310000000000004e-06 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657132+0000", 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7696999999999998e-05 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657256+0000", 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012425000000000001 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657471+0000", 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021535099999999999 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.715589+0000", 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058117380000000003 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047372+0000", 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33178333599999998 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047417+0000", 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5423000000000002e-05 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.047485+0000", 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7242000000000004e-05 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:343 3.3a 3:5da5f84e:::10000000002.00000054:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.31583109, 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39137022999999999, 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 343 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664215+0000", 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.664218+0000", 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5380000000000001e-06 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666946+0000", 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027284710000000001 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666962+0000", 2024-09-19T05:44:26.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5620999999999998e-05 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.666973+0000", 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1199000000000001e-05 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667053+0000", 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9689000000000002e-05 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667166+0000", 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011285699999999999 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667509+0000", 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00034300599999999997 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.675954+0000", 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084449580000000007 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055481+0000", 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37952754500000002 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055535+0000", 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3987999999999999e-05 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.055586+0000", 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0358000000000001e-05 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.046 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:354 3.26 3:6477918e:::10000000002.0000005f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.236639943, 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.468021299, 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 354 2024-09-19T05:44:26.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743407+0000", 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743409+0000", 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7369999999999998e-06 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.747060+0000", 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036510430000000001 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768120+0000", 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021059484 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768139+0000", 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9085000000000001e-05 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768289+0000", 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150252 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768387+0000", 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8169000000000001e-05 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.768573+0000", 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185973 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.776933+0000", 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083599629999999998 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211285+0000", 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43435194300000002 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211363+0000", 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7985000000000006e-05 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211428+0000", 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4665000000000005e-05 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:399 3.26 3:6771a79d:::10000000002.0000008c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.195054067000001, 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.455193454, 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 399 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784992+0000", 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.784997+0000", 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.791076+0000", 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00607888 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793492+0000", 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024155940000000001 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793539+0000", 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7246999999999997e-05 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793694+0000", 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000155589 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793818+0000", 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012335499999999999 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794028+0000", 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000209822 2024-09-19T05:44:26.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839319+0000", 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045291168999999999 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240115+0000", 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40079620599999999 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240147+0000", 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1782000000000002e-05 2024-09-19T05:44:26.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240186+0000", 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9168999999999997e-05 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:401 3.16 3:6bf10b49:::10000000002.0000008e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.182726256999999, 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41436097300000002, 2024-09-19T05:44:26.058 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 401 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797320+0000", 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.797324+0000", 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6239999999999999e-06 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803827+0000", 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065027219999999998 2024-09-19T05:44:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803845+0000", 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8717999999999999e-05 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803858+0000", 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2599000000000001e-05 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.803963+0000", 2024-09-19T05:44:26.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010488 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.804102+0000", 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013866900000000001 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.804299+0000", 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000197624 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.861232+0000", 2024-09-19T05:44:26.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056932356000000003 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211606+0000", 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35037407999999998 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211650+0000", 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4629000000000002e-05 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211681+0000", 2024-09-19T05:44:26.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1072000000000003e-05 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:431 3.26 3:66bf65cf:::10000000002.000000ac:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.126333519999999, 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38660652099999998, 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 431 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853713+0000", 2024-09-19T05:44:26.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.853715+0000", 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.95e-06 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862660+0000", 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008945138 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862684+0000", 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4102999999999999e-05 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862697+0000", 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2899e-05 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862819+0000", 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012190500000000001 2024-09-19T05:44:26.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863007+0000", 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018809600000000001 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.865417+0000", 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002410316 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967698+0000", 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102280148 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240225+0000", 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27252755200000001 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240282+0000", 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6935999999999998e-05 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240320+0000", 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7478000000000003e-05 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:432 3.26 3:66aef6e9:::10000000002.000000ad:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.117314615000002, 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37769895199999998, 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 432 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862732+0000", 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862736+0000", 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1749999999999998e-06 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868431+0000", 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005694569 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868467+0000", 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5809000000000002e-05 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.868482+0000", 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.52e-05 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896790+0000", 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028308262000000001 2024-09-19T05:44:26.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896919+0000", 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012855399999999999 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.897240+0000", 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032181899999999999 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037309+0000", 2024-09-19T05:44:26.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.140068843 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240357+0000", 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20304760699999999 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240400+0000", 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2849000000000003e-05 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240431+0000", 2024-09-19T05:44:26.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1265e-05 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:457 3.26 3:64ff7f1d:::10000000002.000000c6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.094016746000001, 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35451085300000001, 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.076 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 457 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886030+0000", 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.886033+0000", 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889815+0000", 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037826800000000001 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889868+0000", 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2592000000000001e-05 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.889881+0000", 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3251000000000001e-05 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912234+0000", 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022353101 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912416+0000", 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000181695 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.912603+0000", 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000186536 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037342+0000", 2024-09-19T05:44:26.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.124739117 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240463+0000", 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.203121157 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240505+0000", 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2406e-05 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240541+0000", 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5488999999999999e-05 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:460 3.26 3:6430d4f3:::10000000002.000000c9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.084172120000002, 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34478341699999998, 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.082 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 460 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895874+0000", 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.895877+0000", 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.779e-06 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898920+0000", 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030423379999999999 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898937+0000", 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7782999999999999e-05 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898948+0000", 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665e-05 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953289+0000", 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054340672999999999 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953400+0000", 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111527 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.953557+0000", 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015719599999999999 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.130946+0000", 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17738901100000001 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240578+0000", 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10963127 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240625+0000", 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7429000000000003e-05 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.240658+0000", 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2746000000000001e-05 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.087 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:463 3.26 3:65689812:::10000000002.000000cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.081071199, 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34245139299999999, 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 463 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898975+0000", 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.898979+0000", 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2610000000000002e-06 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902351+0000", 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033725270000000002 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902372+0000", 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0699e-05 2024-09-19T05:44:26.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902388+0000", 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6461e-05 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967734+0000", 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065346107 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.967874+0000", 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013911199999999999 2024-09-19T05:44:26.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.968092+0000", 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218438 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.204745+0000", 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23665298800000001 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241370+0000", 2024-09-19T05:44:26.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036625433999999998 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241402+0000", 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1523999999999998e-05 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241427+0000", 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4842e-05 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:467 3.26 3:65bb0d93:::10000000002.000000d0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.077626775999999, 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34422604099999998, 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 467 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.094 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902420+0000", 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902423+0000", 2024-09-19T05:44:26.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.117e-06 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908646+0000", 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062233660000000001 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908664+0000", 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7880000000000002e-05 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908679+0000", 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5129e-05 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211711+0000", 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30303182099999998 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211794+0000", 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2541999999999997e-05 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.211940+0000", 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014651900000000001 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.241451+0000", 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029510593000000002 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246560+0000", 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051091019999999999 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246605+0000", 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123999999999997e-05 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.246646+0000", 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0847999999999999e-05 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.099 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:26.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:26.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:26.026+0000 7fef88b4b640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 msgr2=0x7fef84206f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3690741328 2024-09-19T05:44:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:26.026+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 0x7fef84206f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T05:44:26.102 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:26.100+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:26.187 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:26.187 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:26.187 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:26.188 INFO:tasks.workunit.client.0.smithi002.stdout:iozone test complete. 2024-09-19T05:44:26.188 INFO:tasks.workunit.client.0.smithi002.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-19T05:44:26.201 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:26.200+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:26.253 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:26.253 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:26.253 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:26.253 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:26.254 INFO:tasks.workunit.client.0.smithi002.stdout: Iozone: Performance Test of File I/O 2024-09-19T05:44:26.254 INFO:tasks.workunit.client.0.smithi002.stdout: Version $Revision: 3.494 $ 2024-09-19T05:44:26.254 INFO:tasks.workunit.client.0.smithi002.stdout: Compiled for 64 bit mode. 2024-09-19T05:44:26.254 INFO:tasks.workunit.client.0.smithi002.stdout: Build: linux-AMD64 2024-09-19T05:44:26.254 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:26.254 INFO:tasks.workunit.client.0.smithi002.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Run began: Thu Sep 19 05:44:26 2024 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Include close in write timing 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: Include fsync in write timing 2024-09-19T05:44:26.255 INFO:tasks.workunit.client.0.smithi002.stdout: File size set to 10485760 kB 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Record Size 1024 kB 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Output is in kBytes/sec 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Time Resolution = 0.000001 seconds. 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Processor cache size set to 1024 kBytes. 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Processor cache line size set to 32 bytes. 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: File stride size set to 17 * record size. 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Throughput test with 1 process 2024-09-19T05:44:26.256 INFO:tasks.workunit.client.0.smithi002.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-19T05:44:26.263 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:26.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-19T05:44:26.309 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:26.307+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:26.351 INFO:teuthology.orchestra.run.smithi123.stdout:3.4 2024-09-19T05:44:26.351 INFO:teuthology.orchestra.run.smithi123.stdout:3.3e 2024-09-19T05:44:26.351 INFO:teuthology.orchestra.run.smithi123.stdout:3.3d 2024-09-19T05:44:26.351 INFO:teuthology.orchestra.run.smithi123.stdout:3.3b 2024-09-19T05:44:26.351 INFO:teuthology.orchestra.run.smithi123.stdout:3.38 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.31 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.2a 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.27 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.23 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.22 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.1f 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.1e 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.1a 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.18 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.17 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.10 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.29 2024-09-19T05:44:26.352 INFO:teuthology.orchestra.run.smithi123.stdout:3.0 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:3.28 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.e 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.a 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.4 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.1e 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.1b 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.d 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.1 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.f 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:2.0 2024-09-19T05:44:26.353 INFO:teuthology.orchestra.run.smithi123.stdout:1.0 2024-09-19T05:44:26.379 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:26.379 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:26.379 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:26.379 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:26.379 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:26.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:26.432 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:26.430+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:26.505 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:26.504+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:26.550 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:26.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:11 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.258589585000003, 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53368345399999995, 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287040+0000", 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287043+0000", 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294617+0000", 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075741280000000003 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294641+0000", 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3424000000000001e-05 2024-09-19T05:44:26.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294656+0000", 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4679e-05 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.294841+0000", 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185176 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295155+0000", 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031388199999999997 2024-09-19T05:44:26.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.295811+0000", 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065616700000000003 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.308016+0000", 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012205472 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820590+0000", 2024-09-19T05:44:26.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51257333400000005 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820658+0000", 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8836000000000003e-05 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.820724+0000", 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5426999999999995e-05 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:18 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.207878581999999, 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48181817700000001, 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 18 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337751+0000", 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.337754+0000", 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:26.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346234+0000", 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084795919999999993 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346309+0000", 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4950999999999994e-05 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346324+0000", 2024-09-19T05:44:26.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5199e-05 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346456+0000", 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013213799999999999 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346703+0000", 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024689500000000001 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.346893+0000", 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019002400000000001 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.382206+0000", 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035313485999999998 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819443+0000", 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43723664899999998 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819516+0000", 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3021000000000004e-05 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.819570+0000", 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3696999999999998e-05 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:37 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:26.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.128327179999999, 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.412251706, 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 37 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:26.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417303+0000", 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.417306+0000", 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3790000000000002e-06 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421246+0000", 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039399869999999998 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421273+0000", 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6412999999999999e-05 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421284+0000", 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1856e-05 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.421374+0000", 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9411000000000002e-05 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429518+0000", 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008143908 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429741+0000", 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022306100000000001 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.460095+0000", 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030354269999999999 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829462+0000", 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36936722900000002 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829510+0000", 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7698999999999999e-05 2024-09-19T05:44:26.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829555+0000", 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4493000000000002e-05 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:53 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.115159789000003, 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45610759200000001, 2024-09-19T05:44:26.568 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 53 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430470+0000", 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.430472+0000", 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1320000000000001e-06 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.437581+0000", 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071082300000000001 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444289+0000", 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067080070000000002 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.444322+0000", 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3034000000000003e-05 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445189+0000", 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086775100000000005 2024-09-19T05:44:26.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445402+0000", 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021237099999999999 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.445599+0000", 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019734099999999999 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.501230+0000", 2024-09-19T05:44:26.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055630863000000003 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886462+0000", 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.38523158200000002 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886525+0000", 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3138000000000004e-05 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.886578+0000", 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3143000000000003e-05 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:61 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.088225456000004, 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.424715008, 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 61 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457405+0000", 2024-09-19T05:44:26.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.457409+0000", 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461813+0000", 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044040060000000002 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461834+0000", 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0789e-05 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461876+0000", 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2268000000000003e-05 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.461987+0000", 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110593 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467170+0000", 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051828849999999999 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467417+0000", 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024757199999999999 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490516+0000", 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023098802000000002 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.881965+0000", 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39144917200000001 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882043+0000", 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7830000000000005e-05 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.882120+0000", 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6360999999999996e-05 2024-09-19T05:44:26.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:267 3.14 3:29a5f800:::10000000002.00000008:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.949078926999999, 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52119119999999997, 2024-09-19T05:44:26.580 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 267 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596551+0000", 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.596555+0000", 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.596e-06 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597947+0000", 2024-09-19T05:44:26.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001392517 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597975+0000", 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7993000000000001e-05 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.597989+0000", 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3927999999999999e-05 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598090+0000", 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010057500000000001 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598219+0000", 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012952999999999999 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.598408+0000", 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018851600000000001 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604394+0000", 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059861159999999997 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117572+0000", 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51317804499999997 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117624+0000", 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2126000000000002e-05 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117742+0000", 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011825800000000001 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:274 3.30 3:0ca36612:::10000000002.0000000f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.933436875000002, 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63191469899999997, 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 274 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612193+0000", 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.612198+0000", 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.476e-06 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619545+0000", 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073473399999999999 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619574+0000", 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8994000000000001e-05 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619585+0000", 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1103e-05 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619846+0000", 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026047199999999998 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.619979+0000", 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013353099999999999 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.620203+0000", 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000223945 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629556+0000", 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093532560000000008 2024-09-19T05:44:26.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244028+0000", 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61447147300000005 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244065+0000", 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6968999999999998e-05 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244108+0000", 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3139999999999997e-05 2024-09-19T05:44:26.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:293 3.30 3:0d7ddb42:::10000000002.00000022:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.891938189000001, 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59051000200000003, 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 293 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653692+0000", 2024-09-19T05:44:26.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.653696+0000", 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.33e-06 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658892+0000", 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051954360000000003 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658913+0000", 2024-09-19T05:44:26.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1705999999999999e-05 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.658928+0000", 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4361e-05 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659326+0000", 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000398253 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659446+0000", 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012050200000000001 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659655+0000", 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020868500000000001 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.734308+0000", 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074653061000000007 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244147+0000", 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.50983838800000003 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244175+0000", 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8901999999999999e-05 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244202+0000", 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6378000000000001e-05 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.597 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:309 3.9 3:9387ba5e:::10000000002.00000032:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.876204258000001, 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.505676974, 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.598 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 309 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669426+0000", 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.669428+0000", 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.593e-06 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.676971+0000", 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0075425839999999997 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677009+0000", 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8154000000000001e-05 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677039+0000", 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9788999999999999e-05 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677223+0000", 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000184372 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677347+0000", 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123979 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677516+0000", 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016852499999999999 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732124+0000", 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054607912000000002 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.174998+0000", 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44287454799999998 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175054+0000", 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5476999999999998e-05 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175103+0000", 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9041e-05 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:313 3.2f 3:f7c72662:::10000000002.00000036:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.603 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.868503560000001, 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47726204500000002, 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 313 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677126+0000", 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.677129+0000", 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3479999999999998e-06 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685213+0000", 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080837730000000007 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685237+0000", 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4311000000000002e-05 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685254+0000", 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7428999999999999e-05 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863622+0000", 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.178367422 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863724+0000", 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101938 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.863861+0000", 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013715 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.152292+0000", 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.288431614 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154270+0000", 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019771250000000001 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154330+0000", 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0217999999999998e-05 2024-09-19T05:44:26.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.154388+0000", 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8717e-05 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:317 3.11 3:8969f477:::10000000002.0000003a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.860281191999999, 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56821626300000005, 2024-09-19T05:44:26.609 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 317 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685349+0000", 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.685351+0000", 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3549999999999999e-06 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692638+0000", 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0072869909999999996 2024-09-19T05:44:26.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692666+0000", 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8010000000000001e-05 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692677+0000", 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0776e-05 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692800+0000", 2024-09-19T05:44:26.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012327200000000001 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692936+0000", 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013544100000000001 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.693426+0000", 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00049003600000000001 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.732131+0000", 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038705212000000003 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253472+0000", 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52134096299999999 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253518+0000", 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5731e-05 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253565+0000", 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7475999999999997e-05 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:331 3.30 3:0d2dbd17:::10000000002.00000048:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.796616896, 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.495291814, 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.615 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 331 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749013+0000", 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.749016+0000", 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.754846+0000", 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058302459999999999 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756282+0000", 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001436295 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756297+0000", 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4921e-05 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756370+0000", 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2856999999999997e-05 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756495+0000", 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012511999999999999 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756711+0000", 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216233 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828851+0000", 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072139743000000006 2024-09-19T05:44:26.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244223+0000", 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41537228599999998 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244270+0000", 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7002000000000002e-05 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244305+0000", 2024-09-19T05:44:26.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4527999999999999e-05 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:346 3.11 3:8815798b:::10000000002.00000057:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.789302747000001, 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49739494099999998, 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 346 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756327+0000", 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.756330+0000", 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7839999999999999e-06 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759712+0000", 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033821889999999999 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759737+0000", 2024-09-19T05:44:26.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5069e-05 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759752+0000", 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4372999999999999e-05 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.759847+0000", 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5291999999999995e-05 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760098+0000", 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025112099999999998 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.760422+0000", 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032433199999999998 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855328+0000", 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094905274999999997 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253622+0000", 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39829475600000003 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253679+0000", 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6047000000000001e-05 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253722+0000", 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3702999999999998e-05 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:361 3.30 3:0eedefab:::10000000002.00000066:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.773167227999998, 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47196730199999998, 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 361 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772463+0000", 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772466+0000", 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8279999999999999e-06 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777394+0000", 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049281790000000004 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777413+0000", 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8913999999999999e-05 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777424+0000", 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1192999999999999e-05 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799507+0000", 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022083230999999998 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799656+0000", 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014921300000000001 2024-09-19T05:44:26.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799858+0000", 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020186999999999999 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830773+0000", 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030915148999999999 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244343+0000", 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41356947900000002 2024-09-19T05:44:26.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244392+0000", 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9489e-05 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.244430+0000", 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7756999999999998e-05 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:366 3.11 3:8b4c71c3:::10000000002.0000006b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.768177406, 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47662371999999997, 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 366 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777453+0000", 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777455+0000", 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7640000000000001e-06 2024-09-19T05:44:26.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783373+0000", 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005917533 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783420+0000", 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6709999999999998e-05 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783443+0000", 2024-09-19T05:44:26.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3836999999999999e-05 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808013+0000", 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024569607 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808128+0000", 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011490299999999999 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808375+0000", 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024700299999999998 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937889+0000", 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12951412500000001 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.253987+0000", 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31609763600000002 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254037+0000", 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0383999999999997e-05 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254076+0000", 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9218000000000003e-05 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:368 3.14 3:280f29a3:::10000000002.0000006d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.762132029, 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45471794900000001, 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 368 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783498+0000", 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.783504+0000", 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7749999999999998e-06 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785589+0000", 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020852409999999998 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785606+0000", 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6609999999999999e-05 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.785614+0000", 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4940000000000003e-06 2024-09-19T05:44:26.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789369+0000", 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037552250000000001 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789485+0000", 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116019 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.789654+0000", 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016819299999999999 2024-09-19T05:44:26.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.828885+0000", 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039231901999999999 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238106+0000", 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40922085699999999 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238164+0000", 2024-09-19T05:44:26.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7373000000000001e-05 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.238216+0000", 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2259999999999998e-05 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:382 3.11 3:8818af48:::10000000002.0000007b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.756724108, 2024-09-19T05:44:26.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46533816500000003, 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 382 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788906+0000", 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.788911+0000", 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6609999999999999e-06 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796253+0000", 2024-09-19T05:44:26.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073421060000000002 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796284+0000", 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1383000000000001e-05 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.796299+0000", 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4793e-05 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832493+0000", 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036194021 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832612+0000", 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011929 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832778+0000", 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.066630+0000", 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.233852226 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254144+0000", 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18751385100000001 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254200+0000", 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6028e-05 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254244+0000", 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4373999999999999e-05 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:403 3.9 3:91a8729a:::10000000002.00000090:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.729059061000001, 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41822062799999998, 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 403 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816571+0000", 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.816573+0000", 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.345e-06 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.820472+0000", 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038989060000000002 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827487+0000", 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070152189999999996 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827507+0000", 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9434000000000001e-05 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858059+0000", 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030552598 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.858252+0000", 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019281999999999999 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871029+0000", 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012776648999999999 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.175256+0000", 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30422741199999997 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234683+0000", 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059426432000000001 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234738+0000", 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4923999999999998e-05 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234792+0000", 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3888999999999999e-05 2024-09-19T05:44:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:405 3.11 3:8834f03f:::10000000002.00000092:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.718095742999999, 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42686835699999998, 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 405 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827534+0000", 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827537+0000", 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6620000000000001e-06 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830490+0000", 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029526209999999999 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830506+0000", 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6891000000000001e-05 2024-09-19T05:44:26.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830535+0000", 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8377e-05 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855366+0000", 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024831576000000001 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855488+0000", 2024-09-19T05:44:26.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012119499999999999 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.855676+0000", 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018832899999999999 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233600+0000", 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37792366900000002 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254308+0000", 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020708684000000002 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254362+0000", 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3430000000000002e-05 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254403+0000", 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0923000000000001e-05 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:410 3.11 3:8988b152:::10000000002.00000097:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.715073572000001, 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42399440900000002, 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 410 2024-09-19T05:44:26.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830556+0000", 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.830559+0000", 2024-09-19T05:44:26.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835124+0000", 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045654520000000002 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835145+0000", 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0750999999999998e-05 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:26.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.835179+0000", 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4013000000000001e-05 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.937947+0000", 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102767884 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938050+0000", 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010270199999999999 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:26.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938225+0000", 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000175494 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.234859+0000", 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.296633759 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254459+0000", 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019599979999999999 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254510+0000", 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1431000000000003e-05 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.254551+0000", 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0391e-05 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:26.667 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:26.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:26.669 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:26.667+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:26.770 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:26.768+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:26.772 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:26.772 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:26.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:26.827+0000 7fef88b4b640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 msgr2=0x7fef84206f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3690741328 2024-09-19T05:44:26.829 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:26.827+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 0x7fef84206f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T05:44:26.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.18 --file /var/log/ceph/exp.3.18.5 2024-09-19T05:44:26.872 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:26.870+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:26.961 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:26.961 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:26.961 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:26.961 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:26.961 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:26.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:26.973 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:26.971+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:27.075 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:27.072+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:27.078 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:27.078 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:13 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.761230198, 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41922241599999999, 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.079 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 13 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313315+0000", 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.978e-06 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329516+0000", 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016201601999999999 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329535+0000", 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9157e-05 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329545+0000", 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9909999999999997e-06 2024-09-19T05:44:27.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329609+0000", 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3931999999999997e-05 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329764+0000", 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015456700000000001 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329895+0000", 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013116700000000001 2024-09-19T05:44:27.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.336041+0000", 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061460580000000003 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732421+0000", 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.396379484 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732484+0000", 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3100000000000002e-05 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732535+0000", 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1379999999999999e-05 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:64 3.2 3:40abce4f:::10000000002.0000003d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:27.085 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.612600712000003, 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29680736000000002, 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 64 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:27.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461942+0000", 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.461945+0000", 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6860000000000002e-06 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478410+0000", 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016465418999999999 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478422+0000", 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1732e-05 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478431+0000", 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8890000000000008e-06 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478503+0000", 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.25e-05 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478662+0000", 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000158696 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.478774+0000", 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011198500000000001 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.481734+0000", 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002960018 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758602+0000", 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27686762799999998 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758681+0000", 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9566000000000004e-05 2024-09-19T05:44:27.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.758750+0000", 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8240999999999999e-05 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:82 3.2b 3:d50470f2:::10000000002.0000004f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.582806259999998, 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29488188799999998, 2024-09-19T05:44:27.091 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 82 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491737+0000", 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.491754+0000", 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7186000000000001e-05 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.506963+0000", 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015209675000000001 2024-09-19T05:44:27.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.506991+0000", 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7872e-05 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507003+0000", 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1719000000000001e-05 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507080+0000", 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7327e-05 2024-09-19T05:44:27.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507223+0000", 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014263500000000001 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.507439+0000", 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000215873 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.511608+0000", 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041688610000000003 2024-09-19T05:44:27.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786537+0000", 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27492934899999999 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786576+0000", 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8575000000000003e-05 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786619+0000", 2024-09-19T05:44:27.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2815999999999998e-05 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:132 3.2b 3:d52309a9:::10000000002.00000081:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.501932160000003, 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.21416674099999999, 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.097 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 132 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572611+0000", 2024-09-19T05:44:27.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.572614+0000", 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9289999999999998e-06 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585573+0000", 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012959205999999999 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585590+0000", 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7326999999999999e-05 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585605+0000", 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5061e-05 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585665+0000", 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9991999999999998e-05 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585800+0000", 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013432599999999999 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.585932+0000", 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013247199999999999 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.588624+0000", 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0026916520000000001 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786688+0000", 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.198064771 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786738+0000", 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9951999999999999e-05 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.786777+0000", 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9053e-05 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.102 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:53 3.9 e53/20 3:9387ba5e:::10000000002.00000032:head v 53'1, mlcod=0'0)", 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.479624536000003, 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28964351700000002, 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.103 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594918+0000", 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.594919+0000", 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0100000000000003e-07 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882027+0000", 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28710772299999998 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882048+0000", 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1081e-05 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882056+0000", 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9049999999999997e-06 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882114+0000", 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7762e-05 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.882132+0000", 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8762000000000001e-05 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.884525+0000", 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002392595 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.884562+0000", 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6788000000000001e-05 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:161 3.2b 3:d691b431:::10000000002.0000009e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.474745224999999, 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.206406749, 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.107 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 161 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599798+0000", 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.599800+0000", 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.655e-06 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609524+0000", 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097240499999999997 2024-09-19T05:44:27.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609544+0000", 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9772e-05 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609553+0000", 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3130000000000004e-06 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609604+0000", 2024-09-19T05:44:27.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0933999999999997e-05 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609704+0000", 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9746000000000003e-05 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609799+0000", 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4823999999999999e-05 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.613360+0000", 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035613950000000002 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806134+0000", 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19277316 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806173+0000", 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9873000000000003e-05 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806204+0000", 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1027e-05 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:168 3.2b 3:d5eb8d7b:::10000000002.000000a5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.464966250000003, 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19673869999999999, 2024-09-19T05:44:27.113 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 168 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609577+0000", 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.609579+0000", 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6819999999999999e-06 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619452+0000", 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0098723679999999994 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619470+0000", 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8515000000000001e-05 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619479+0000", 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8650000000000007e-06 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619556+0000", 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6569000000000005e-05 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619691+0000", 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013584900000000001 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619804+0000", 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112989 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.626514+0000", 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067099350000000002 2024-09-19T05:44:27.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806244+0000", 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.179729837 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806278+0000", 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4221999999999999e-05 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806315+0000", 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6869000000000002e-05 2024-09-19T05:44:27.118 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:172 3.2b 3:d5480d6b:::10000000002.000000a9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.455037159, 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.186920695, 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 172 2024-09-19T05:44:27.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619506+0000", 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.619508+0000", 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5000000000000002e-06 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630572+0000", 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011063454 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630589+0000", 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6860000000000001e-05 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630596+0000", 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6420000000000007e-06 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630659+0000", 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3145999999999995e-05 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630785+0000", 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000125788 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630894+0000", 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001087 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.635728+0000", 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048346090000000001 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806354+0000", 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.170625477 2024-09-19T05:44:27.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806396+0000", 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2252999999999997e-05 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806426+0000", 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0266000000000001e-05 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:179 3.2 3:42ce5726:::10000000002.000000b0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 33.44392208, 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17570660699999999, 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 179 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630621+0000", 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.630624+0000", 2024-09-19T05:44:27.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7870000000000002e-06 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641403+0000", 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01077914 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641420+0000", 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7564e-05 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641442+0000", 2024-09-19T05:44:27.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1355000000000001e-05 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641517+0000", 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5712999999999998e-05 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641680+0000", 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016307800000000001 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.641848+0000", 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016733500000000001 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.647878+0000", 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060299159999999997 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806222+0000", 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.158344174 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806290+0000", 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8201999999999995e-05 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.806327+0000", 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7342999999999998e-05 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:269 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.130 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.481425258000002, 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.44962057300000002, 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 269 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593119+0000", 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5099999999999999e-06 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.132 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614489+0000", 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.021369545 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614511+0000", 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2353000000000001e-05 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614525+0000", 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4311000000000001e-05 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614691+0000", 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016586800000000001 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614811+0000", 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119552 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614956+0000", 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014476499999999999 2024-09-19T05:44:27.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.619618+0000", 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046627020000000003 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042637+0000", 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.42301865999999999 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042689+0000", 2024-09-19T05:44:27.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1876000000000003e-05 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042738+0000", 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9431000000000002e-05 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:320 3.2 3:40abce4f:::10000000002.0000003d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.337001310000002, 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30531098200000001, 2024-09-19T05:44:27.136 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 320 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737544+0000", 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6539999999999999e-06 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760554+0000", 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023009943000000001 2024-09-19T05:44:27.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760577+0000", 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2538e-05 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760585+0000", 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9219999999999997e-06 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760691+0000", 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106606 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760791+0000", 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100239 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760966+0000", 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017421299999999999 2024-09-19T05:44:27.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.766414+0000", 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005448067 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042779+0000", 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27636495599999999 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042822+0000", 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3359000000000003e-05 2024-09-19T05:44:27.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042853+0000", 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0485e-05 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:338 3.2b 3:d50470f2:::10000000002.0000004f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.297023009, 2024-09-19T05:44:27.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30392799300000001, 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 338 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777520+0000", 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.777522+0000", 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3539999999999998e-06 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.793967+0000", 2024-09-19T05:44:27.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016445120000000001 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794010+0000", 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3143999999999999e-05 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794026+0000", 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5404999999999999e-05 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794124+0000", 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7806999999999994e-05 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794224+0000", 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100318 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.794397+0000", 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017272500000000001 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.800396+0000", 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059990449999999997 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081329+0000", 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.28093356000000003 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081398+0000", 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8786000000000002e-05 2024-09-19T05:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081448+0000", 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9728999999999998e-05 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:388 3.2b 3:d52309a9:::10000000002.00000081:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.183311980999999, 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19034685300000001, 2024-09-19T05:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 388 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.151 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891231+0000", 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.891233+0000", 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4059999999999999e-06 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905206+0000", 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013972507 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905225+0000", 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9167000000000001e-05 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905259+0000", 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3633999999999997e-05 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905360+0000", 2024-09-19T05:44:27.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101512 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905437+0000", 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6964000000000004e-05 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.905546+0000", 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108933 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.909399+0000", 2024-09-19T05:44:27.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.003853249 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081492+0000", 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.172092667 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081535+0000", 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3287000000000001e-05 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.081578+0000", 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2527000000000001e-05 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:417 3.2b 3:d691b431:::10000000002.0000009e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.157025652000002, 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19858436900000001, 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 417 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917517+0000", 2024-09-19T05:44:27.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.917519+0000", 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2330000000000001e-06 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936253+0000", 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018733107999999998 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936269+0000", 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6441999999999999e-05 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936278+0000", 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8370000000000003e-06 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936338+0000", 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0318000000000001e-05 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936414+0000", 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6249999999999997e-05 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936508+0000", 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3194000000000003e-05 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.941616+0000", 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0051086789999999997 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116003+0000", 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17438693 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116057+0000", 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3829999999999998e-05 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116102+0000", 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4548000000000001e-05 2024-09-19T05:44:27.161 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:424 3.2b 3:d5eb8d7b:::10000000002.000000a5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.138239539000001, 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17989760599999999, 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 424 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936303+0000", 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.936305+0000", 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9910000000000001e-06 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950048+0000", 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013742437 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950078+0000", 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9904e-05 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950092+0000", 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4543e-05 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950182+0000", 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0236999999999997e-05 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950294+0000", 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111993 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950451+0000", 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015673400000000001 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.954644+0000", 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041925249999999999 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116134+0000", 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16149000999999999 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116179+0000", 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4919000000000001e-05 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.116201+0000", 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2313000000000001e-05 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.167 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:428 3.2b 3:d5480d6b:::10000000002.000000a9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.123881127000001, 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17460092799999999, 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 428 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.168 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950662+0000", 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.950665+0000", 2024-09-19T05:44:27.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.931e-06 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967645+0000", 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016980439 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967663+0000", 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7657999999999999e-05 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967670+0000", 2024-09-19T05:44:27.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3529999999999997e-06 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967735+0000", 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4379000000000001e-05 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.967844+0000", 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109863 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.968022+0000", 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177987 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.973507+0000", 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054845980000000003 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125044+0000", 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15153746700000001 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125149+0000", 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104286 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.125263+0000", 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113967 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.173 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:439 3.1b 3:db82751e:::10000000002.000000b4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.094777625999999, 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19035659999999999, 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 439 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979765+0000", 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.979767+0000", 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6819999999999999e-06 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993421+0000", 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013654314000000001 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993442+0000", 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0911e-05 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993451+0000", 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.918e-06 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993525+0000", 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4259999999999997e-05 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993635+0000", 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109851 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993795+0000", 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015986599999999999 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.000943+0000", 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071475469999999998 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170055+0000", 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.16911190800000001 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170090+0000", 2024-09-19T05:44:27.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5222999999999998e-05 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.170122+0000", 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2119999999999997e-05 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:443 3.1b 3:da9dcad1:::10000000002.000000b8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.081066051000001, 2024-09-19T05:44:27.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.186683074, 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 443 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.180 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993477+0000", 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.993478+0000", 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.398e-06 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.181 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004522+0000", 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011043536 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004541+0000", 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9301999999999999e-05 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004551+0000", 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7589999999999992e-06 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.182 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004657+0000", 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106069 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004805+0000", 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148197 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.004912+0000", 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010655800000000001 2024-09-19T05:44:27.183 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.007998+0000", 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0030858679999999999 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180063+0000", 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.17206524200000001 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.184 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180114+0000", 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1122999999999998e-05 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.180160+0000", 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6022000000000001e-05 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:490 3.2 3:437e6408:::10000000002.000000e7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:27.185 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.029984537000001, 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.15379004299999999, 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 490 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.186 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044558+0000", 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.044560+0000", 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.685e-06 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.187 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054251+0000", 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096909700000000001 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054267+0000", 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5755000000000001e-05 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054277+0000", 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0504e-05 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.188 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054333+0000", 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6001000000000001e-05 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054396+0000", 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2852000000000001e-05 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054483+0000", 2024-09-19T05:44:27.189 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7332999999999999e-05 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.057123+0000", 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.002639705 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198289+0000", 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.14116562599999999 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.190 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198321+0000", 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2543000000000003e-05 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.198348+0000", 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7069000000000001e-05 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:502 3.2 3:4041dd16:::10000000002.000000f3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e55)", 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:27.191 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 28.02024492, 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.14802890099999999, 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 502 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:27.192 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054298+0000", 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.054300+0000", 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.012e-06 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.193 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063321+0000", 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090211760000000005 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063333+0000", 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2308e-05 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063340+0000", 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7819999999999996e-06 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.194 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063427+0000", 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6948999999999997e-05 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063536+0000", 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109066 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.063706+0000", 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000169906 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.195 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.068352+0000", 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046461590000000004 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202270+0000", 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.133917486 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202302+0000", 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2237999999999999e-05 2024-09-19T05:44:27.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.202327+0000", 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4819000000000001e-05 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:27.197 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:27.199 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:27.199 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:27.197+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:27.300 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:27.298+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:27.381 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:27.391 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:27.433 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:27.431+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:27.504 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:27.502+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:27.572 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:27.572 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:27.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:27.627 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:27.626+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:27.703 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:27.701+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:27.742 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:27.742 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:27.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:27.805 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:27.802+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:27.808 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:44:27.904 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:27.902+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:27.956 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:27.956 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:27.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:28.015 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:28.014+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.067+0000 7f12e4815640 1 Processor -- start 2024-09-19T05:44:28.070 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.067+0000 7f12e4815640 1 -- start start 2024-09-19T05:44:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.070+0000 7f12e4815640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d80453d0 0x7f12d80db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.070+0000 7f12e4815640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d80068c0 con 0x7f12d8005de0 2024-09-19T05:44:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.070+0000 7f12e4815640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d8006ba0 con 0x7f12d804c3f0 2024-09-19T05:44:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.070+0000 7f12e4815640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f12d8006e80 con 0x7f12d80453d0 2024-09-19T05:44:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.071+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d80453d0 0x7f12d80db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.071+0000 7f12dffff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f12d8005de0 0x7f12d804b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:38572/0 (socket says 172.21.15.2:38572) 2024-09-19T05:44:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.071+0000 7f12dffff640 1 -- 172.21.15.2:0/705521707 learned_addr learned my addr 172.21.15.2:0/705521707 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:28.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.072+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297983088 0 0) 0x7f12d80068c0 con 0x7f12d8005de0 2024-09-19T05:44:28.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.072+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12d0003520 con 0x7f12d8005de0 2024-09-19T05:44:28.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.072+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1522680160 0 0) 0x7f12d0003520 con 0x7f12d8005de0 2024-09-19T05:44:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.072+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d80453d0 msgr2=0x7f12d80db500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.072+0000 7f12de7fc640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d80453d0 0x7f12d80db500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.072+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 >> v1:172.21.15.123:6789/0 conn(0x7f12d804c3f0 legacy=0x7f12d8007960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.075+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d8007160 con 0x7f12d8005de0 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.075+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f12e0063850 con 0x7f12d8005de0 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.075+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 >> v1:172.21.15.2:6789/0 conn(0x7f12d8005de0 legacy=0x7f12d804b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.076+0000 7f12de7fc640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d80453d0 0x7f12d00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.076+0000 7f12de7fc640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0003f40 0x7f12d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.076+0000 7f12de7fc640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d0004d40 0x7f12d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.076+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8006e80 con 0x7f12d0003f40 2024-09-19T05:44:28.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.076+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12d000bad0 con 0x7f12d0004d40 2024-09-19T05:44:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.076+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f12d000bcd0 con 0x7f12d80453d0 2024-09-19T05:44:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.077+0000 7f12dffff640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d0004d40 0x7f12d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:28.079 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.077+0000 7f12deffd640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0003f40 0x7f12d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.078+0000 7f12dffff640 1 -- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d80453d0 msgr2=0x7f12d00039e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:44:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.078+0000 7f12dffff640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d80453d0 0x7f12d00039e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.078+0000 7f12dffff640 1 -- 172.21.15.2:0/705521707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0003f40 msgr2=0x7f12d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.078+0000 7f12dffff640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0003f40 0x7f12d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.078+0000 7f12dffff640 1 -- 172.21.15.2:0/705521707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d0003520 con 0x7f12d0004d40 2024-09-19T05:44:28.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.078+0000 7f12deffd640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0003f40 0x7f12d0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:28.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.079+0000 7f12dffff640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d0004d40 0x7f12d00051c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12e0051790 tx=0x7f12d0005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f197baa47a61eb00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:28.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.080+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00b3070 con 0x7f12d0004d40 2024-09-19T05:44:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.080+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e00ae400 con 0x7f12d0004d40 2024-09-19T05:44:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.081+0000 7f12de7fc640 1 -- 172.21.15.2:0/705521707 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e004ffb0 con 0x7f12d0004d40 2024-09-19T05:44:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.082+0000 7f12e4815640 1 -- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d0004d40 msgr2=0x7f12d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.082+0000 7f12e4815640 1 --2- 172.21.15.2:0/705521707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d0004d40 0x7f12d00051c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12e0051790 tx=0x7f12d0005830 comp rx=0 tx=0).stop 2024-09-19T05:44:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.082+0000 7f12dffff640 1 -- 172.21.15.2:0/705521707 reap_dead start 2024-09-19T05:44:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.082+0000 7f12e4815640 1 -- 172.21.15.2:0/705521707 shutdown_connections 2024-09-19T05:44:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.082+0000 7f12e4815640 1 -- 172.21.15.2:0/705521707 >> 172.21.15.2:0/705521707 conn(0x7f12d8050530 msgr2=0x7f12d8050940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.082+0000 7f12e4815640 1 -- 172.21.15.2:0/705521707 shutdown_connections 2024-09-19T05:44:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.082+0000 7f12e4815640 1 -- 172.21.15.2:0/705521707 wait complete. 2024-09-19T05:44:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.085+0000 7f12e4815640 1 Processor -- start 2024-09-19T05:44:28.090 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.086+0000 7f12e4815640 1 -- start start 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12e4815640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0004d40 0x7f12d80dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12e4815640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d804f7c0 0x7f12d80d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12e4815640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d804fc00 0x7f12d80d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12e4815640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f12d8047260 con 0x7f12d0004d40 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12e4815640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12d804cbe0 con 0x7f12d804fc00 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12e4815640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f12d80473e0 con 0x7f12d804f7c0 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0004d40 0x7f12d80dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0004d40 0x7f12d80dd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58622/0 (socket says 172.21.15.2:58622) 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12df7fe640 1 -- 172.21.15.2:0/2051487304 learned_addr learned my addr 172.21.15.2:0/2051487304 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.088+0000 7f12deffd640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d804f7c0 0x7f12d80d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.089+0000 7f12dffff640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d804fc00 0x7f12d80d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.089+0000 7f12df7fe640 1 -- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d804f7c0 msgr2=0x7f12d80d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.089+0000 7f12df7fe640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d804f7c0 0x7f12d80d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.089+0000 7f12df7fe640 1 -- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d804fc00 msgr2=0x7f12d80d6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.089+0000 7f12df7fe640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d804fc00 0x7f12d80d6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.089+0000 7f12df7fe640 1 -- 172.21.15.2:0/2051487304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d80478d0 con 0x7f12d0004d40 2024-09-19T05:44:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.090+0000 7f12df7fe640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0004d40 0x7f12d80dd6a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00e820 tx=0x7f12cc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.091+0000 7f12cbfff640 1 -- 172.21.15.2:0/2051487304 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc017070 con 0x7f12d0004d40 2024-09-19T05:44:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.091+0000 7f12cbfff640 1 -- 172.21.15.2:0/2051487304 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12cc00c860 con 0x7f12d0004d40 2024-09-19T05:44:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.091+0000 7f12cbfff640 1 -- 172.21.15.2:0/2051487304 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00cc30 con 0x7f12d0004d40 2024-09-19T05:44:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.092+0000 7f12e4815640 1 -- 172.21.15.2:0/2051487304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d8047b70 con 0x7f12d0004d40 2024-09-19T05:44:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.092+0000 7f12e4815640 1 -- 172.21.15.2:0/2051487304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d80dea30 con 0x7f12d0004d40 2024-09-19T05:44:28.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.096+0000 7f12cbfff640 1 -- 172.21.15.2:0/2051487304 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f12cc01a070 con 0x7f12d0004d40 2024-09-19T05:44:28.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.098+0000 7f12cbfff640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f12c406c300 0x7f12c406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:28.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.099+0000 7f12deffd640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f12c406c300 0x7f12c406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:28.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.099+0000 7f12e4815640 1 -- 172.21.15.2:0/2051487304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c0005290 con 0x7f12d0004d40 2024-09-19T05:44:28.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.100+0000 7f12cbfff640 1 -- 172.21.15.2:0/2051487304 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5799+0+0 (secure 0 0 0) 0x7f12cc00cdd0 con 0x7f12d0004d40 2024-09-19T05:44:28.106 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:28.104+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:28.112 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:28.112 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:28.114 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.107+0000 7f12deffd640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f12c406c300 0x7f12c406e7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f12d8036810 tx=0x7f12d4007cc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:28.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.119+0000 7f12cbfff640 1 -- 172.21.15.2:0/2051487304 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f12cc058b50 con 0x7f12d0004d40 2024-09-19T05:44:28.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:28.209 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:28.205+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:28.301 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:28.302 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:28.308 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:28.306+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:28.315 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:28.412 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:28.407+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:28.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.428+0000 7fef88b4b640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 msgr2=0x7fef84206f00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.123:6800/3690741328 2024-09-19T05:44:28.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.428+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 0x7fef84206f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-19T05:44:28.523 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:28.512+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:28.536 INFO:teuthology.orchestra.run.smithi123.stderr:Exporting 3.18 info 3.18( v 62'23 (0'0,62'23] local-lis/les=20/21 n=4 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-19T05:44:28.542 INFO:teuthology.orchestra.run.smithi123.stderr:Read #3:186b5309:::100000001f7.000000fd:head# 2024-09-19T05:44:28.549 INFO:teuthology.orchestra.run.smithi123.stderr:Read #3:191fe66d:::100000001f7.000000a0:head# 2024-09-19T05:44:28.556 INFO:teuthology.orchestra.run.smithi123.stderr:Read #3:19643448:::100000001f7.00000065:head# 2024-09-19T05:44:28.568 INFO:teuthology.orchestra.run.smithi123.stderr:Read #3:19a9fe8d:::100000001f7.000000b4:head# 2024-09-19T05:44:28.578 INFO:teuthology.orchestra.run.smithi123.stderr:Export successful 2024-09-19T05:44:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.606+0000 7f12e4815640 1 -- 172.21.15.2:0/2051487304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f12c0005850 con 0x7f12d0004d40 2024-09-19T05:44:28.621 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:28.610+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:1522 3.2c 3:35b187b2:::100000001f8.000001ef:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e69)", 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.545337+0000", 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.093456730000000002, 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.093488071000000006, 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:28.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1522 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.545337+0000", 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.545337+0000", 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.545339+0000", 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0829999999999998e-06 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.550454+0000", 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051154210000000002 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.550503+0000", 2024-09-19T05:44:28.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8121999999999998e-05 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.550522+0000", 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9428000000000001e-05 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.575680+0000", 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025158508999999999 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:28.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.605411+0000", 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029731003999999998 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.605576+0000", 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016462900000000001 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.629114+0000", 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023537467999999999 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.645 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:1529 3.34 3:2eaad5e5:::100000001f8.000001f6:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e69)", 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.550556+0000", 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.088238074999999999, 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.088357927000000003, 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:28.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1529 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.550556+0000", 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.550556+0000", 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.550558+0000", 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8890000000000002e-06 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.558714+0000", 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0081554060000000005 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.558753+0000", 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9617999999999999e-05 2024-09-19T05:44:28.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.558771+0000", 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7563000000000001e-05 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.575846+0000", 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017075428 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:28.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.588728+0000", 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012881369 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.588888+0000", 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016068699999999999 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:1539 3.3f 3:ff28424c:::100000001f8.00000200:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e69)", 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.570986+0000", 2024-09-19T05:44:28.650 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.067807485000000001, 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068000153999999993, 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1539 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.570986+0000", 2024-09-19T05:44:28.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.570986+0000", 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.570990+0000", 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5379999999999998e-06 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.574765+0000", 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037753940000000001 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.574781+0000", 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6008e-05 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.574793+0000", 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1464000000000001e-05 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.588526+0000", 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013733772 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.588728+0000", 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020169 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.589233+0000", 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050510699999999997 2024-09-19T05:44:28.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.616543+0000", 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027309835000000001 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:1548 3.33 e69/20 3:ceffbe40:::100000001f8.00000209:head v 69'37, mlcod=69'35)", 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.595155+0000", 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.043638928, 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905996000000003, 2024-09-19T05:44:28.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.595155+0000", 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.595155+0000", 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.595156+0000", 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.319e-06 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.600307+0000", 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051507050000000002 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.600327+0000", 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9888e-05 2024-09-19T05:44:28.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.600339+0000", 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2313999999999999e-05 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.601993+0000", 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016543490000000001 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.602021+0000", 2024-09-19T05:44:28.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7651e-05 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:1559 3.24 e69/20 3:245c653b:::100000001f8.00000214:head v 69'39, mlcod=69'37)", 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.612861+0000", 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.025932452000000002, 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026254899000000002, 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.612861+0000", 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.612861+0000", 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.612863+0000", 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.519e-06 2024-09-19T05:44:28.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.614797+0000", 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019341899999999999 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.614817+0000", 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9860999999999998e-05 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.614829+0000", 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.243e-05 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.619810+0000", 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049807870000000004 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.619868+0000", 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-05 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:1566 3.33 e69/20 3:cf99ff6f:::100000001f8.0000021b:head v 69'38, mlcod=69'35)", 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.614858+0000", 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.023935495000000001, 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024312212999999999, 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.614858+0000", 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.614858+0000", 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.614859+0000", 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4499999999999996e-07 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.629842+0000", 2024-09-19T05:44:28.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014983435 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.629867+0000", 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4505999999999999e-05 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.629879+0000", 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.629957+0000", 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8191999999999999e-05 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.629990+0000", 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2663999999999998e-05 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4534.0:1529 3.34 e69/38)", 2024-09-19T05:44:28.666 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.631652+0000", 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0071418239999999997, 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.007569676, 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.631652+0000", 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.631652+0000", 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.631654+0000", 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4990000000000001e-06 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.631668+0000", 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3797e-05 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.631691+0000", 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2901999999999999e-05 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.631707+0000", 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6005e-05 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:1555 3.13 e69/20 3:cafe20c7:::100000001f8.00000210:head v 69'40, mlcod=69'38)", 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.627391+0000", 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011402285, 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011886465000000001, 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:28.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.627391+0000", 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.627391+0000", 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.627393+0000", 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0490000000000002e-06 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:28.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.638545+0000", 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011151866999999999 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.638573+0000", 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7424999999999999e-05 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.638586+0000", 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3032e-05 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.638686+0000", 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100445 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.638712+0000", 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6295000000000002e-05 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 8 2024-09-19T05:44:28.673 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:28.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:28.710 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:28.709+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:28.811 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:28.809+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:28.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.813+0000 7f12cbfff640 1 -- 172.21.15.2:0/2051487304 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v70) ==== 97+0+0 (secure 0 0 0) 0x7f12cc05ce50 con 0x7f12d0004d40 2024-09-19T05:44:28.818 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:44:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 -- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f12c406c300 msgr2=0x7f12c406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f12c406c300 0x7f12c406e7c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f12d8036810 tx=0x7f12d4007cc0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 -- 172.21.15.2:0/2051487304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0004d40 msgr2=0x7f12d80dd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0004d40 0x7f12d80dd6a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00e820 tx=0x7f12cc00ece0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 -- 172.21.15.2:0/2051487304 shutdown_connections 2024-09-19T05:44:28.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f12c406c300 0x7f12c406e7c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d804fc00 0x7f12d80d6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f12d804f7c0 0x7f12d80d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 --2- 172.21.15.2:0/2051487304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f12d0004d40 0x7f12d80dd6a0 secure :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00e820 tx=0x7f12cc00ece0 comp rx=0 tx=0).stop 2024-09-19T05:44:28.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.822+0000 7f12c9ffb640 1 -- 172.21.15.2:0/2051487304 >> 172.21.15.2:0/2051487304 conn(0x7f12d8050530 msgr2=0x7f12d80cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:28.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.823+0000 7f12c9ffb640 1 -- 172.21.15.2:0/2051487304 shutdown_connections 2024-09-19T05:44:28.825 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:28.823+0000 7f12c9ffb640 1 -- 172.21.15.2:0/2051487304 wait complete. 2024-09-19T05:44:28.912 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:28.910+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:29.004 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:29.005 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:29.012 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:29.010+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:29.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:29.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.18 2024-09-19T05:44:29.136 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:29.134+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:29.232 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:29.225+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:29.315 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:29.313+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:12 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.027757743999999, 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.301330818, 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 12 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285074+0000", 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.285078+0000", 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8979999999999998e-06 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288881+0000", 2024-09-19T05:44:29.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038028900000000002 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288908+0000", 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6831000000000001e-05 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.288922+0000", 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4647999999999999e-05 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289034+0000", 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011129 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289158+0000", 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012398199999999999 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.289283+0000", 2024-09-19T05:44:29.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012596100000000001 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.302442+0000", 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01315887 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586307+0000", 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28386498300000002 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586357+0000", 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9406000000000001e-05 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586405+0000", 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8059000000000002e-05 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:16 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:29.324 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.013634003999996, 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28733763699999998, 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 16 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299198+0000", 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299338+0000", 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013993499999999999 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305742+0000", 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064047180000000002 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305795+0000", 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2988000000000002e-05 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305808+0000", 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.307e-05 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.305883+0000", 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4265999999999997e-05 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306078+0000", 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195281 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.306298+0000", 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000219662 2024-09-19T05:44:29.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.319662+0000", 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0133641 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586451+0000", 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26678922700000002 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586494+0000", 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3102999999999997e-05 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.586535+0000", 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1287e-05 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:36 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:29.330 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.996295617000001, 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54259264799999996, 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 36 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:29.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316536+0000", 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.316539+0000", 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1310000000000002e-06 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326313+0000", 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0097740320000000002 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326341+0000", 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8178e-05 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326354+0000", 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2767e-05 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326454+0000", 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9741999999999995e-05 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326598+0000", 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000144454 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326740+0000", 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014189899999999999 2024-09-19T05:44:29.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.381404+0000", 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054663549999999998 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859031+0000", 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47762663900000002 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859074+0000", 2024-09-19T05:44:29.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3578999999999997e-05 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859129+0000", 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4676999999999998e-05 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:40 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.986406270000003, 2024-09-19T05:44:29.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29103886899999998, 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 40 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326425+0000", 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.326428+0000", 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9160000000000001e-06 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343603+0000", 2024-09-19T05:44:29.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017174545999999999 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343619+0000", 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6260999999999999e-05 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343630+0000", 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0774999999999999e-05 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343729+0000", 2024-09-19T05:44:29.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8694000000000001e-05 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343846+0000", 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117588 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.343969+0000", 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012271600000000001 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.376370+0000", 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032401093999999998 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617348+0000", 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24097808100000001 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617405+0000", 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6577999999999999e-05 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.617464+0000", 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9620000000000002e-05 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:101 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.845720567999997, 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.392178684, 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 101 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467111+0000", 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.467114+0000", 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.269e-06 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.477820+0000", 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010705696000000001 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.500766+0000", 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022946191000000001 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508150+0000", 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073836930000000002 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508253+0000", 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103222 2024-09-19T05:44:29.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508541+0000", 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028815600000000002 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.508725+0000", 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183766 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.543910+0000", 2024-09-19T05:44:29.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035184897 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859196+0000", 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31528614500000002 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859243+0000", 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6727000000000002e-05 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.859290+0000", 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6922000000000002e-05 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:268 3.2c 3:354fa1eb:::10000000002.00000009:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.712979434000001, 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37283424700000001, 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 268 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599852+0000", 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.599857+0000", 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.228e-06 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604581+0000", 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047240240000000003 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604601+0000", 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0738e-05 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.604613+0000", 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621794+0000", 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017181279000000001 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.621899+0000", 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104218 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.622059+0000", 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015997200000000001 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.629055+0000", 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069966560000000004 2024-09-19T05:44:29.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972534+0000", 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34347880400000003 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972610+0000", 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6398999999999998e-05 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972687+0000", 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6038999999999995e-05 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:272 3.34 3:2fc55728:::10000000002.0000000d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.696427482000001, 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40633524799999998, 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.354 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 272 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616404+0000", 2024-09-19T05:44:29.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.616407+0000", 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.993e-06 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631509+0000", 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015101787 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631525+0000", 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5829999999999999e-05 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631535+0000", 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0224000000000001e-05 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631631+0000", 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5696e-05 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631752+0000", 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000121605 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.632659+0000", 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090671499999999999 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.708071+0000", 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075412347000000005 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022627+0000", 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31455541599999998 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022684+0000", 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7543000000000001e-05 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022739+0000", 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5092e-05 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.359 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:292 3.3b 3:ddac2f4e:::10000000002.00000021:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.665164732000001, 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53039040900000001, 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 292 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647667+0000", 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.647670+0000", 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5830000000000001e-06 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.657999+0000", 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010329481 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667975+0000", 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0099757179999999997 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.667984+0000", 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8470000000000002e-06 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668049+0000", 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4999000000000006e-05 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668145+0000", 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6514000000000005e-05 2024-09-19T05:44:29.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.668295+0000", 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015011399999999999 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.729044+0000", 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060748914000000001 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.177943+0000", 2024-09-19T05:44:29.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44889912500000001 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178002+0000", 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8829000000000002e-05 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178057+0000", 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5284999999999997e-05 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:296 3.f 3:f045ad74:::10000000002.00000025:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.640984621000001, 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43521636699999999, 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 296 2024-09-19T05:44:29.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671847+0000", 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.671851+0000", 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.659e-06 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688220+0000", 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016369595000000001 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688250+0000", 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9499e-05 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688262+0000", 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2031999999999999e-05 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688539+0000", 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027760399999999998 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688662+0000", 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000122031 2024-09-19T05:44:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.688813+0000", 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015106399999999999 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.765112+0000", 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076299661000000005 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106912+0000", 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34180015600000002 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.106973+0000", 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0927999999999997e-05 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.107063+0000", 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0137999999999997e-05 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:288 3.31 e54/20 3:8d4d39bf:::10000000002.0000001d:head v 54'5, mlcod=53'3)", 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.614913822999998, 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48371873900000001, 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697918+0000", 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697920+0000", 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7719999999999999e-06 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099856+0000", 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40193617399999998 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099878+0000", 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2306999999999998e-05 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.099891+0000", 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2524000000000001e-05 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132365+0000", 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032473872000000001 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.132403+0000", 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8396999999999998e-05 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181593+0000", 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049190325 2024-09-19T05:44:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.181637+0000", 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3368000000000002e-05 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:337 3.34 3:2c8ffcca:::10000000002.0000004e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.534837123999999, 2024-09-19T05:44:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24486184699999999, 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 337 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777995+0000", 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.777997+0000", 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.17e-06 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781851+0000", 2024-09-19T05:44:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038539429999999999 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781870+0000", 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9012e-05 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781882+0000", 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1864e-05 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.781987+0000", 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105691 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782084+0000", 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7072999999999997e-05 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.782457+0000", 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037225700000000002 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.799269+0000", 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016812247999999998 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022778+0000", 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22350931499999999 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022823+0000", 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4502999999999997e-05 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.022856+0000", 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3770999999999999e-05 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:357 3.3b 3:de1f5bf2:::10000000002.00000062:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.472752128, 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33810015799999998, 2024-09-19T05:44:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 357 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840080+0000", 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.840082+0000", 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2029999999999999e-06 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842172+0000", 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020904890000000001 2024-09-19T05:44:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842193+0000", 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1058999999999999e-05 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.842207+0000", 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3444e-05 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848376+0000", 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061690909999999998 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848486+0000", 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000109979 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.848644+0000", 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015821699999999999 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.894080+0000", 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045435709999999997 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178104+0000", 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28402411700000002 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178148+0000", 2024-09-19T05:44:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4492e-05 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.178180+0000", 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1356999999999998e-05 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:363 3.3f 3:ff5df6dc:::10000000002.00000068:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.462450775000001, 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32356367800000002, 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 363 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850381+0000", 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850383+0000", 2024-09-19T05:44:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0600000000000002e-06 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862283+0000", 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011899873 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862330+0000", 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7546000000000002e-05 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.862352+0000", 2024-09-19T05:44:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1141e-05 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.877929+0000", 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015577153 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878045+0000", 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011612000000000001 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.878204+0000", 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158832 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.896926+0000", 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018722714000000001 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173814+0000", 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27688780099999999 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173871+0000", 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6907000000000003e-05 2024-09-19T05:44:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.173945+0000", 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3530999999999998e-05 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:365 3.c 3:3214729d:::10000000002.0000006a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.424430375, 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27874868600000002, 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 365 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888401+0000", 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.888405+0000", 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7620000000000001e-06 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892569+0000", 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041637540000000004 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892661+0000", 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2134999999999998e-05 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.892675+0000", 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4173e-05 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908596+0000", 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015920815000000001 2024-09-19T05:44:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908712+0000", 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011617599999999999 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.908878+0000", 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165935 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972488+0000", 2024-09-19T05:44:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063609938000000005 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167039+0000", 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19455101699999999 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167097+0000", 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7776999999999999e-05 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167150+0000", 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3204000000000001e-05 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:372 3.34 3:2ebe227a:::10000000002.00000071:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.410727471000001, 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.264918877, 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 372 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902104+0000", 2024-09-19T05:44:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.902109+0000", 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6990000000000002e-06 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910547+0000", 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084382120000000005 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910566+0000", 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9111e-05 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910579+0000", 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2330999999999999e-05 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923326+0000", 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012747596 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923501+0000", 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000174802 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.923775+0000", 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027407199999999999 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.023005+0000", 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099229554999999997 2024-09-19T05:44:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.157725+0000", 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13472034199999999 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.158104+0000", 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037887499999999998 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.167023+0000", 2024-09-19T05:44:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089192820000000006 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:315 3.1c e54/38 3:394f8a26:::10000000002.00000038:head v 54'8, mlcod=53'5)", 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.407307066000001, 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28226446599999999, 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905525+0000", 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.905527+0000", 2024-09-19T05:44:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5739999999999998e-06 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939034+0000", 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033507185000000002 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939067+0000", 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2186e-05 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.939084+0000", 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6982000000000001e-05 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972466+0000", 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033381985000000003 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.972491+0000", 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5789e-05 2024-09-19T05:44:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187219+0000", 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21472724700000001 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187789+0000", 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000570518 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:373 3.36 3:6d44e697:::10000000002.00000072:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.402223909, 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29574129399999999, 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 373 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910608+0000", 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.910611+0000", 2024-09-19T05:44:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5779999999999999e-06 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.919588+0000", 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089768220000000006 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938096+0000", 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018507920000000001 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.938107+0000", 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0488e-05 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944322+0000", 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062149900000000001 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944546+0000", 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022402300000000001 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944884+0000", 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033806299999999999 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.074757+0000", 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.129873715 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206239+0000", 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13148132000000001 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206302+0000", 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3046000000000006e-05 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.206349+0000", 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7329e-05 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:348 3.1c e54/38 3:3aa14ed1:::10000000002.00000059:head v 54'9, mlcod=53'5)", 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.368160648, 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24318706500000001, 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944671+0000", 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.944672+0000", 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1790000000000001e-06 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947488+0000", 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028159130000000002 2024-09-19T05:44:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947504+0000", 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5367e-05 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.947511+0000", 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3300000000000001e-06 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006421+0000", 2024-09-19T05:44:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058910000999999997 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.006451+0000", 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0576e-05 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187796+0000", 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18134434799999999 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187858+0000", 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2351e-05 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:393 3.36 3:6ccb00e9:::10000000002.00000086:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.325838902000001, 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24664645599999999, 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 393 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986993+0000", 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.986998+0000", 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9899999999999997e-06 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995844+0000", 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0088464340000000002 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995859+0000", 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4329e-05 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.995866+0000", 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8539999999999997e-06 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037368+0000", 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041501563999999998 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037440+0000", 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1773999999999994e-05 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.037555+0000", 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011548399999999999 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.117549+0000", 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079993623 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233527+0000", 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11597800699999999 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233591+0000", 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3789999999999997e-05 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.233639+0000", 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8606999999999997e-05 2024-09-19T05:44:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:1354 3.35 3:ac1713fb:::100000001f8.00000147:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e69)", 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:28.055552+0000", 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.2572801710000001, 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24748878499999999, 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1354 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.055552+0000", 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.055552+0000", 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.055555+0000", 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8890000000000004e-06 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.062654+0000", 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070989679999999998 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.063432+0000", 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00077760700000000004 2024-09-19T05:44:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.063685+0000", 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025310600000000001 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.064022+0000", 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033715500000000001 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.064176+0000", 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015322400000000001 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.064336+0000", 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000160545 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.120645+0000", 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056308472999999998 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.302926+0000", 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18228124500000001 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.302983+0000", 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6779e-05 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:28.303040+0000", 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7794000000000003e-05 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.435 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:29.436 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:29.437 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:29.434+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:29.537 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:29.536+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:29.638 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:29.636+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:29.740 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:29.738+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:29.742 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:29.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:1934 3.21 e70/38 3:85befcb5:::100000001f8.0000038b:head v 70'53, mlcod=70'51)", 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.716999+0000", 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022614057, 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022685371999999999, 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.716999+0000", 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.716999+0000", 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.717002+0000", 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3809999999999999e-06 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.727157+0000", 2024-09-19T05:44:29.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010155034 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.727190+0000", 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2403999999999997e-05 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.727204+0000", 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4426e-05 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.727342+0000", 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013724000000000001 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.727373+0000", 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1034999999999997e-05 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:1947 3.2c e70/38 3:3557c0c2:::100000001f8.00000398:head v 70'41, mlcod=70'39)", 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.730830+0000", 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0087833849999999995, 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0089419860000000007, 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.730830+0000", 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.730830+0000", 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.730832+0000", 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6699999999999998e-06 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.733828+0000", 2024-09-19T05:44:29.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029955110000000002 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.733849+0000", 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1605000000000001e-05 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.733863+0000", 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3064999999999999e-05 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:29.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.733941+0000", 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8753999999999998e-05 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.733968+0000", 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6347999999999999e-05 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:29.750 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:29.751 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 2 2024-09-19T05:44:29.751 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:29.767 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:29.840 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:29.838+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:29.948 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:29.942+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:30.041 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:30.039+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:30.106 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:30.107 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:30.107 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:30.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:30.159 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:30.145+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:30.244 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:30.241+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:30.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.335+0000 7fef7a7fc640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=68}) -- 0x7fef600034d0 con 0x7fef8415aa90 2024-09-19T05:44:30.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.335+0000 7fef7a7fc640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] -- ping magic: 0 -- 0x7fef600038a0 con 0x7fef8402d5a0 2024-09-19T05:44:30.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.337+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 7 ==== osd_map(68..70 src has 1..70) ==== 1515+0+0 (secure 0 0 0) 0x7fef7c088f30 con 0x7fef8415aa90 2024-09-19T05:44:30.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.337+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef84155160 con 0x7fef8415aa90 2024-09-19T05:44:30.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.337+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 msgr2=0x7fef84206f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:44:30.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.337+0000 7fef79ffb640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6800/3690741328,v1:172.21.15.123:6801/3690741328] conn(0x7fef8402d5a0 0x7fef84206f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.338+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fef7c0587e0 con 0x7fef8415aa90 2024-09-19T05:44:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.339+0000 7fef8b5b0640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fef84207950 0x7fef84209d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:30.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.339+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fef8420cb10 con 0x7fef84207950 2024-09-19T05:44:30.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.340+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fef84207950 0x7fef84209d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:30.344 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.341+0000 7fef88b4b640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fef84207950 0x7fef84209d40 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:30.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.343+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef8420cb10 con 0x7fef84207950 2024-09-19T05:44:30.350 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:30.343+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:30.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.378+0000 7fef8b5b0640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fef8420a280 con 0x7fef84207950 2024-09-19T05:44:30.383 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.381+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fef8420a280 con 0x7fef84207950 2024-09-19T05:44:30.383 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:30.384 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:30.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.382+0000 7fef6f7fe640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fef54001650 0x7fef54003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:30.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.382+0000 7fef6f7fe640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fef54006bb0 con 0x7fef54001650 2024-09-19T05:44:30.384 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.382+0000 7fef89b4d640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fef54001650 0x7fef54003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:30.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.385+0000 7fef89b4d640 1 --2- 172.21.15.2:0/460346772 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fef54001650 0x7fef54003b10 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:30.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.388+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef54006bb0 con 0x7fef54001650 2024-09-19T05:44:30.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.425+0000 7fef6f7fe640 1 -- 172.21.15.2:0/460346772 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fef540058d0 con 0x7fef54001650 2024-09-19T05:44:30.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:30.427+0000 7fef79ffb640 1 -- 172.21.15.2:0/460346772 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fef540058d0 con 0x7fef54001650 2024-09-19T05:44:30.431 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:30.431 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:30.438 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:30.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:30.458 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:30.458 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:30.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:30.458 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:30.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.458 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 127.575161077, 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:44:30.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:44:30.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:44:30.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:7 3.2a 3:557f550e:::10000000002.00000004:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.171712771999999, 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27494626900000002, 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 7 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282493+0000", 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.282495+0000", 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4609999999999998e-06 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286920+0000", 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044248229999999996 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286972+0000", 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1659000000000001e-05 2024-09-19T05:44:30.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.286986+0000", 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3783e-05 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287117+0000", 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131328 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287358+0000", 2024-09-19T05:44:30.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024141700000000001 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.287543+0000", 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018436700000000001 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.299025+0000", 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011482327000000001 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557305+0000", 2024-09-19T05:44:30.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25828001099999998 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557360+0000", 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5074000000000002e-05 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.557439+0000", 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9018999999999997e-05 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:92 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.038564753999999, 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37051118500000002, 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 92 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415641+0000", 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.415644+0000", 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7070000000000001e-06 2024-09-19T05:44:30.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.422983+0000", 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073390779999999997 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423020+0000", 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7660000000000002e-05 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423036+0000", 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5359999999999999e-05 2024-09-19T05:44:30.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423261+0000", 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000225217 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423458+0000", 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000196555 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.423737+0000", 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027922100000000002 2024-09-19T05:44:30.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.435447+0000", 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011709855999999999 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.785961+0000", 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35051446400000003 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786086+0000", 2024-09-19T05:44:30.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012537599999999999 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.786152+0000", 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691000000000005e-05 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:104 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.024338432, 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28556600799999998, 2024-09-19T05:44:30.476 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429867+0000", 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.429870+0000", 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199999999999999e-06 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446619+0000", 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016749447000000001 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446650+0000", 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0218000000000001e-05 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446666+0000", 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6920999999999999e-05 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.448857+0000", 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021902699999999998 2024-09-19T05:44:30.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449011+0000", 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015441700000000001 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.449168+0000", 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156527 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489828+0000", 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040659908000000002 2024-09-19T05:44:30.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715285+0000", 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22545759500000001 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715345+0000", 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0269999999999997e-05 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715433+0000", 2024-09-19T05:44:30.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7814999999999999e-05 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:108 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.007487415, 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26890963299999998, 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 108 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446718+0000", 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.446721+0000", 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8250000000000001e-06 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451943+0000", 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052222140000000002 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451961+0000", 2024-09-19T05:44:30.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8117e-05 2024-09-19T05:44:30.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.451970+0000", 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7420000000000003e-06 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473034+0000", 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021064282 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473242+0000", 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020716699999999999 2024-09-19T05:44:30.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.473446+0000", 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204396 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489832+0000", 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016385994000000001 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715506+0000", 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22567427700000001 2024-09-19T05:44:30.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715577+0000", 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0548000000000003e-05 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.715628+0000", 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1070999999999999e-05 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:139 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.990676008999998, 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.328123891, 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 139 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463530+0000", 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.463532+0000", 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0339999999999999e-06 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468331+0000", 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047995099999999999 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468365+0000", 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3667000000000001e-05 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468382+0000", 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6651e-05 2024-09-19T05:44:30.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.468915+0000", 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053293299999999997 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469199+0000", 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028406900000000002 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.469482+0000", 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028390000000000002 2024-09-19T05:44:30.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.485026+0000", 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015543509 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791538+0000", 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30651197400000002 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791602+0000", 2024-09-19T05:44:30.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3780000000000003e-05 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.791654+0000", 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1863999999999997e-05 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:148 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.972225975000001, 2024-09-19T05:44:30.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34695000399999998, 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 148 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481980+0000", 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.481982+0000", 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6309999999999999e-06 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489765+0000", 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0077821310000000003 2024-09-19T05:44:30.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489778+0000", 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3604999999999999e-05 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489787+0000", 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489875+0000", 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8039000000000002e-05 2024-09-19T05:44:30.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490101+0000", 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022636299999999999 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.490426+0000", 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032453799999999997 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.506325+0000", 2024-09-19T05:44:30.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015899149000000001 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828797+0000", 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32247239900000002 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828865+0000", 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7521999999999995e-05 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.828930+0000", 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4795999999999994e-05 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:156 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.964370768000002, 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26935146300000001, 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 156 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489835+0000", 2024-09-19T05:44:30.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.489837+0000", 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2689999999999998e-06 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495152+0000", 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053150209999999996 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495356+0000", 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203537 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.495373+0000", 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496071+0000", 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069810200000000003 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496246+0000", 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017547299999999999 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.496469+0000", 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022283200000000001 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.540840+0000", 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044371318999999999 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759082+0000", 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21824185500000001 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759137+0000", 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645000000000003e-05 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.759186+0000", 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9499000000000002e-05 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:185 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 36.951690923999998, 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32688410400000001, 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 185 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.506 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502515+0000", 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.502518+0000", 2024-09-19T05:44:30.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.509973+0000", 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074553919999999999 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510031+0000", 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8356999999999997e-05 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510046+0000", 2024-09-19T05:44:30.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932e-05 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510135+0000", 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8548000000000007e-05 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.510886+0000", 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075159700000000001 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.511079+0000", 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019261500000000001 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.532794+0000", 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021714989000000001 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829312+0000", 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29651747699999997 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829360+0000", 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8396000000000002e-05 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:53.829399+0000", 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8961999999999997e-05 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:264 3.1c 3:3ae0713d:::10000000002.00000005:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.861772213999998, 2024-09-19T05:44:30.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29060550499999999, 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 264 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:30.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592434+0000", 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.592436+0000", 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5629999999999999e-06 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595027+0000", 2024-09-19T05:44:30.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025912600000000002 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595055+0000", 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8164999999999999e-05 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595069+0000", 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3953e-05 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595156+0000", 2024-09-19T05:44:30.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6981e-05 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595260+0000", 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010363 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.595417+0000", 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015732100000000001 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.605722+0000", 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010305099999999999 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882929+0000", 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27720663499999998 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.882988+0000", 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9237000000000002e-05 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.883039+0000", 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0659999999999999e-05 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:315 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.822811894000001, 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55644120600000002, 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 315 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:30.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631396+0000", 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5490000000000001e-06 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636257+0000", 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048607859999999998 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636331+0000", 2024-09-19T05:44:30.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3992000000000006e-05 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636355+0000", 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3813999999999999e-05 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636445+0000", 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0180999999999996e-05 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636569+0000", 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123991 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636849+0000", 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000279664 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659414+0000", 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022565053000000002 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187715+0000", 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52830067000000003 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187783+0000", 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8509000000000004e-05 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187835+0000", 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1996999999999997e-05 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:347 3.2a 3:5747858d:::10000000002.00000058:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.814739068000002, 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48789592399999998, 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 347 2024-09-19T05:44:30.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639467+0000", 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.639470+0000", 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.145e-06 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.644022+0000", 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045525940000000001 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646323+0000", 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023007879999999998 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646342+0000", 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8766999999999998e-05 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646435+0000", 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3394999999999996e-05 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646564+0000", 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012848800000000001 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646738+0000", 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017418499999999999 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.741685+0000", 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094946825999999998 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127231+0000", 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.385546526 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127301+0000", 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9741999999999997e-05 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127363+0000", 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1468000000000002e-05 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.528 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:348 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.807836578, 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54159870300000001, 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 348 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646372+0000", 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9110000000000002e-06 2024-09-19T05:44:30.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.678772+0000", 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032399869999999997 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681572+0000", 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002799714 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681592+0000", 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0298000000000001e-05 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682777+0000", 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011847190000000001 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682909+0000", 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000132146 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.683102+0000", 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019280499999999999 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.702631+0000", 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019529271000000001 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187923+0000", 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48529252 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187948+0000", 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5004999999999999e-05 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187968+0000", 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9443999999999999e-05 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:360 3.2a 3:563615cd:::10000000002.00000065:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:30.534 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.764460162999999, 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43778335400000001, 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 360 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:30.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689746+0000", 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.689748+0000", 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7690000000000001e-06 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692563+0000", 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028147329999999998 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692593+0000", 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9476999999999999e-05 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692616+0000", 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3340999999999999e-05 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723279+0000", 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030662759000000001 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723430+0000", 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151247 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.723629+0000", 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199426 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793213+0000", 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069583435999999999 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127443+0000", 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33423067000000001 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127491+0000", 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8000999999999997e-05 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127529+0000", 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7494999999999999e-05 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:364 3.2a 3:57b579b8:::10000000002.00000069:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.761556446, 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43506039200000002, 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 364 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692649+0000", 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.692652+0000", 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9280000000000002e-06 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.697100+0000", 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004447678 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701619+0000", 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045192829999999998 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.701638+0000", 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8365000000000001e-05 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742561+0000", 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040923560999999997 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742686+0000", 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012482899999999999 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.742839+0000", 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015348 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808887+0000", 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066047260999999996 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127632+0000", 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.31874497099999999 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127672+0000", 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0244999999999998e-05 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.127710+0000", 2024-09-19T05:44:30.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7790999999999998e-05 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:395 3.1c 3:384ad1d9:::10000000002.00000088:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.748814601999999, 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48275294299999999, 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 395 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705391+0000", 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.705394+0000", 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7800000000000001e-06 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720169+0000", 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014775346 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720185+0000", 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5721000000000001e-05 2024-09-19T05:44:30.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720194+0000", 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4609999999999997e-06 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720294+0000", 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100005 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720392+0000", 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7547999999999995e-05 2024-09-19T05:44:30.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720531+0000", 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000139099 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.767968+0000", 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.047436846999999997 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188062+0000", 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42009412899999998 2024-09-19T05:44:30.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188107+0000", 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5269000000000003e-05 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188144+0000", 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6738e-05 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.551 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:404 3.1c 3:3afba7d1:::10000000002.00000091:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.733377436000001, 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46750329699999998, 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 404 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720828+0000", 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.720832+0000", 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8469999999999998e-06 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726318+0000", 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005485775 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726490+0000", 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172339 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.726505+0000", 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4467999999999999e-05 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743219+0000", 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016714046 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743320+0000", 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101548 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.743459+0000", 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013899499999999999 2024-09-19T05:44:30.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.775025+0000", 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031565658000000003 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188247+0000", 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.413221583 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188297+0000", 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0586e-05 2024-09-19T05:44:30.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188332+0000", 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4452000000000002e-05 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:412 3.2a 3:577cc1b8:::10000000002.00000099:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.715758632, 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40023217100000003, 2024-09-19T05:44:30.557 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 412 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738447+0000", 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.738450+0000", 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7e-06 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.758682+0000", 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020232357999999999 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.772625+0000", 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013943232 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.774996+0000", 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023703890000000001 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793240+0000", 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018244646999999999 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793347+0000", 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106276 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.793511+0000", 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016406099999999999 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808895+0000", 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015384296 2024-09-19T05:44:30.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138570+0000", 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32967460500000001 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138630+0000", 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0087999999999998e-05 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.138679+0000", 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9518999999999998e-05 2024-09-19T05:44:30.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:441 3.1c 3:3913eadd:::10000000002.000000b6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.660050084000002, 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.39435722000000001, 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 441 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794156+0000", 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.794159+0000", 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.579e-06 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.802843+0000", 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0086840579999999997 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.807986+0000", 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051424890000000001 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808000+0000", 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808113+0000", 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011294900000000001 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808200+0000", 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6792999999999994e-05 2024-09-19T05:44:30.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.808612+0000", 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00041192799999999999 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.827676+0000", 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019063875000000001 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188432+0000", 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36075675699999998 2024-09-19T05:44:30.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188479+0000", 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6149000000000001e-05 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.188513+0000", 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4310999999999997e-05 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:454 3.37 3:ec162ecd:::10000000002.000000c3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.621286611999999, 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32323945999999998, 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 454 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832919+0000", 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.832922+0000", 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6280000000000001e-06 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.839384+0000", 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064625180000000004 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850436+0000", 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011051365 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850464+0000", 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8719000000000001e-05 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850532+0000", 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7212000000000006e-05 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850663+0000", 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013160799999999999 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.850877+0000", 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000213716 2024-09-19T05:44:30.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.871083+0000", 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020205864 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156086+0000", 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.285003532 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156125+0000", 2024-09-19T05:44:30.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8569999999999998e-05 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.156159+0000", 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3728e-05 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.574 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:30.575 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:30.456+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:30.575 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:30.625 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:30.616+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:30.684 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:30.675+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:30.779 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:30.776+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:30.878 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:44:30.879 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:30.878+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:30.962 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:30.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2010 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.863520+0000", 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.070002954, 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.070038295, 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863520+0000", 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863520+0000", 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863520+0000", 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1300000000000001e-07 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863527+0000", 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8889999999999996e-06 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863533+0000", 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0290000000000002e-06 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863538+0000", 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5539999999999997e-06 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2029 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966166+0000", 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96735688399999997, 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96746710999999996, 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966166+0000", 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966166+0000", 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966166+0000", 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966174+0000", 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2530000000000004e-06 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966181+0000", 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5590000000000002e-06 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966186+0000", 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4539999999999998e-06 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.968 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2044 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.021865+0000", 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.91165729900000003, 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91182887700000004, 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021865+0000", 2024-09-19T05:44:30.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021865+0000", 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021866+0000", 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021869+0000", 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7290000000000002e-06 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021875+0000", 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6230000000000002e-06 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021880+0000", 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9509999999999997e-06 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2059 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.043813+0000", 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.88970928400000004, 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88994615799999999, 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043813+0000", 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043813+0000", 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043814+0000", 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9199999999999999e-07 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043832+0000", 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7727000000000002e-05 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043842+0000", 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0142999999999999e-05 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043851+0000", 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8969999999999997e-06 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2107 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266342+0000", 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66718019299999998, 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66747931699999996, 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266342+0000", 2024-09-19T05:44:30.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266342+0000", 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266343+0000", 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6100000000000002e-07 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266346+0000", 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.207e-06 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266352+0000", 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7490000000000004e-06 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266357+0000", 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0130000000000001e-06 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2162 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466881+0000", 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46664140300000001, 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.46700404299999998, 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466881+0000", 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466881+0000", 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466882+0000", 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9599999999999999e-07 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466890+0000", 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8000000000000004e-06 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466898+0000", 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2629999999999997e-06 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466903+0000", 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000001e-06 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2183 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.562602+0000", 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.37092056000000001, 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37134115600000001, 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562602+0000", 2024-09-19T05:44:30.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562602+0000", 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562602+0000", 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562613+0000", 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0939e-05 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562622+0000", 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0819999999999999e-06 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562628+0000", 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5610000000000004e-06 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:2298 3.11 e70/20 3:8996f682:::100000001f8.000004f7:head v 70'77, mlcod=70'74)", 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.858051+0000", 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.075471260999999998, 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075956024999999996, 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.858051+0000", 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.858051+0000", 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.858052+0000", 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1799999999999999e-06 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862129+0000", 2024-09-19T05:44:30.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040761060000000003 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862144+0000", 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5775999999999999e-05 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862154+0000", 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4250000000000004e-06 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:30.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.874812+0000", 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012658111999999999 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.874843+0000", 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.167e-05 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2030 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.987 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966193+0000", 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.967329353, 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96789134200000004, 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966193+0000", 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966193+0000", 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966194+0000", 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.46e-07 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966198+0000", 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999999e-06 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966204+0000", 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8510000000000004e-06 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966208+0000", 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6380000000000003e-06 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2045 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.990 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.021887+0000", 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.91163580200000005, 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91226102799999997, 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021887+0000", 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021887+0000", 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021887+0000", 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9999999999999998e-07 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021896+0000", 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.789e-06 2024-09-19T05:44:30.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021902+0000", 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.584e-06 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021907+0000", 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2084 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.156254+0000", 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.77726824000000005, 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77796135700000002, 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156254+0000", 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156254+0000", 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.161613+0000", 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053587699999999997 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.161632+0000", 2024-09-19T05:44:30.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9264000000000001e-05 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.161652+0000", 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9817e-05 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.161664+0000", 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2357999999999999e-05 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2108 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266362+0000", 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66716031499999995, 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66792174900000001, 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266362+0000", 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266362+0000", 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266362+0000", 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4900000000000002e-07 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266370+0000", 2024-09-19T05:44:30.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4030000000000001e-06 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266376+0000", 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2740000000000004e-06 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266381+0000", 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1919999999999996e-06 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:30.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2163 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466911+0000", 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46661142999999999, 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.467418581, 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466911+0000", 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466911+0000", 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466912+0000", 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1500000000000005e-07 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466916+0000", 2024-09-19T05:44:31.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9049999999999996e-06 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466923+0000", 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6229999999999999e-06 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466929+0000", 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9020000000000004e-06 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2184 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.562637+0000", 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.37088571300000001, 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.371720361, 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562637+0000", 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562637+0000", 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562637+0000", 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1699999999999999e-07 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562641+0000", 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7459999999999998e-06 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562647+0000", 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7559999999999996e-06 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562652+0000", 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0989999999999998e-06 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2240 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.082034604999999997, 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082900258000000004, 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:31.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862188+0000", 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010699774 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862213+0000", 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5536000000000001e-05 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862225+0000", 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1328000000000001e-05 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862231+0000", 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.55e-06 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2031 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966215+0000", 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96730728600000004, 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96820481400000002, 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966215+0000", 2024-09-19T05:44:31.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966215+0000", 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966216+0000", 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4299999999999999e-07 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966225+0000", 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1120000000000002e-06 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966231+0000", 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9499999999999998e-06 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966235+0000", 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6739999999999996e-06 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2046 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.021914+0000", 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.91160850000000004, 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91253297700000002, 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021914+0000", 2024-09-19T05:44:31.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021914+0000", 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021914+0000", 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9200000000000002e-07 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021927+0000", 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2539e-05 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021934+0000", 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0949999999999998e-06 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021940+0000", 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8039999999999998e-06 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2060 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.043864+0000", 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.88965802199999999, 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89062595700000002, 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.015 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043864+0000", 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043864+0000", 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051248+0000", 2024-09-19T05:44:31.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.007383934 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051286+0000", 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7524999999999997e-05 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051302+0000", 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5851000000000001e-05 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051314+0000", 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2634e-05 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2085 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.161690+0000", 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.77183213100000003, 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77282707399999995, 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.018 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.161690+0000", 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.161690+0000", 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.165276+0000", 2024-09-19T05:44:31.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035855100000000001 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.165285+0000", 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0960000000000007e-06 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.165300+0000", 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4919000000000001e-05 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.165309+0000", 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5389999999999996e-06 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2095 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.211709+0000", 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72181326000000001, 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72285078899999999, 2024-09-19T05:44:31.021 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211709+0000", 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211709+0000", 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.231860+0000", 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020150520000000002 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.231975+0000", 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011477899999999999 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.231990+0000", 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5687000000000001e-05 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.232002+0000", 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1749999999999999e-05 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2109 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266387+0000", 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66713509500000001, 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66820077200000005, 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266387+0000", 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266387+0000", 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266388+0000", 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2800000000000003e-07 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266391+0000", 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.619e-06 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266397+0000", 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5350000000000004e-06 2024-09-19T05:44:31.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266402+0000", 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2079999999999999e-06 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2185 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.562658+0000", 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.370864525, 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37195521300000001, 2024-09-19T05:44:31.027 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562658+0000", 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562658+0000", 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562658+0000", 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6300000000000001e-07 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562671+0000", 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2587e-05 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562678+0000", 2024-09-19T05:44:31.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8909999999999998e-06 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562684+0000", 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0039999999999996e-06 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2200 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641968+0000", 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.291554851, 2024-09-19T05:44:31.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29267739700000001, 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641968+0000", 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641968+0000", 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683492+0000", 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041524221 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683504+0000", 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2296000000000001e-05 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683517+0000", 2024-09-19T05:44:31.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2661e-05 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683532+0000", 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932999999999999e-05 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2233 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.130588921, 2024-09-19T05:44:31.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13173987500000001, 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807300+0000", 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043666679999999998 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807312+0000", 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2199e-05 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807327+0000", 2024-09-19T05:44:31.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4260999999999999e-05 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807339+0000", 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2241 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862244+0000", 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071278227999999999, 2024-09-19T05:44:31.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072454917999999993, 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862244+0000", 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862244+0000", 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862245+0000", 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862249+0000", 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1869999999999999e-06 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862256+0000", 2024-09-19T05:44:31.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0539999999999997e-06 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862262+0000", 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7019999999999998e-06 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2032 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966242+0000", 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96728015700000003, 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96848731799999999, 2024-09-19T05:44:31.039 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966242+0000", 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966242+0000", 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966243+0000", 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3700000000000001e-07 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966247+0000", 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9020000000000001e-06 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966252+0000", 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6300000000000003e-06 2024-09-19T05:44:31.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966257+0000", 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5600000000000004e-06 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2047 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.021948+0000", 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.91157462899999997, 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91280874099999998, 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021948+0000", 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021948+0000", 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021948+0000", 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4900000000000001e-07 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021952+0000", 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5889999999999999e-06 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021958+0000", 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2899999999999999e-06 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021964+0000", 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.339e-06 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2061 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.051334+0000", 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.88218840499999995, 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88345317700000003, 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051334+0000", 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051334+0000", 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051335+0000", 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5099999999999999e-07 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051339+0000", 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6630000000000001e-06 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051348+0000", 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2979999999999992e-06 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051354+0000", 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4489999999999996e-06 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2071 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.067720+0000", 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86580208700000005, 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.867113676, 2024-09-19T05:44:31.048 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067720+0000", 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067720+0000", 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090365+0000", 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022644816000000002 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090379+0000", 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3393999999999999e-05 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090395+0000", 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6039999999999999e-05 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090406+0000", 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0984e-05 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2096 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.232020+0000", 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.70150237800000004, 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70286557400000005, 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.232020+0000", 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.232020+0000", 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.233271+0000", 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012508230000000001 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.233283+0000", 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.254e-05 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.233946+0000", 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00066228499999999998 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.233968+0000", 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1764e-05 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2110 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266407+0000", 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66711502499999997, 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66852774000000004, 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.054 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266407+0000", 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266407+0000", 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266408+0000", 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4200000000000002e-07 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266423+0000", 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4834999999999999e-05 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266429+0000", 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8290000000000002e-06 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266435+0000", 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4829999999999999e-06 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2186 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.562692+0000", 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.370830836, 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37229873000000002, 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.057 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562692+0000", 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562692+0000", 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562692+0000", 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8599999999999999e-07 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562709+0000", 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7405999999999999e-05 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562717+0000", 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0990000000000001e-06 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562722+0000", 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4940000000000003e-06 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2195 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.607631+0000", 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.32589146899999999, 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32740966700000002, 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.607631+0000", 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.607631+0000", 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641771+0000", 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034139835 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641786+0000", 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5062000000000001e-05 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641801+0000", 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.558e-05 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641811+0000", 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9729999999999992e-06 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2201 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683559+0000", 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24996332399999999, 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25152781099999999, 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683559+0000", 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683559+0000", 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683560+0000", 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4499999999999997e-07 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683572+0000", 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2097e-05 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683579+0000", 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1890000000000001e-06 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683585+0000", 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6509999999999998e-06 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2234 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.12613516899999999, 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12774556200000001, 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807388+0000", 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-07 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807399+0000", 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1338999999999999e-05 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807409+0000", 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807416+0000", 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1180000000000003e-06 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2242 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862279+0000", 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071243197999999994, 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.072908078000000001, 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862279+0000", 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862279+0000", 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862280+0000", 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3799999999999998e-07 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862294+0000", 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4318000000000001e-05 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862301+0000", 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5239999999999997e-06 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862306+0000", 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3680000000000001e-06 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2033 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966275+0000", 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96724737199999999, 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96896438699999998, 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966275+0000", 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966275+0000", 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966276+0000", 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9500000000000003e-07 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966284+0000", 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6149999999999997e-06 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966291+0000", 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6699999999999997e-06 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966312+0000", 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0732000000000001e-05 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2072 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.090431+0000", 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.84309188700000004, 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84485967799999995, 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090431+0000", 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090431+0000", 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090431+0000", 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.13e-07 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090444+0000", 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2788e-05 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090453+0000", 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7299999999999994e-06 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090459+0000", 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1290000000000001e-06 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2097 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.233997+0000", 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.69952534399999999, 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70134345300000001, 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.233997+0000", 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.233997+0000", 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.233999+0000", 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.234009+0000", 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3230000000000004e-06 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.234017+0000", 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7590000000000003e-06 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.234021+0000", 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9099999999999998e-06 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2176 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.499284+0000", 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43423864600000001, 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43610079499999999, 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.081 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499284+0000", 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499284+0000", 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.515357+0000", 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016072702000000001 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.515378+0000", 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1475999999999999e-05 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.515391+0000", 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3083e-05 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.515399+0000", 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0199999999999994e-06 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2187 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.562731+0000", 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.37079196199999997, 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37272079699999999, 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562731+0000", 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562731+0000", 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562731+0000", 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5900000000000002e-07 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562740+0000", 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5969999999999992e-06 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562746+0000", 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5529999999999998e-06 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562751+0000", 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0980000000000001e-06 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2196 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641839+0000", 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.291683215, 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29366865800000003, 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.087 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641839+0000", 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641839+0000", 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641840+0000", 2024-09-19T05:44:31.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641850+0000", 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0135999999999999e-05 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641858+0000", 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9310000000000008e-06 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641865+0000", 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6320000000000002e-06 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2202 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24992974300000001, 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25197476499999999, 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.090 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4099999999999999e-07 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683597+0000", 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683603+0000", 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683607+0000", 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6160000000000004e-06 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2235 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.12609595500000001, 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.128302314, 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4000000000000002e-07 2024-09-19T05:44:31.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807435+0000", 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.072e-06 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807478+0000", 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2460999999999999e-05 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807486+0000", 2024-09-19T05:44:31.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5350000000000004e-06 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2243 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862313+0000", 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071209692000000005, 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073479002000000002, 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862313+0000", 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862313+0000", 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862313+0000", 2024-09-19T05:44:31.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862317+0000", 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.512e-06 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862323+0000", 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9630000000000003e-06 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862328+0000", 2024-09-19T05:44:31.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.75e-06 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2034 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966321+0000", 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96720170299999997, 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96953919200000005, 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.099 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966321+0000", 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966321+0000", 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966321+0000", 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2599999999999998e-07 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966332+0000", 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0558e-05 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966339+0000", 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8680000000000001e-06 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966344+0000", 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7780000000000004e-06 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2073 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.090470+0000", 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.843052778, 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84629177099999997, 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090470+0000", 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090470+0000", 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090470+0000", 2024-09-19T05:44:31.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090474+0000", 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9890000000000003e-06 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090481+0000", 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0650000000000001e-06 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090487+0000", 2024-09-19T05:44:31.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8969999999999996e-06 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2111 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266441+0000", 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66708121499999995, 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.67041945199999997, 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266441+0000", 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266441+0000", 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.269476+0000", 2024-09-19T05:44:31.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030348020000000001 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.269488+0000", 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2034e-05 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.269497+0000", 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0080000000000008e-06 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.269503+0000", 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1879999999999999e-06 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2120 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.288332+0000", 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.64519006899999998, 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.64863852399999999, 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.108 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288332+0000", 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288332+0000", 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319562+0000", 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031229528999999999 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319597+0000", 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5009999999999999e-05 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319623+0000", 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6199000000000001e-05 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319640+0000", 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7076e-05 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2197 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641875+0000", 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.29164704000000002, 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29514326299999999, 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.111 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641875+0000", 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641875+0000", 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641876+0000", 2024-09-19T05:44:31.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7100000000000002e-07 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641880+0000", 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.357e-06 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641889+0000", 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3270000000000001e-06 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641894+0000", 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.955e-06 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2203 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24990894999999999, 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25346628100000002, 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.114 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683633+0000", 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9006999999999999e-05 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683640+0000", 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683648+0000", 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4449999999999996e-06 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2236 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.126025838, 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12966181299999999, 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:31.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807502+0000", 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4850000000000003e-06 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807509+0000", 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0859999999999996e-06 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807515+0000", 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9179999999999999e-06 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2244 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862336+0000", 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071186604000000001, 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074904754000000004, 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862336+0000", 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862336+0000", 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862336+0000", 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4499999999999997e-07 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862344+0000", 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.038e-06 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862351+0000", 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9890000000000003e-06 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862357+0000", 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6180000000000003e-06 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2035 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966351+0000", 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96717137099999995, 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97099453700000005, 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966351+0000", 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966351+0000", 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966351+0000", 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7300000000000002e-07 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966355+0000", 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3850000000000002e-06 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966360+0000", 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5659999999999997e-06 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966365+0000", 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9869999999999999e-06 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2074 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.090496+0000", 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.84302694600000005, 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84707359199999999, 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.126 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090496+0000", 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090496+0000", 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090496+0000", 2024-09-19T05:44:31.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8700000000000001e-07 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090504+0000", 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5350000000000004e-06 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090512+0000", 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3529999999999997e-06 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090518+0000", 2024-09-19T05:44:31.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7440000000000004e-06 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2068 3.3c 3.674775bc (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.244903+0000", 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.68861969499999998, 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.69272683700000004, 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2068 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.244903+0000", 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.244903+0000", 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.265504+0000", 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020601310000000001 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.270658+0000", 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051535000000000001 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.270673+0000", 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5182e-05 2024-09-19T05:44:31.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.270682+0000", 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9930000000000001e-06 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2121 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.319672+0000", 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.61385067599999998, 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61802384499999996, 2024-09-19T05:44:31.132 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319672+0000", 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319672+0000", 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319672+0000", 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-07 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319719+0000", 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6128000000000003e-05 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319729+0000", 2024-09-19T05:44:31.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0641e-05 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319738+0000", 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0729999999999994e-06 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2198 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.29162181300000001, 2024-09-19T05:44:31.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.295889501, 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.03e-07 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641909+0000", 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5660000000000001e-06 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641917+0000", 2024-09-19T05:44:31.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9680000000000006e-06 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641923+0000", 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6189999999999996e-06 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2204 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683655+0000", 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24986704700000001, 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.254212893, 2024-09-19T05:44:31.138 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683655+0000", 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683655+0000", 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683656+0000", 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2300000000000002e-07 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683659+0000", 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4070000000000002e-06 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683665+0000", 2024-09-19T05:44:31.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683670+0000", 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.78e-06 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2221 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.722007+0000", 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21151556099999999, 2024-09-19T05:44:31.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.215957554, 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.722007+0000", 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.722007+0000", 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732818+0000", 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010811368999999999 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732829+0000", 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1021e-05 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732847+0000", 2024-09-19T05:44:31.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7357e-05 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732858+0000", 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1234e-05 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2237 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.126001165, 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13051320899999999, 2024-09-19T05:44:31.144 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807522+0000", 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807530+0000", 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8930000000000005e-06 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807577+0000", 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7064999999999997e-05 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807606+0000", 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8963e-05 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2245 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071158689999999997, 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075729765000000004, 2024-09-19T05:44:31.147 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.08e-07 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862368+0000", 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7929999999999999e-06 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862375+0000", 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6680000000000004e-06 2024-09-19T05:44:31.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862380+0000", 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2036 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966372+0000", 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.967150811, 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97178052999999998, 2024-09-19T05:44:31.150 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966372+0000", 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966372+0000", 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966372+0000", 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1899999999999998e-07 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966379+0000", 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4599999999999997e-06 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966386+0000", 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.319e-06 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966390+0000", 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2075 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.090525+0000", 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.84299702499999996, 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84772499099999998, 2024-09-19T05:44:31.153 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090525+0000", 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090525+0000", 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090526+0000", 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6400000000000003e-07 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090534+0000", 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6650000000000003e-06 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090541+0000", 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.391e-06 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090546+0000", 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-06 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2098 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.234026+0000", 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.699496432, 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70429995000000001, 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.234026+0000", 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.234026+0000", 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242539+0000", 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0085131809999999999 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242556+0000", 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6305000000000001e-05 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242567+0000", 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1439e-05 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242573+0000", 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4969999999999998e-06 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2177 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.515414+0000", 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.41810839999999999, 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42296818400000002, 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.159 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.515414+0000", 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.515414+0000", 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.523882+0000", 2024-09-19T05:44:31.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084676479999999995 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.523892+0000", 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.061e-05 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.523907+0000", 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4358e-05 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.523917+0000", 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0254999999999999e-05 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2188 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.562758+0000", 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.37076411199999998, 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.375724734, 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.162 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562758+0000", 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562758+0000", 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.566968+0000", 2024-09-19T05:44:31.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042092060000000001 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.566984+0000", 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6518999999999999e-05 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567004+0000", 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0279999999999999e-05 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567016+0000", 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1753000000000001e-05 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2199 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.29158381799999999, 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29767692400000001, 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.165 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:31.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7199999999999999e-07 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641947+0000", 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6820000000000003e-06 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641954+0000", 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8990000000000004e-06 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641959+0000", 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9779999999999996e-06 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2205 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24984741499999999, 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.255991682, 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-19T05:44:31.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683682+0000", 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9249999999999998e-06 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683688+0000", 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6579999999999999e-06 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683692+0000", 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5079999999999999e-06 2024-09-19T05:44:31.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2246 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071134775999999997, 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077340402000000003, 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.96e-07 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862397+0000", 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862404+0000", 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862409+0000", 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1320000000000002e-06 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.173 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2037 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966397+0000", 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96712543799999995, 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97337508399999995, 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966397+0000", 2024-09-19T05:44:31.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966397+0000", 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966397+0000", 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966401+0000", 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4089999999999999e-06 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966406+0000", 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1050000000000003e-06 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966410+0000", 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5329999999999996e-06 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2076 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.090553+0000", 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.84296929899999995, 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84927158599999997, 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090553+0000", 2024-09-19T05:44:31.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090553+0000", 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090554+0000", 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2200000000000004e-07 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090558+0000", 2024-09-19T05:44:31.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9419999999999997e-06 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090566+0000", 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8140000000000007e-06 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090572+0000", 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8510000000000004e-06 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2099 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.242586+0000", 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.69093635200000003, 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.69728556900000005, 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242586+0000", 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242586+0000", 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242587+0000", 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0500000000000003e-07 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242595+0000", 2024-09-19T05:44:31.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5210000000000002e-06 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242601+0000", 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242607+0000", 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6280000000000002e-06 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2189 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.567036+0000", 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.366486425, 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37288853900000002, 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567036+0000", 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567036+0000", 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567037+0000", 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.68e-07 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567056+0000", 2024-09-19T05:44:31.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9062000000000001e-05 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567064+0000", 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5499999999999995e-06 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567070+0000", 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1680000000000001e-06 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2140 3.3c 3.f014bd7c (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34133337899999999, 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34778690800000001, 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2140 2024-09-19T05:44:31.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641694+0000", 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049504800000000002 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646652+0000", 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049580040000000002 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646670+0000", 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646681+0000", 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1555e-05 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2206 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683698+0000", 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24982415799999999, 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25633678500000001, 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.189 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683698+0000", 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683698+0000", 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683699+0000", 2024-09-19T05:44:31.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683705+0000", 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6760000000000001e-06 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683723+0000", 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7589e-05 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683728+0000", 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0379999999999999e-06 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2247 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071107694999999999, 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077743766000000006, 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862426+0000", 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1090999999999999e-05 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862433+0000", 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8070000000000002e-06 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862438+0000", 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1599999999999997e-06 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2038 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966416+0000", 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96710602400000001, 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97379148299999996, 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966416+0000", 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966416+0000", 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966417+0000", 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3500000000000002e-07 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966437+0000", 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0081e-05 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966444+0000", 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7950000000000002e-06 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966449+0000", 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9910000000000002e-06 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2207 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683735+0000", 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24978735299999999, 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25653997200000001, 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683735+0000", 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683735+0000", 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683736+0000", 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683739+0000", 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5860000000000001e-06 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683745+0000", 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.694e-06 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683750+0000", 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2549999999999997e-06 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2243 3.20 3:0770db00:::100000001f8.000004c0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.861332+0000", 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.072190298999999999, 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079004294000000003, 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2243 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.861332+0000", 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.861332+0000", 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.861335+0000", 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.695e-06 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.863660+0000", 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023254180000000001 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.863674+0000", 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3689000000000001e-05 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.863682+0000", 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6170000000000001e-06 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.932442+0000", 2024-09-19T05:44:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068760751999999994 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.932957+0000", 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00051452799999999999 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.933224+0000", 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026715799999999998 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2039 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966456+0000", 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96706634999999996, 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97396679500000005, 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966456+0000", 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966456+0000", 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966456+0000", 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6899999999999998e-07 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966460+0000", 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3759999999999999e-06 2024-09-19T05:44:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966465+0000", 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6200000000000004e-06 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966470+0000", 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4220000000000002e-06 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2077 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.090580+0000", 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.84294296899999999, 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84989658700000004, 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090580+0000", 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.090580+0000", 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098272+0000", 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076929759999999998 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098311+0000", 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8167999999999998e-05 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098326+0000", 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5121e-05 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098372+0000", 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5837999999999997e-05 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2122 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.319752+0000", 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.61377062199999999, 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.620780043, 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319752+0000", 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.319752+0000", 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330578+0000", 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010826021999999999 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330590+0000", 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.241e-05 2024-09-19T05:44:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330608+0000", 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7833e-05 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330617+0000", 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6340000000000007e-06 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2208 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.249766768, 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.256824039, 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9999999999999999e-07 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683773+0000", 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6860000000000001e-05 2024-09-19T05:44:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683779+0000", 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3249999999999996e-06 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683784+0000", 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9660000000000004e-06 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(mds.0.4:2005 2.14 e70/19 ondisk, result = 0)", 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.871310+0000", 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.062212017000000001, 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069316864000000006, 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.871310+0000", 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.871310+0000", 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.871312+0000", 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0780000000000001e-06 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.871324+0000", 2024-09-19T05:44:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2099999999999999e-05 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.871337+0000", 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3801000000000001e-05 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.871349+0000", 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1328000000000001e-05 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.872314+0000", 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096569799999999997 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.872323+0000", 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2770000000000005e-06 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2040 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966476+0000", 2024-09-19T05:44:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96704642100000004, 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97421575100000002, 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966476+0000", 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966476+0000", 2024-09-19T05:44:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966476+0000", 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4000000000000002e-07 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966485+0000", 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1119999999999996e-06 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966490+0000", 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5500000000000002e-06 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966495+0000", 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2959999999999998e-06 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2078 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.098465+0000", 2024-09-19T05:44:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.83505721399999999, 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84227848900000002, 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098465+0000", 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098465+0000", 2024-09-19T05:44:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098466+0000", 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-07 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098470+0000", 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2980000000000002e-06 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098479+0000", 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5089999999999993e-06 2024-09-19T05:44:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098485+0000", 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0510000000000002e-06 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2112 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.269515+0000", 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66400741299999999, 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.67128069999999995, 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.269515+0000", 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.269515+0000", 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.273986+0000", 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044713460000000002 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.274005+0000", 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8762000000000001e-05 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.274028+0000", 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2657999999999999e-05 2024-09-19T05:44:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.274043+0000", 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4986e-05 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2123 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.330634+0000", 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.602888957, 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61021381299999999, 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330634+0000", 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330634+0000", 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330634+0000", 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9299999999999998e-07 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330654+0000", 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9572000000000002e-05 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330661+0000", 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7430000000000002e-06 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330671+0000", 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6490000000000003e-06 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2133 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.374230+0000", 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55929229999999996, 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56667478199999999, 2024-09-19T05:44:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374230+0000", 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374230+0000", 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388276+0000", 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014045627999999999 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388298+0000", 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2484999999999999e-05 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388313+0000", 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4351e-05 2024-09-19T05:44:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388323+0000", 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448000000000001e-05 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2222 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.732878+0000", 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.200644252, 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.208077025, 2024-09-19T05:44:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732878+0000", 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732878+0000", 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756284+0000", 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023406138999999999 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756314+0000", 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9529000000000001e-05 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756344+0000", 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9998999999999999e-05 2024-09-19T05:44:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756364+0000", 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9956e-05 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2062 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.051365+0000", 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.88215793099999995, 2024-09-19T05:44:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88965182700000001, 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051365+0000", 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.051365+0000", 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.059734+0000", 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083692890000000002 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.059753+0000", 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8899999999999999e-05 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.059765+0000", 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2136e-05 2024-09-19T05:44:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.059773+0000", 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9840000000000001e-06 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2079 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.098493+0000", 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.835028997, 2024-09-19T05:44:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84257787200000001, 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098493+0000", 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098493+0000", 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098494+0000", 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098501+0000", 2024-09-19T05:44:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3579999999999997e-06 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098508+0000", 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0319999999999998e-06 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098513+0000", 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0289999999999996e-06 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2124 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.330680+0000", 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.60284259799999995, 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61044725600000005, 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330680+0000", 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330680+0000", 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330680+0000", 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7300000000000001e-07 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330690+0000", 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0067e-05 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330696+0000", 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0070000000000003e-06 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330701+0000", 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0509999999999996e-06 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2134 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.388347+0000", 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.54517507600000004, 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55283367100000003, 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388347+0000", 2024-09-19T05:44:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388347+0000", 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388348+0000", 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4899999999999995e-07 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388352+0000", 2024-09-19T05:44:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9260000000000002e-06 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388359+0000", 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1040000000000001e-06 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388364+0000", 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6779999999999999e-06 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2011 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.863546+0000", 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.069976762, 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.077684375, 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863546+0000", 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863546+0000", 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.886173+0000", 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022627423000000001 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.886183+0000", 2024-09-19T05:44:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0142999999999999e-05 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.886198+0000", 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4824000000000001e-05 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.886209+0000", 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.131e-05 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2048 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.021971+0000", 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.91155160199999996, 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91931962, 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021971+0000", 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021971+0000", 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.029279+0000", 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073077849999999998 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.029299+0000", 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0723000000000001e-05 2024-09-19T05:44:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.029317+0000", 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7779999999999999e-05 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.029327+0000", 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088000000000001e-05 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2063 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.059789+0000", 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.87373317800000005, 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88155465799999999, 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.059789+0000", 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.059789+0000", 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061274+0000", 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014847949999999999 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061299+0000", 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4864000000000001e-05 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061313+0000", 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3888e-05 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061322+0000", 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3309999999999993e-06 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2080 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.098521+0000", 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.83500108699999998, 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84287746600000002, 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098521+0000", 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.098521+0000", 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099079+0000", 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00055809000000000002 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099085+0000", 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4890000000000003e-06 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099262+0000", 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176968 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099275+0000", 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3147e-05 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2086 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.165327+0000", 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.76819583700000005, 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.776124959, 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.165327+0000", 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.165327+0000", 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191702+0000", 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026375338000000002 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191723+0000", 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0608e-05 2024-09-19T05:44:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191738+0000", 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5906e-05 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191751+0000", 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2043e-05 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2113 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.274072+0000", 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.65945047800000001, 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66743540400000001, 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.274072+0000", 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.274072+0000", 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.277273+0000", 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003200952 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.277287+0000", 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3749e-05 2024-09-19T05:44:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.277299+0000", 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1924e-05 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.277305+0000", 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5579999999999997e-06 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2125 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.330708+0000", 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.60281442900000004, 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61085272499999999, 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330708+0000", 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330708+0000", 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330708+0000", 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0999999999999999e-07 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330716+0000", 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.875e-06 2024-09-19T05:44:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330723+0000", 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.285e-06 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330727+0000", 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.814e-06 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2279 3.16 3:680f8022:::100000001f8.000004e4:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.863698+0000", 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.069824396999999996, 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077915337000000001, 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2279 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.863698+0000", 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.863698+0000", 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.873278+0000", 2024-09-19T05:44:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0095795589999999996 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876031+0000", 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027537550000000001 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876053+0000", 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1861e-05 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876066+0000", 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3188e-05 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876149+0000", 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2520000000000003e-05 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876314+0000", 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165155 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876473+0000", 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158777 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.929965+0000", 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053491711999999997 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2041 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966502+0000", 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96702051200000005, 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97520485099999998, 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966502+0000", 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966502+0000", 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.978379+0000", 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011877067 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.978463+0000", 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4400999999999994e-05 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.978485+0000", 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1217999999999999e-05 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.978529+0000", 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4299999999999999e-05 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2064 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.061340+0000", 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.87218285500000003, 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88042292099999997, 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061340+0000", 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061340+0000", 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061340+0000", 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6000000000000004e-07 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061344+0000", 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0620000000000002e-06 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061352+0000", 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5070000000000003e-06 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061357+0000", 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.677e-06 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2081 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.099287+0000", 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.83423518100000005, 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84253084099999997, 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099287+0000", 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099287+0000", 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099288+0000", 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5199999999999997e-07 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099303+0000", 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5469e-05 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099312+0000", 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5080000000000005e-06 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099317+0000", 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6609999999999997e-06 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2087 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.191773+0000", 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.74174921400000005, 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75010403299999995, 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191773+0000", 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191773+0000", 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191774+0000", 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9500000000000002e-07 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191796+0000", 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2313000000000001e-05 2024-09-19T05:44:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191804+0000", 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191810+0000", 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0700000000000003e-06 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2126 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.330734+0000", 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.60278863500000002, 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61120447600000005, 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330734+0000", 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330734+0000", 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330734+0000", 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0600000000000001e-07 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330738+0000", 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0300000000000004e-06 2024-09-19T05:44:31.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330744+0000", 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8479999999999997e-06 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330749+0000", 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4390000000000002e-06 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2223 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.756404+0000", 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.17711824500000001, 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18558698600000001, 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756404+0000", 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756404+0000", 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769113+0000", 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01270903 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769143+0000", 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9624999999999998e-05 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769160+0000", 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7255e-05 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769173+0000", 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3079e-05 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.296 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2292 3.20 3.be2f78e0 (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.876110+0000", 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.057412812000000001, 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065927586999999996, 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2292 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876110+0000", 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876110+0000", 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.876113+0000", 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3720000000000001e-06 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.880716+0000", 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046030489999999997 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.880756+0000", 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9684000000000002e-05 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.880769+0000", 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3354e-05 2024-09-19T05:44:31.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2049 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.029345+0000", 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.90417725800000004, 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91274927299999997, 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.029345+0000", 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.029345+0000", 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033391+0000", 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040460369999999997 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033404+0000", 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3018e-05 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033429+0000", 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4978999999999999e-05 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033445+0000", 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5628999999999999e-05 2024-09-19T05:44:31.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2065 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.061366+0000", 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.87215672600000005, 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88077675300000002, 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061366+0000", 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061366+0000", 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061366+0000", 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1099999999999996e-07 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061374+0000", 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1799999999999996e-06 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061382+0000", 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2439999999999996e-06 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061387+0000", 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7200000000000003e-06 2024-09-19T05:44:31.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2088 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.191819+0000", 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.74170389299999995, 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75037193599999996, 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191819+0000", 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191819+0000", 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191819+0000", 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1100000000000001e-07 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191822+0000", 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3919999999999999e-06 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191828+0000", 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8769999999999998e-06 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191833+0000", 2024-09-19T05:44:31.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9779999999999996e-06 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2127 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.330754+0000", 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.60276813500000004, 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61148401699999999, 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330754+0000", 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330754+0000", 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330755+0000", 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2300000000000002e-07 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330762+0000", 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3950000000000003e-06 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330768+0000", 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2500000000000003e-06 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330774+0000", 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2650000000000004e-06 2024-09-19T05:44:31.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2135 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.388371+0000", 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.54515166999999998, 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55394610399999999, 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.312 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388371+0000", 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.388371+0000", 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422304+0000", 2024-09-19T05:44:31.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033933377000000001 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422324+0000", 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0128000000000001e-05 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422340+0000", 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6079e-05 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422351+0000", 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1071e-05 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2146 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.442401+0000", 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.49112139399999999, 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.499962976, 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.442401+0000", 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.442401+0000", 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462277+0000", 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019876028 2024-09-19T05:44:31.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462296+0000", 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9279999999999998e-05 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462318+0000", 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1637e-05 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462339+0000", 2024-09-19T05:44:31.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1352000000000002e-05 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2190 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.567080+0000", 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.36644243700000001, 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37533149399999999, 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567080+0000", 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.567080+0000", 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.584734+0000", 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017653762 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.584880+0000", 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014589599999999999 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.584994+0000", 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114362 2024-09-19T05:44:31.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585007+0000", 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2866e-05 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2224 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.769203+0000", 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.16431980900000001, 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.173258202, 2024-09-19T05:44:31.321 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769203+0000", 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769203+0000", 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.774066+0000", 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048633319999999997 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.774074+0000", 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9929999999999995e-06 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.774091+0000", 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7139999999999999e-05 2024-09-19T05:44:31.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.774101+0000", 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0305e-05 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2050 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.033477+0000", 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.90004585299999995, 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90903183799999998, 2024-09-19T05:44:31.324 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033477+0000", 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033477+0000", 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033477+0000", 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3e-07 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033491+0000", 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3314e-05 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033498+0000", 2024-09-19T05:44:31.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4540000000000001e-06 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033505+0000", 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.567e-06 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2089 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.191839+0000", 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.74168321400000004, 2024-09-19T05:44:31.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75071759100000002, 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191839+0000", 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191839+0000", 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191840+0000", 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4299999999999999e-07 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191847+0000", 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.328e-06 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191853+0000", 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9560000000000002e-06 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191863+0000", 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9219999999999992e-06 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2128 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.330780+0000", 2024-09-19T05:44:31.330 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.60274264, 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61182503700000002, 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330780+0000", 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.330780+0000", 2024-09-19T05:44:31.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334433+0000", 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036532280000000001 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334461+0000", 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.773e-05 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334475+0000", 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4151999999999999e-05 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334485+0000", 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0107e-05 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2136 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.333 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.422378+0000", 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51114423600000003, 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52027635000000005, 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422378+0000", 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422378+0000", 2024-09-19T05:44:31.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422379+0000", 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5799999999999999e-07 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422385+0000", 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7420000000000003e-06 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422393+0000", 2024-09-19T05:44:31.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.174e-06 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422399+0000", 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1369999999999999e-06 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2147 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.462373+0000", 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.47114983599999999, 2024-09-19T05:44:31.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48033028900000002, 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462373+0000", 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462373+0000", 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462373+0000", 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1999999999999999e-07 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462387+0000", 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.329e-05 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462396+0000", 2024-09-19T05:44:31.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6099999999999995e-06 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462403+0000", 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4300000000000003e-06 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2191 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.585032+0000", 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34849063200000002, 2024-09-19T05:44:31.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35772001599999997, 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585032+0000", 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585032+0000", 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585033+0000", 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9700000000000005e-07 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585049+0000", 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6722999999999999e-05 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585058+0000", 2024-09-19T05:44:31.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5869999999999993e-06 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585064+0000", 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4470000000000003e-06 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2305 3.5 3:a0445fe6:::100000001f8.000004fe:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.880800+0000", 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.052722390000000001, 2024-09-19T05:44:31.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.062005658999999998, 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2305 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.880800+0000", 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.880800+0000", 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.880802+0000", 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2010000000000002e-06 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.883709+0000", 2024-09-19T05:44:31.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029067390000000002 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.883731+0000", 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2433999999999999e-05 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.883745+0000", 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3183000000000001e-05 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.884054+0000", 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030981599999999998 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.884243+0000", 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000188316 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.884492+0000", 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024910900000000002 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2051 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.033513+0000", 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.90000972700000004, 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.909365541, 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033513+0000", 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033513+0000", 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033513+0000", 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.84e-07 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033521+0000", 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.554e-06 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033528+0000", 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2429999999999999e-06 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033533+0000", 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5670000000000002e-06 2024-09-19T05:44:31.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2090 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.191869+0000", 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.74165310200000001, 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75105841799999995, 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191869+0000", 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191869+0000", 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191870+0000", 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2100000000000002e-07 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191873+0000", 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6370000000000001e-06 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191879+0000", 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3090000000000003e-06 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191883+0000", 2024-09-19T05:44:31.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5430000000000004e-06 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2129 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.334503+0000", 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.59901949499999996, 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60847616699999996, 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334503+0000", 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334503+0000", 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334503+0000", 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8599999999999998e-07 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334508+0000", 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2939999999999999e-06 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334515+0000", 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0040000000000002e-06 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334520+0000", 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4079999999999997e-06 2024-09-19T05:44:31.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2137 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.422407+0000", 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51111537500000004, 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52062042200000003, 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422407+0000", 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422407+0000", 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422408+0000", 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2800000000000002e-07 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422415+0000", 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6699999999999994e-06 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422423+0000", 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3679999999999996e-06 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422430+0000", 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6420000000000007e-06 2024-09-19T05:44:31.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2148 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.462412+0000", 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.47111001699999999, 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48065978999999998, 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462412+0000", 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462412+0000", 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462413+0000", 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4000000000000001e-07 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462418+0000", 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5879999999999999e-06 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462424+0000", 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5719999999999999e-06 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462429+0000", 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.169e-06 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.361 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2178 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.523935+0000", 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.40958761399999999, 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41918471000000002, 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.523935+0000", 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.523935+0000", 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540285+0000", 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016350414000000001 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540307+0000", 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1421e-05 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540327+0000", 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0426000000000001e-05 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540341+0000", 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3774000000000001e-05 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.364 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2192 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.585077+0000", 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34844498499999998, 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35808769000000001, 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585077+0000", 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585077+0000", 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585078+0000", 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8299999999999997e-07 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585083+0000", 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.916e-06 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585090+0000", 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.934e-06 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585095+0000", 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7060000000000003e-06 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.367 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2225 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.774122+0000", 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.159400663, 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16909422700000001, 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.774122+0000", 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.774122+0000", 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.783408+0000", 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0092857570000000004 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.783523+0000", 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115003 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.783541+0000", 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8635999999999999e-05 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.783598+0000", 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6607999999999997e-05 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:2303 3.36 e70/20 3:6f9fe141:::100000001f8.000004fc:head v 70'62, mlcod=70'60)", 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.884428+0000", 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.049094764999999999, 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058851239999999999, 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.884428+0000", 2024-09-19T05:44:31.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.884428+0000", 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.884431+0000", 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3380000000000001e-06 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.894766+0000", 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010335165 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.894789+0000", 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3237000000000001e-05 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.894804+0000", 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5019e-05 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.894989+0000", 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018407800000000001 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.895015+0000", 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6645999999999999e-05 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2052 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.033541+0000", 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.89998159600000005, 2024-09-19T05:44:31.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90979378799999999, 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033541+0000", 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033541+0000", 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033541+0000", 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9999999999999998e-07 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033545+0000", 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2619999999999999e-06 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033550+0000", 2024-09-19T05:44:31.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7950000000000004e-06 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033555+0000", 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9599999999999999e-06 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2066 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.061396+0000", 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.87212688199999999, 2024-09-19T05:44:31.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88198690300000004, 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061396+0000", 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.061396+0000", 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067424+0000", 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060285499999999997 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067543+0000", 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118489 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067558+0000", 2024-09-19T05:44:31.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4868999999999999e-05 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067566+0000", 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5399999999999996e-06 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2091 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.191890+0000", 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.74163244699999997, 2024-09-19T05:44:31.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75154069199999995, 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191890+0000", 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191890+0000", 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191890+0000", 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8799999999999998e-07 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191943+0000", 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2831000000000003e-05 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191978+0000", 2024-09-19T05:44:31.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4650999999999997e-05 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.191985+0000", 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7800000000000003e-06 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2114 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.277320+0000", 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.65620259800000003, 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66615842300000006, 2024-09-19T05:44:31.383 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.277320+0000", 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.277320+0000", 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.284978+0000", 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076579980000000001 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.284993+0000", 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5034999999999999e-05 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285015+0000", 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1676e-05 2024-09-19T05:44:31.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285027+0000", 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2391e-05 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2130 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.334528+0000", 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.59899454299999999, 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60899620700000001, 2024-09-19T05:44:31.386 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334528+0000", 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334528+0000", 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334528+0000", 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334536+0000", 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0229999999999992e-06 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334543+0000", 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334549+0000", 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4229999999999996e-06 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2138 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.422439+0000", 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51108386800000005, 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52234275600000002, 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422439+0000", 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422439+0000", 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422439+0000", 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9500000000000003e-07 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422442+0000", 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3129999999999999e-06 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422449+0000", 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422455+0000", 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7919999999999998e-06 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2149 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.462439+0000", 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.47108390300000003, 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48239100600000001, 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462439+0000", 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.462439+0000", 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464275+0000", 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018364049999999999 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464295+0000", 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0462000000000002e-05 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464308+0000", 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2797e-05 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464318+0000", 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4490000000000005e-06 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2164 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466936+0000", 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46658634300000001, 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47793912199999999, 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466936+0000", 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466936+0000", 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498740+0000", 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031803605999999998 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498800+0000", 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0303000000000002e-05 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498817+0000", 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6674999999999999e-05 2024-09-19T05:44:31.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498828+0000", 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1671000000000001e-05 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2179 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.540364+0000", 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.39315841000000001, 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40455643400000002, 2024-09-19T05:44:31.398 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540364+0000", 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540364+0000", 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540365+0000", 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3e-07 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540370+0000", 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3310000000000003e-06 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540378+0000", 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9179999999999994e-06 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540385+0000", 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2193 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.585102+0000", 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34842030499999999, 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35986469199999999, 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585102+0000", 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585102+0000", 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585103+0000", 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8799999999999998e-07 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585110+0000", 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7279999999999995e-06 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585117+0000", 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6019999999999996e-06 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585122+0000", 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9200000000000003e-06 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2248 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.071077696999999995, 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082567123000000006, 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877451+0000", 2024-09-19T05:44:31.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015005956000000001 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877461+0000", 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0034000000000001e-05 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.896137+0000", 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018675967000000002 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909217+0000", 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013080629999999999 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2053 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.033561+0000", 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.89996116800000003, 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91149718099999999, 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.407 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033561+0000", 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033561+0000", 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033562+0000", 2024-09-19T05:44:31.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.41e-07 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033570+0000", 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4400000000000005e-06 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033576+0000", 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9279999999999998e-06 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033581+0000", 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9949999999999996e-06 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2067 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.067582+0000", 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86594020699999996, 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.877522472, 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.410 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067582+0000", 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067582+0000", 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067583+0000", 2024-09-19T05:44:31.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7899999999999998e-07 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067603+0000", 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0166999999999999e-05 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067610+0000", 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3080000000000002e-06 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067617+0000", 2024-09-19T05:44:31.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2650000000000002e-06 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2115 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.285057+0000", 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.64846516099999996, 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66009211499999998, 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285057+0000", 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285057+0000", 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285058+0000", 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2500000000000005e-07 2024-09-19T05:44:31.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285078+0000", 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0513000000000002e-05 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285087+0000", 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7399999999999993e-06 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285094+0000", 2024-09-19T05:44:31.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3910000000000003e-06 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2139 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.422462+0000", 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51106094000000002, 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52273369199999997, 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422462+0000", 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422462+0000", 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422462+0000", 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9099999999999999e-07 2024-09-19T05:44:31.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.420 INFO:teuthology.orchestra.run.smithi123.stdout: marking collection for removal 2024-09-19T05:44:31.420 INFO:teuthology.orchestra.run.smithi123.stdout:setting '_remove' omap key 2024-09-19T05:44:31.420 INFO:teuthology.orchestra.run.smithi123.stdout:finish_remove_pgs 3.18_head removing 3.18 2024-09-19T05:44:31.420 INFO:teuthology.orchestra.run.smithi123.stdout:Remove successful 2024-09-19T05:44:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422469+0000", 2024-09-19T05:44:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2389999999999996e-06 2024-09-19T05:44:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422476+0000", 2024-09-19T05:44:31.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.233e-06 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422482+0000", 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5310000000000001e-06 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2150 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.464338+0000", 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46918464999999998, 2024-09-19T05:44:31.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48114077500000002, 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464338+0000", 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464338+0000", 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464338+0000", 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9999999999999997e-07 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464343+0000", 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5229999999999997e-06 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464350+0000", 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1149999999999996e-06 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464356+0000", 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8109999999999999e-06 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2165 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.498854+0000", 2024-09-19T05:44:31.426 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.434668518, 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44667015900000001, 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498854+0000", 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498854+0000", 2024-09-19T05:44:31.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498855+0000", 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5900000000000005e-07 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498862+0000", 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1439999999999997e-06 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498871+0000", 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0659999999999993e-06 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498878+0000", 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6329999999999999e-06 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2180 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.540395+0000", 2024-09-19T05:44:31.429 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.39312784299999998, 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.405176329, 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540395+0000", 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540395+0000", 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540395+0000", 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1900000000000003e-07 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540405+0000", 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9110000000000005e-06 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540412+0000", 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0779999999999998e-06 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540418+0000", 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6210000000000001e-06 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.432 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2209 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683791+0000", 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.249731394, 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26182721399999997, 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683791+0000", 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683791+0000", 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721534+0000", 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037743092999999998 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721554+0000", 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0052e-05 2024-09-19T05:44:31.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721595+0000", 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0649000000000003e-05 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721619+0000", 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3777e-05 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2249 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.909254+0000", 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024268499999999998, 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036411865000000002, 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909254+0000", 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909254+0000", 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909257+0000", 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5620000000000002e-06 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909822+0000", 2024-09-19T05:44:31.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056531599999999995 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909845+0000", 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2921e-05 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909860+0000", 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.505e-05 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2054 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.033587+0000", 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.899935496, 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91213219999999995, 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033587+0000", 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033587+0000", 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033587+0000", 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5999999999999999e-07 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033591+0000", 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1750000000000001e-06 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033596+0000", 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.772e-06 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033649+0000", 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2809000000000002e-05 2024-09-19T05:44:31.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2068 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.067625+0000", 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.865897743, 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87814181499999999, 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067625+0000", 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067625+0000", 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067625+0000", 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067629+0000", 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1309999999999999e-06 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067635+0000", 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7169999999999996e-06 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067640+0000", 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5739999999999997e-06 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.444 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2116 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.285103+0000", 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.648419309, 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66071553500000002, 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285103+0000", 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285103+0000", 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285104+0000", 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5700000000000002e-07 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285112+0000", 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5629999999999992e-06 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285119+0000", 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1049999999999997e-06 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285124+0000", 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9699999999999998e-06 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2140 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.422490+0000", 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51103278900000004, 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52337801699999997, 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422490+0000", 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422490+0000", 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422490+0000", 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8299999999999997e-07 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422512+0000", 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1347000000000001e-05 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422519+0000", 2024-09-19T05:44:31.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.909e-06 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422526+0000", 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5050000000000004e-06 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2166 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.498889+0000", 2024-09-19T05:44:31.453 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43463392000000001, 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44702701500000003, 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498889+0000", 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498889+0000", 2024-09-19T05:44:31.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498889+0000", 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9400000000000004e-07 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498904+0000", 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4430000000000001e-05 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498912+0000", 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6090000000000001e-06 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498920+0000", 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1600000000000001e-06 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2210 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721662+0000", 2024-09-19T05:44:31.456 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21186033600000001, 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22430512999999999, 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721662+0000", 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721662+0000", 2024-09-19T05:44:31.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721663+0000", 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7199999999999997e-07 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721685+0000", 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1999999999999999e-05 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721694+0000", 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3379999999999994e-06 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721703+0000", 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7669999999999992e-06 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2238 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.12590330999999999, 2024-09-19T05:44:31.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13839663099999999, 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834275+0000", 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026656197999999999 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834287+0000", 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1691e-05 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834301+0000", 2024-09-19T05:44:31.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3974e-05 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834310+0000", 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8659999999999995e-06 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:2316 3.14 e70/20 3:2870de94:::100000001f8.00000509:head v 70'53, mlcod=70'49)", 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.911306+0000", 2024-09-19T05:44:31.462 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022215995999999998, 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034764431999999998, 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.911306+0000", 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.911306+0000", 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.911308+0000", 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.95e-06 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.912930+0000", 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016215660000000001 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915484+0000", 2024-09-19T05:44:31.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025544299999999999 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915504+0000", 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9656999999999998e-05 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.937851+0000", 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022346469000000001 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.937874+0000", 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3524e-05 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2012 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.886235+0000", 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0472878370000001, 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.059897699, 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.886235+0000", 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.886235+0000", 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.909697+0000", 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023462218999999999 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.909721+0000", 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4345000000000002e-05 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.909742+0000", 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0715000000000001e-05 2024-09-19T05:44:31.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.909756+0000", 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.411e-05 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2018 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.911490+0000", 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0220325859999999, 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.034699115, 2024-09-19T05:44:31.469 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911490+0000", 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911490+0000", 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965852+0000", 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054361797000000003 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965865+0000", 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3332999999999999e-05 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965883+0000", 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8383e-05 2024-09-19T05:44:31.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965895+0000", 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1731e-05 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2055 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.033659+0000", 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.89986337999999999, 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91258045799999998, 2024-09-19T05:44:31.472 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033659+0000", 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.033659+0000", 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.035279+0000", 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016194849999999999 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.035316+0000", 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7749000000000001e-05 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.035341+0000", 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5049999999999999e-05 2024-09-19T05:44:31.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.035358+0000", 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6690000000000001e-05 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2069 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.067646+0000", 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86587682899999996, 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87865791800000004, 2024-09-19T05:44:31.475 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067646+0000", 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067646+0000", 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067646+0000", 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5199999999999998e-07 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067654+0000", 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2290000000000003e-06 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067661+0000", 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3439999999999997e-06 2024-09-19T05:44:31.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067665+0000", 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999998e-06 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2117 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.285132+0000", 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.64839056799999994, 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66121993499999998, 2024-09-19T05:44:31.478 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285132+0000", 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.285132+0000", 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288149+0000", 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003017311 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288165+0000", 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6252e-05 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288187+0000", 2024-09-19T05:44:31.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1296000000000001e-05 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288202+0000", 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5345e-05 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2141 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.422535+0000", 2024-09-19T05:44:31.481 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51098767899999997, 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52386423900000001, 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422535+0000", 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422535+0000", 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422535+0000", 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0999999999999999e-07 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422539+0000", 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4599999999999999e-06 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422545+0000", 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422551+0000", 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.716e-06 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.484 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2151 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.464364+0000", 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46915846, 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48208601400000001, 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464364+0000", 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.464364+0000", 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466321+0000", 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00195707 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466361+0000", 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9743999999999998e-05 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466384+0000", 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3164e-05 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466404+0000", 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0199000000000001e-05 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2167 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.498931+0000", 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43459188999999998, 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44756621099999999, 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498931+0000", 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498931+0000", 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498932+0000", 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5199999999999995e-07 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498938+0000", 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4899999999999997e-06 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498945+0000", 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3810000000000001e-06 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498950+0000", 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9049999999999996e-06 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2211 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.490 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721715+0000", 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21180707800000001, 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.224829738, 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721715+0000", 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721715+0000", 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721716+0000", 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8299999999999996e-07 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721738+0000", 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1985e-05 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721745+0000", 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.267e-06 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721752+0000", 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1870000000000002e-06 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2311 3.5 3.fb81ea05 (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.883857+0000", 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.049665111999999997, 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.062731704999999999, 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2311 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.883857+0000", 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.883857+0000", 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.883860+0000", 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7159999999999999e-06 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.887561+0000", 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037008840000000002 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915851+0000", 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028289696 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915865+0000", 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4674e-05 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.496 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2019 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.965916+0000", 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.967606823, 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98073273100000002, 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965916+0000", 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965916+0000", 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965916+0000", 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9500000000000003e-07 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965930+0000", 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3375e-05 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965936+0000", 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5690000000000001e-06 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965942+0000", 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4539999999999998e-06 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2070 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.067672+0000", 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.86585062000000002, 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87902193799999995, 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067672+0000", 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067672+0000", 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067672+0000", 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5699999999999998e-07 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067679+0000", 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.99e-06 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067686+0000", 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3840000000000002e-06 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.067691+0000", 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4890000000000003e-06 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2118 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.288233+0000", 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.64528928500000005, 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65850628099999997, 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288233+0000", 2024-09-19T05:44:31.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288233+0000", 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288234+0000", 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.06e-07 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288248+0000", 2024-09-19T05:44:31.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3691e-05 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288256+0000", 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7909999999999993e-06 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288294+0000", 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7542000000000001e-05 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2142 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.422558+0000", 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.51096397999999998, 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52423114900000001, 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422558+0000", 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.422558+0000", 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429152+0000", 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065935940000000004 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429174+0000", 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1759e-05 2024-09-19T05:44:31.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429185+0000", 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1429e-05 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429193+0000", 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9209999999999992e-06 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.508 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2152 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466445+0000", 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.467077307, 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48039242500000001, 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466445+0000", 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466445+0000", 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466446+0000", 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4300000000000002e-07 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466455+0000", 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3789999999999995e-06 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466464+0000", 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6969999999999999e-06 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466472+0000", 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6750000000000002e-06 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2168 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.511 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.498958+0000", 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43456466900000001, 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44792830900000002, 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498958+0000", 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498958+0000", 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498958+0000", 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498973+0000", 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4941e-05 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498981+0000", 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2230000000000007e-06 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498990+0000", 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8859999999999993e-06 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2139 3.3c 3.3bf1883c (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.514 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.585619+0000", 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34790381300000001, 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36131238700000001, 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2139 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585619+0000", 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585619+0000", 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.589843+0000", 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042238459999999999 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592123+0000", 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022805809999999998 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592147+0000", 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4358999999999998e-05 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592163+0000", 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5545000000000001e-05 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2212 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721760+0000", 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21176281799999999, 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22522824399999999, 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721760+0000", 2024-09-19T05:44:31.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721760+0000", 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721760+0000", 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3799999999999997e-07 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721764+0000", 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.495e-06 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721770+0000", 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5490000000000003e-06 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721777+0000", 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.596e-06 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.520 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4534.0:2305 3.5 e70/20)", 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.912702+0000", 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.020820861, 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034328524999999999, 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.912702+0000", 2024-09-19T05:44:31.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.912702+0000", 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.912704+0000", 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.576e-06 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.912719+0000", 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4392999999999999e-05 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920718+0000", 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0079998780000000002 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920742+0000", 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3589000000000001e-05 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2013 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.909785+0000", 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0237371, 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0372902820000001, 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.909785+0000", 2024-09-19T05:44:31.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.909785+0000", 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911324+0000", 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015386 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911333+0000", 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0289999999999994e-06 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911345+0000", 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1997999999999999e-05 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911352+0000", 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3839999999999999e-06 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2020 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.965950+0000", 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96757277799999997, 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98117422899999995, 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965950+0000", 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965950+0000", 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965950+0000", 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9799999999999999e-07 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965954+0000", 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8079999999999998e-06 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965960+0000", 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6489999999999996e-06 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965965+0000", 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0810000000000001e-06 2024-09-19T05:44:31.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2119 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.288306+0000", 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.645216609, 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65886370699999997, 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288306+0000", 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288306+0000", 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288306+0000", 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8299999999999997e-07 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288311+0000", 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7480000000000001e-06 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288318+0000", 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2180000000000002e-06 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.288324+0000", 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6699999999999999e-06 2024-09-19T05:44:31.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2143 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.429211+0000", 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.50431181199999997, 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51800730900000003, 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429211+0000", 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429211+0000", 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429211+0000", 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3399999999999999e-07 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429215+0000", 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0380000000000001e-06 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429222+0000", 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.153e-06 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429228+0000", 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2769999999999996e-06 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.535 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2153 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466482+0000", 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46704037799999998, 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48077848699999998, 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466482+0000", 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466482+0000", 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466483+0000", 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2600000000000002e-07 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466488+0000", 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0270000000000003e-06 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466496+0000", 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5979999999999997e-06 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466503+0000", 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2169 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.498999+0000", 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43452384999999999, 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44831510600000002, 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498999+0000", 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498999+0000", 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.498999+0000", 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4799999999999997e-07 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499030+0000", 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0378999999999998e-05 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499045+0000", 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5092000000000001e-05 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499052+0000", 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9210000000000003e-06 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2194 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.585130+0000", 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34839222800000003, 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36222850600000001, 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585130+0000", 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.585130+0000", 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.607560+0000", 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022430248 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.607584+0000", 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3394e-05 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.607598+0000", 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3879e-05 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.607608+0000", 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.006e-05 2024-09-19T05:44:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2213 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721784+0000", 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21173818899999999, 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22562030299999999, 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721784+0000", 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721784+0000", 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721785+0000", 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9999999999999997e-07 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721794+0000", 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.901e-06 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721800+0000", 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-06 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721806+0000", 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7810000000000002e-06 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2226 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.783707+0000", 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.149815111, 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16375018799999999, 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.783707+0000", 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.783707+0000", 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802634+0000", 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018927009000000002 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802661+0000", 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.641e-05 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802695+0000", 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4137000000000002e-05 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802714+0000", 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9216999999999999e-05 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:2318 3.2b e70/20 3:d476c459:::100000001f8.0000050b:head v 70'40, mlcod=70'38)", 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.915991+0000", 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017531492999999999, 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031515709000000003, 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915991+0000", 2024-09-19T05:44:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915991+0000", 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915992+0000", 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1170000000000001e-06 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920808+0000", 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048163110000000002 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920824+0000", 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5654e-05 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920832+0000", 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4549999999999998e-06 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.921297+0000", 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00046533000000000002 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.921329+0000", 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1917e-05 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2014 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.911367+0000", 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.022155328, 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.036205906, 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911367+0000", 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911367+0000", 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911368+0000", 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3099999999999998e-07 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911379+0000", 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1589e-05 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911386+0000", 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911391+0000", 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5289999999999999e-06 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2021 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.965971+0000", 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96755123300000001, 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98165457199999995, 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965971+0000", 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965971+0000", 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965972+0000", 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1300000000000001e-07 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965980+0000", 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460000000000003e-06 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965986+0000", 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1550000000000004e-06 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965992+0000", 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6370000000000004e-06 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2144 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.429239+0000", 2024-09-19T05:44:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.50428372600000004, 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51844386799999997, 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429239+0000", 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429239+0000", 2024-09-19T05:44:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429239+0000", 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4300000000000003e-07 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429248+0000", 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6130000000000004e-06 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429254+0000", 2024-09-19T05:44:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2829999999999998e-06 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429259+0000", 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0989999999999998e-06 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2154 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466512+0000", 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46701051599999999, 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48122079800000001, 2024-09-19T05:44:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466512+0000", 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466512+0000", 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466513+0000", 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1099999999999995e-07 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466522+0000", 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7389999999999994e-06 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466531+0000", 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3639999999999999e-06 2024-09-19T05:44:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466537+0000", 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.297e-06 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2170 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.499063+0000", 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43445909300000002, 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44871871400000002, 2024-09-19T05:44:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499063+0000", 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499063+0000", 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499064+0000", 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499072+0000", 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499080+0000", 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8869999999999992e-06 2024-09-19T05:44:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499085+0000", 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1499999999999998e-06 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2214 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721814+0000", 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21170851900000001, 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.226019373, 2024-09-19T05:44:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721814+0000", 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721814+0000", 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721814+0000", 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9999999999999998e-07 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721818+0000", 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6490000000000001e-06 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721825+0000", 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.798e-06 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721831+0000", 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2369999999999997e-06 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2227 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802743+0000", 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.13077944599999999, 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.145139672, 2024-09-19T05:44:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802743+0000", 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802743+0000", 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802744+0000", 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.99e-07 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802758+0000", 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4302e-05 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802766+0000", 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5229999999999995e-06 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802773+0000", 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8929999999999999e-06 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:2281 3.14 e70/20 3:29b04b41:::100000001f8.000004e6:head v 70'51, mlcod=70'49)", 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.848908+0000", 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.084614124999999998, 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099026647999999995, 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.848908+0000", 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.848908+0000", 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.848909+0000", 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.84e-07 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.850908+0000", 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001999392 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.850929+0000", 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1188000000000001e-05 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.850947+0000", 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7992999999999999e-05 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.858146+0000", 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071983280000000004 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.858167+0000", 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1056999999999998e-05 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:2324 3.2 e70/20 3:4006f985:::100000001f8.00000511:head v 70'44, mlcod=70'42)", 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.920856+0000", 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.012666778, 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027133630999999998, 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920856+0000", 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920856+0000", 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.920857+0000", 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4330000000000001e-06 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.928823+0000", 2024-09-19T05:44:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0079659899999999992 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.928855+0000", 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2094000000000001e-05 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.928870+0000", 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4857e-05 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.928991+0000", 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000121125 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.929017+0000", 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5394000000000001e-05 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2015 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.911400+0000", 2024-09-19T05:44:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.022122231, 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0366443240000001, 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911400+0000", 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911400+0000", 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911401+0000", 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7899999999999999e-07 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911405+0000", 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0380000000000001e-06 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911411+0000", 2024-09-19T05:44:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8549999999999998e-06 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911416+0000", 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2022 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.965998+0000", 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96752440299999998, 2024-09-19T05:44:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98208595700000001, 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965998+0000", 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965998+0000", 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.965998+0000", 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966007+0000", 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9409999999999996e-06 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966014+0000", 2024-09-19T05:44:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2990000000000002e-06 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966019+0000", 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2580000000000003e-06 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2100 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.242615+0000", 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.69090721499999996, 2024-09-19T05:44:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70551545599999999, 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242615+0000", 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.242615+0000", 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.257284+0000", 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014669068 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.257318+0000", 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3624000000000002e-05 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.257352+0000", 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4051000000000003e-05 2024-09-19T05:44:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.257375+0000", 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2526000000000001e-05 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2155 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466546+0000", 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46697642299999997, 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.481629429, 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466546+0000", 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466546+0000", 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466547+0000", 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7e-07 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466550+0000", 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7749999999999999e-06 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466560+0000", 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6840000000000008e-06 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466566+0000", 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3099999999999997e-06 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2171 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.499095+0000", 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43442777900000001, 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44911445, 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499095+0000", 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499095+0000", 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499095+0000", 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4499999999999997e-07 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499112+0000", 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.683e-05 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499121+0000", 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5979999999999997e-06 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499126+0000", 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.87e-06 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2215 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721838+0000", 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21168421500000001, 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22640201500000001, 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721838+0000", 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721838+0000", 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721839+0000", 2024-09-19T05:44:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7899999999999999e-07 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721854+0000", 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5248999999999999e-05 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721861+0000", 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7349999999999999e-06 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721867+0000", 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5899999999999996e-06 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2228 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802783+0000", 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.13073901399999999, 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.145495702, 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802783+0000", 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802783+0000", 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802784+0000", 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1800000000000001e-07 2024-09-19T05:44:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802792+0000", 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2120000000000003e-06 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802800+0000", 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8890000000000002e-06 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802806+0000", 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1279999999999996e-06 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2239 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.099193326999999998, 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113985405, 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851429+0000", 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017099436999999999 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851444+0000", 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5449000000000001e-05 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851460+0000", 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.556e-05 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851470+0000", 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0001e-05 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2016 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.911423+0000", 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0220998699999999, 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.036917053, 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911423+0000", 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911423+0000", 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911423+0000", 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9000000000000002e-07 2024-09-19T05:44:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911432+0000", 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9469999999999992e-06 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911438+0000", 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911443+0000", 2024-09-19T05:44:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2023 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966025+0000", 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96749729100000004, 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98233870300000004, 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966025+0000", 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966025+0000", 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966026+0000", 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3999999999999997e-07 2024-09-19T05:44:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966029+0000", 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2660000000000002e-06 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966035+0000", 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.224e-06 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966041+0000", 2024-09-19T05:44:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7049999999999996e-06 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2101 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.257436+0000", 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.67608624500000003, 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.69095438799999997, 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.257436+0000", 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.257436+0000", 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266103+0000", 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0086662860000000005 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266126+0000", 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3045e-05 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266144+0000", 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7961999999999998e-05 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266152+0000", 2024-09-19T05:44:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8489999999999995e-06 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2156 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466576+0000", 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46694681900000001, 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48184221399999999, 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466576+0000", 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466576+0000", 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466576+0000", 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1099999999999995e-07 2024-09-19T05:44:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466605+0000", 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8804000000000001e-05 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466614+0000", 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5590000000000006e-06 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466620+0000", 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4239999999999998e-06 2024-09-19T05:44:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2172 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.499135+0000", 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.434387933, 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44930955500000003, 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499135+0000", 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499135+0000", 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499135+0000", 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3599999999999999e-07 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499139+0000", 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2150000000000003e-06 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499146+0000", 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5080000000000002e-06 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499151+0000", 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2216 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721875+0000", 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21164738599999999, 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.226593816, 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721875+0000", 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721875+0000", 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721876+0000", 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.06e-07 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721879+0000", 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1530000000000001e-06 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721885+0000", 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0759999999999999e-06 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721891+0000", 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0419999999999999e-06 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2229 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802815+0000", 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.130707027, 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14568530800000001, 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802815+0000", 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802815+0000", 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802816+0000", 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.58e-07 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802820+0000", 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802827+0000", 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1899999999999998e-06 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802833+0000", 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8560000000000003e-06 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4534.0:2212 3.20 e70/20)", 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.933654+0000", 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4294967295.9998684, 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014889615, 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.933654+0000", 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.933654+0000", 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.933656+0000", 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6419999999999999e-06 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.933665+0000", 2024-09-19T05:44:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9369999999999993e-06 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.933678+0000", 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2870999999999999e-05 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.933687+0000", 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7370000000000001e-06 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2017 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.911450+0000", 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.02207264, 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.03712012, 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911450+0000", 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911450+0000", 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911450+0000", 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6600000000000002e-07 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911470+0000", 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9831e-05 2024-09-19T05:44:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911477+0000", 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8129999999999998e-06 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.911482+0000", 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3469999999999998e-06 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2024 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966047+0000", 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96747574199999997, 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.982552753, 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966047+0000", 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966047+0000", 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966047+0000", 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966054+0000", 2024-09-19T05:44:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.092e-06 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966061+0000", 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5980000000000002e-06 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966066+0000", 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2490000000000001e-06 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2102 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266174+0000", 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66734859199999996, 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68244956800000001, 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266174+0000", 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266174+0000", 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266174+0000", 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7400000000000003e-07 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266179+0000", 2024-09-19T05:44:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5650000000000003e-06 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266187+0000", 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9100000000000005e-06 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266193+0000", 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3749999999999999e-06 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2157 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466630+0000", 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46689291399999999, 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48202984999999998, 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466630+0000", 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466630+0000", 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466630+0000", 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9399999999999995e-07 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466639+0000", 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6689999999999995e-06 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466646+0000", 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2119999999999997e-06 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466652+0000", 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7989999999999999e-06 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2173 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.499160+0000", 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43436287400000001, 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.44954953800000003, 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499160+0000", 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499160+0000", 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499160+0000", 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9799999999999999e-07 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499182+0000", 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2062999999999999e-05 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499190+0000", 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4069999999999993e-06 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499197+0000", 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8190000000000003e-06 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2181 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.540426+0000", 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.39309607400000002, 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.408342123, 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540426+0000", 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.540426+0000", 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.550030+0000", 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0096032510000000001 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.550082+0000", 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2816999999999999e-05 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.550134+0000", 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1374e-05 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.550156+0000", 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2541999999999999e-05 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2217 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721898+0000", 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21162412999999999, 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22692273900000001, 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721898+0000", 2024-09-19T05:44:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721898+0000", 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721899+0000", 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6700000000000003e-07 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721908+0000", 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6289999999999999e-06 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721914+0000", 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5740000000000001e-06 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721920+0000", 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8359999999999997e-06 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2230 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802841+0000", 2024-09-19T05:44:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.13068148600000001, 2024-09-19T05:44:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14603467000000001, 2024-09-19T05:44:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802841+0000", 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802841+0000", 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802842+0000", 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8899999999999999e-07 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802854+0000", 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2692e-05 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802862+0000", 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9899999999999997e-06 2024-09-19T05:44:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802869+0000", 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.409e-06 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2250 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.023636846999999999, 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039047541999999998, 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939724+0000", 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029838587 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939743+0000", 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9077e-05 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941640+0000", 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018968310000000001 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941651+0000", 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0379e-05 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2025 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966072+0000", 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96745015199999995, 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98291979399999996, 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966072+0000", 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966072+0000", 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966073+0000", 2024-09-19T05:44:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9000000000000002e-07 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966076+0000", 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4510000000000001e-06 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966082+0000", 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1290000000000001e-06 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966087+0000", 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9749999999999998e-06 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:1966 3.3c 3.ec74c13c (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.005797+0000", 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.92772560000000004, 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.94324935700000001, 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1966 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.005797+0000", 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.005797+0000", 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.013281+0000", 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0074839540000000001 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.017203+0000", 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039223319999999997 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.017224+0000", 2024-09-19T05:44:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0701999999999999e-05 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.017237+0000", 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.342e-05 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2103 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266203+0000", 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66731975099999996, 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68291693200000003, 2024-09-19T05:44:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266203+0000", 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266203+0000", 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266203+0000", 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4900000000000001e-07 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266213+0000", 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0006999999999999e-05 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266221+0000", 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3730000000000004e-06 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266226+0000", 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4029999999999998e-06 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2158 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466733+0000", 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46678932899999998, 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48244906300000001, 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466733+0000", 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466733+0000", 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466734+0000", 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.82e-07 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466739+0000", 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9640000000000002e-06 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466748+0000", 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6840000000000008e-06 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466755+0000", 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6850000000000004e-06 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2174 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.499207+0000", 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43431504100000001, 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45003058000000001, 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499207+0000", 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499207+0000", 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499208+0000", 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8899999999999998e-07 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499220+0000", 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1409999999999999e-05 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499227+0000", 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1459999999999999e-06 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499233+0000", 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3090000000000001e-06 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2218 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21159563200000001, 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22737317700000001, 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4099999999999999e-07 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721936+0000", 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4800000000000001e-06 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721941+0000", 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6300000000000003e-06 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721946+0000", 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2320000000000003e-06 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2231 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802878+0000", 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.13064494900000001, 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.146479784, 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802878+0000", 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802878+0000", 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802878+0000", 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9499999999999998e-07 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802882+0000", 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.053e-06 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802889+0000", 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0450000000000003e-06 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802895+0000", 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9000000000000003e-06 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:2289 3.14 e70/20 3:2b493931:::100000001f8.000004ee:head v 70'52, mlcod=70'49)", 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.850994+0000", 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.082528183000000005, 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.098428106000000001, 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.850994+0000", 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.850994+0000", 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.850996+0000", 2024-09-19T05:44:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.593e-06 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.854052+0000", 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030563019999999999 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.854081+0000", 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9023e-05 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.854096+0000", 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4825e-05 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.872079+0000", 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017982925 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.872104+0000", 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5221000000000001e-05 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2312 3.1e 3:7a2a8d26:::100000001f8.00000505:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.915954+0000", 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017568291, 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033549263000000003, 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2312 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915954+0000", 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.915954+0000", 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941646+0000", 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025691398000000001 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.945380+0000", 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037344420000000001 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.945398+0000", 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8077999999999999e-05 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.945413+0000", 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5233e-05 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.945552+0000", 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013832199999999999 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.946044+0000", 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00049202899999999995 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2007 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.863425+0000", 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0700976019999999, 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0861617749999999, 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863425+0000", 2024-09-19T05:44:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863425+0000", 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863425+0000", 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6199999999999998e-07 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863438+0000", 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2848e-05 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863446+0000", 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5109999999999997e-06 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863452+0000", 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8220000000000003e-06 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2026 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966093+0000", 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96742953300000001, 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98355413999999997, 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966093+0000", 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966093+0000", 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966093+0000", 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966103+0000", 2024-09-19T05:44:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4720000000000001e-06 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966109+0000", 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4779999999999996e-06 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966114+0000", 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1279999999999999e-06 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2056 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.035400+0000", 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.89812227300000003, 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.91430260100000005, 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.035400+0000", 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.035400+0000", 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043277+0000", 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.007877294 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043296+0000", 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8238999999999999e-05 2024-09-19T05:44:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043314+0000", 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7958e-05 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043326+0000", 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2237e-05 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2092 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.193106+0000", 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.74041623899999998, 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75665803399999998, 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.193106+0000", 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.193106+0000", 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211593+0000", 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018486875 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211613+0000", 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.003e-05 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211626+0000", 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2914e-05 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211634+0000", 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.439e-06 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2104 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266234+0000", 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66728882, 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68358380500000004, 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266234+0000", 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266234+0000", 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266234+0000", 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4900000000000001e-07 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266250+0000", 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6345999999999999e-05 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266257+0000", 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6239999999999996e-06 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266262+0000", 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0150000000000003e-06 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2159 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466767+0000", 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46675542800000003, 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48310367300000001, 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466767+0000", 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466767+0000", 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466768+0000", 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.58e-07 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466779+0000", 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1728e-05 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466788+0000", 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3829999999999992e-06 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466793+0000", 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2175 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.499242+0000", 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43428090699999999, 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45068654200000002, 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499242+0000", 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499242+0000", 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499242+0000", 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9400000000000001e-07 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499245+0000", 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2909999999999999e-06 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499256+0000", 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0688e-05 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.499272+0000", 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6248000000000001e-05 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2232 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.130619017, 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14708309999999999, 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802904+0000", 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-07 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802912+0000", 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9489999999999996e-06 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802920+0000", 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5970000000000003e-06 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802925+0000", 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5040000000000002e-06 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2196 3.20 3:06e4cd98:::100000001f8.00000491:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.809069+0000", 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.12445300099999999, 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14097616099999999, 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2196 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.809069+0000", 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.809069+0000", 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.852473+0000", 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043403394999999997 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855068+0000", 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025947190000000001 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855090+0000", 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2367000000000001e-05 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855104+0000", 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4260999999999999e-05 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855411+0000", 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030672400000000003 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855598+0000", 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000187247 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855782+0000", 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018349800000000001 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.932407+0000", 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076624831000000004 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2008 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.863468+0000", 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0700543570000001, 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.086667074, 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863468+0000", 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863468+0000", 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863469+0000", 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6899999999999998e-07 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863472+0000", 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7850000000000002e-06 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863479+0000", 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7830000000000001e-06 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863484+0000", 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2209999999999996e-06 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2027 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966121+0000", 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96740145300000002, 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98407193900000001, 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966121+0000", 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966121+0000", 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966121+0000", 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2899999999999999e-07 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966124+0000", 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8619999999999999e-06 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966130+0000", 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966135+0000", 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8380000000000001e-06 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2042 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.978563+0000", 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.95495929499999999, 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.97168448799999996, 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.978563+0000", 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.978563+0000", 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021770+0000", 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043206736000000003 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021783+0000", 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2728000000000001e-05 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021797+0000", 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4449e-05 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021807+0000", 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8600000000000005e-06 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2057 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.043661+0000", 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.88986178900000001, 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90665715000000002, 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043661+0000", 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043661+0000", 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043662+0000", 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6099999999999999e-07 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043674+0000", 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2075999999999999e-05 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043685+0000", 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1301000000000001e-05 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043693+0000", 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5469999999999999e-06 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2082 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.099325+0000", 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.83419733699999998, 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.85104924999999998, 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099325+0000", 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.099325+0000", 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156185+0000", 2024-09-19T05:44:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056859536000000002 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156198+0000", 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3241000000000001e-05 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156207+0000", 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5240000000000006e-06 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156214+0000", 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2093 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.211650+0000", 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72187213800000005, 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73885235999999999, 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211650+0000", 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211650+0000", 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211651+0000", 2024-09-19T05:44:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0999999999999998e-07 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211657+0000", 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1850000000000001e-06 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211665+0000", 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5680000000000002e-06 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211671+0000", 2024-09-19T05:44:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2105 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266278+0000", 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.66724443700000002, 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68436145100000001, 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266278+0000", 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266278+0000", 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266278+0000", 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.32e-07 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266282+0000", 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7100000000000001e-06 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266289+0000", 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3690000000000003e-06 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266294+0000", 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.972e-06 2024-09-19T05:44:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2131 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.334556+0000", 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.59896683500000003, 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61645832099999998, 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334556+0000", 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.334556+0000", 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374124+0000", 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039568427000000003 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374136+0000", 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1868000000000001e-05 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374154+0000", 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7884e-05 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374166+0000", 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1945e-05 2024-09-19T05:44:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2145 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.429281+0000", 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.50424174899999996, 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52266550199999995, 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429281+0000", 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.429281+0000", 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.442362+0000", 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013081124 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.442375+0000", 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3338e-05 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.442384+0000", 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7730000000000005e-06 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.442390+0000", 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8950000000000003e-06 2024-09-19T05:44:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2160 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466802+0000", 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.466720354, 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48519001, 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466802+0000", 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466802+0000", 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466803+0000", 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.39e-07 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466806+0000", 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6830000000000001e-06 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466813+0000", 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9709999999999998e-06 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466829+0000", 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6036e-05 2024-09-19T05:44:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2182 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.550192+0000", 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.38333071099999999, 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.40184963000000001, 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.550192+0000", 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.550192+0000", 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562531+0000", 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012339091999999999 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562561+0000", 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0465999999999999e-05 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562571+0000", 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4890000000000001e-06 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.562582+0000", 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1216000000000001e-05 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2219 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21157081799999999, 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23013545799999999, 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721960+0000", 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3760000000000008e-06 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721967+0000", 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1469999999999998e-06 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721971+0000", 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2009 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.863492+0000", 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.070030109, 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.088638298, 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863492+0000", 2024-09-19T05:44:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863492+0000", 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863493+0000", 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.58e-07 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863502+0000", 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8389999999999996e-06 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863508+0000", 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3820000000000001e-06 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.863513+0000", 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6820000000000002e-06 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2028 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:29.966141+0000", 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.96738143899999995, 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.98603660699999995, 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966141+0000", 2024-09-19T05:44:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966141+0000", 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966141+0000", 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966149+0000", 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2139999999999999e-06 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966155+0000", 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:29.966159+0000", 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3970000000000004e-06 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2043 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.021828+0000", 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.91169404600000004, 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93039236599999997, 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021828+0000", 2024-09-19T05:44:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021828+0000", 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021829+0000", 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8700000000000005e-07 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021843+0000", 2024-09-19T05:44:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3638999999999999e-05 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021851+0000", 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7970000000000001e-06 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.021857+0000", 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2639999999999997e-06 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2058 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.043704+0000", 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.88981862, 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90857439100000004, 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043704+0000", 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043704+0000", 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043704+0000", 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5300000000000004e-07 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043729+0000", 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4766e-05 2024-09-19T05:44:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043740+0000", 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0549e-05 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.043748+0000", 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0320000000000003e-06 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2083 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.156228+0000", 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.77729419300000002, 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.79609685399999996, 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156228+0000", 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156228+0000", 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156229+0000", 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9800000000000004e-07 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156233+0000", 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9530000000000001e-06 2024-09-19T05:44:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156239+0000", 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5710000000000003e-06 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.156246+0000", 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.764e-06 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2094 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.211679+0000", 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72184347000000004, 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.74069749900000004, 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211679+0000", 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211679+0000", 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211679+0000", 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6699999999999999e-07 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211689+0000", 2024-09-19T05:44:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1519999999999998e-06 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211696+0000", 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4070000000000004e-06 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.211702+0000", 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6380000000000001e-06 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2106 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.266300+0000", 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.667222867, 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68612247699999995, 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266300+0000", 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266300+0000", 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266300+0000", 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8299999999999998e-07 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266323+0000", 2024-09-19T05:44:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2543000000000001e-05 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266330+0000", 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2060000000000001e-06 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.266335+0000", 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3900000000000001e-06 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2132 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.374197+0000", 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55932543700000004, 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.578271863, 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374197+0000", 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374197+0000", 2024-09-19T05:44:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374198+0000", 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.37e-07 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374209+0000", 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1338999999999999e-05 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374216+0000", 2024-09-19T05:44:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4989999999999999e-06 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.374222+0000", 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.032e-06 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2161 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.466838+0000", 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.46668437400000001, 2024-09-19T05:44:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48568254999999999, 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466838+0000", 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466838+0000", 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466839+0000", 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0200000000000002e-07 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466859+0000", 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9797999999999999e-05 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466867+0000", 2024-09-19T05:44:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.541e-06 2024-09-19T05:44:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.466872+0000", 2024-09-19T05:44:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1329999999999998e-06 2024-09-19T05:44:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2220 2.14 2.6e5f474 (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21154585300000001, 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23059406199999999, 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7300000000000002e-07 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721988+0000", 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1056999999999999e-05 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721994+0000", 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1310000000000003e-06 2024-09-19T05:44:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.722000+0000", 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7910000000000001e-06 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2212 3.20 3:046cacbc:::100000001f8.000004a1:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.855141+0000", 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.078381862999999996, 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.097490729999999998, 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2212 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855141+0000", 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.855141+0000", 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.858671+0000", 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00353053 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.861235+0000", 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025642920000000001 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.861271+0000", 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5250999999999998e-05 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.861288+0000", 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7048e-05 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.904733+0000", 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043445029000000003 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.904899+0000", 2024-09-19T05:44:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016666700000000001 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.905072+0000", 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017211200000000001 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 266 2024-09-19T05:44:31.787 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:31.788 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:31.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.18.5 2024-09-19T05:44:31.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.182+0000 7fd656039640 1 Processor -- start 2024-09-19T05:44:31.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.182+0000 7fd656039640 1 -- start start 2024-09-19T05:44:31.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.184+0000 7fd656039640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd65006ebe0 0x7fd6500c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.184+0000 7fd656039640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6500c86a0 0x7fd650071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.184+0000 7fd656039640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd650071f50 con 0x7fd65006fd60 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.184+0000 7fd656039640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd650157410 con 0x7fd65006ebe0 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.184+0000 7fd656039640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd6500c7430 con 0x7fd6500c86a0 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd655838640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd65006fd60 0x7fd65006efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:38796/0 (socket says 172.21.15.2:38796) 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd655838640 1 -- 172.21.15.2:0/332386625 learned_addr learned my addr 172.21.15.2:0/332386625 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd654836640 1 --2- 172.21.15.2:0/332386625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6500c86a0 0x7fd650071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd655037640 1 --2- 172.21.15.2:0/332386625 >> v2:172.21.15.123:3300/0 conn(0x7fd65006ebe0 0x7fd6500c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd654836640 1 -- 172.21.15.2:0/332386625 >> v2:172.21.15.123:3300/0 conn(0x7fd65006ebe0 msgr2=0x7fd6500c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd654836640 1 --2- 172.21.15.2:0/332386625 >> v2:172.21.15.123:3300/0 conn(0x7fd65006ebe0 0x7fd6500c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd654836640 1 -- 172.21.15.2:0/332386625 >> v1:172.21.15.123:6789/0 conn(0x7fd65006fd60 legacy=0x7fd65006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.185+0000 7fd654836640 1 -- 172.21.15.2:0/332386625 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd650157920 con 0x7fd6500c86a0 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.188+0000 7fd654836640 1 --2- 172.21.15.2:0/332386625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6500c86a0 0x7fd650071a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd648009d90 tx=0x7fd64802a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d732e946c21a30c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.189+0000 7fd647fff640 1 -- 172.21.15.2:0/332386625 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648007740 con 0x7fd6500c86a0 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.189+0000 7fd647fff640 1 -- 172.21.15.2:0/332386625 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6480078e0 con 0x7fd6500c86a0 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.189+0000 7fd647fff640 1 -- 172.21.15.2:0/332386625 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648007cf0 con 0x7fd6500c86a0 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.190+0000 7fd656039640 1 -- 172.21.15.2:0/332386625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6500c86a0 msgr2=0x7fd650071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.190+0000 7fd656039640 1 --2- 172.21.15.2:0/332386625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6500c86a0 0x7fd650071a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd648009d90 tx=0x7fd64802a6a0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.193+0000 7fd656039640 1 -- 172.21.15.2:0/332386625 shutdown_connections 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.193+0000 7fd656039640 1 --2- 172.21.15.2:0/332386625 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6500c86a0 0x7fd650071a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.193+0000 7fd656039640 1 --2- 172.21.15.2:0/332386625 >> v2:172.21.15.123:3300/0 conn(0x7fd65006ebe0 0x7fd6500c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.193+0000 7fd656039640 1 -- 172.21.15.2:0/332386625 >> 172.21.15.2:0/332386625 conn(0x7fd6500cc0d0 msgr2=0x7fd6500cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.196+0000 7fd656039640 1 -- 172.21.15.2:0/332386625 shutdown_connections 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.196+0000 7fd656039640 1 -- 172.21.15.2:0/332386625 wait complete. 2024-09-19T05:44:31.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.198+0000 7fd656039640 1 Processor -- start 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.198+0000 7fd656039640 1 -- start start 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd656039640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd65006ebe0 0x7fd65014b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd656039640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd65006fd60 0x7fd650141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd656039640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6500c86a0 0x7fd65014bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd656039640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd6500c6b00 con 0x7fd65006ebe0 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd656039640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd65006f6d0 con 0x7fd6500c86a0 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd656039640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd6500c6de0 con 0x7fd65006fd60 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd655838640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6500c86a0 0x7fd65014bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd655838640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6500c86a0 0x7fd65014bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:56398/0 (socket says 172.21.15.2:56398) 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.200+0000 7fd655838640 1 -- 172.21.15.2:0/2787369737 learned_addr learned my addr 172.21.15.2:0/2787369737 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.201+0000 7fd655838640 1 -- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd65006fd60 msgr2=0x7fd650141ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.201+0000 7fd655838640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd65006fd60 0x7fd650141ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.201+0000 7fd655838640 1 -- 172.21.15.2:0/2787369737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd65006ebe0 msgr2=0x7fd65014b5d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:44:31.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.201+0000 7fd655838640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd65006ebe0 0x7fd65014b5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.201+0000 7fd655838640 1 -- 172.21.15.2:0/2787369737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6501f4b10 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.201+0000 7fd655838640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6500c86a0 0x7fd65014bfd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b4f0 tx=0x7fd64000b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.201+0000 7fd645ffb640 1 -- 172.21.15.2:0/2787369737 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640014070 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.202+0000 7fd656039640 1 -- 172.21.15.2:0/2787369737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65013e930 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.202+0000 7fd656039640 1 -- 172.21.15.2:0/2787369737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65013eef0 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.204+0000 7fd656039640 1 -- 172.21.15.2:0/2787369737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65013f0f0 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.206+0000 7fd645ffb640 1 -- 172.21.15.2:0/2787369737 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd640009040 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.206+0000 7fd645ffb640 1 -- 172.21.15.2:0/2787369737 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64000f850 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.209+0000 7fd645ffb640 1 -- 172.21.15.2:0/2787369737 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd640017070 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.210+0000 7fd645ffb640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd63406c380 0x7fd63406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.211+0000 7fd655037640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd63406c380 0x7fd63406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.211+0000 7fd645ffb640 1 -- 172.21.15.2:0/2787369737 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5799+0+0 (secure 0 0 0) 0x7fd64008b830 con 0x7fd6500c86a0 2024-09-19T05:44:31.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.211+0000 7fd655037640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd63406c380 0x7fd63406e840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:31.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.214+0000 7fd645ffb640 1 -- 172.21.15.2:0/2787369737 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd640059f20 con 0x7fd6500c86a0 2024-09-19T05:44:31.799 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:31.417+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:31.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.797+0000 7fd656039640 1 -- 172.21.15.2:0/2787369737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fd6500c70c0 con 0x7fd6500c86a0 2024-09-19T05:44:31.800 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:31.791+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:31.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.866+0000 7fd645ffb640 1 -- 172.21.15.2:0/2787369737 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v71) ==== 91+0+0 (secure 0 0 0) 0x7fd640059820 con 0x7fd6500c86a0 2024-09-19T05:44:31.869 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:44:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.873+0000 7fd63b7fe640 1 -- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd63406c380 msgr2=0x7fd63406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:31.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.873+0000 7fd63b7fe640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd63406c380 0x7fd63406e840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd64c006fd0 tx=0x7fd64c006a20 comp rx=0 tx=0).stop 2024-09-19T05:44:31.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.873+0000 7fd63b7fe640 1 -- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6500c86a0 msgr2=0x7fd65014bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:31.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.873+0000 7fd63b7fe640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6500c86a0 0x7fd65014bfd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b4f0 tx=0x7fd64000b9b0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.874+0000 7fd63b7fe640 1 -- 172.21.15.2:0/2787369737 shutdown_connections 2024-09-19T05:44:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.874+0000 7fd63b7fe640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd63406c380 0x7fd63406e840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.874+0000 7fd63b7fe640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6500c86a0 0x7fd65014bfd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.874+0000 7fd63b7fe640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd65006fd60 0x7fd650141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.874+0000 7fd63b7fe640 1 --2- 172.21.15.2:0/2787369737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd65006ebe0 0x7fd65014b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:31.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.874+0000 7fd63b7fe640 1 -- 172.21.15.2:0/2787369737 >> 172.21.15.2:0/2787369737 conn(0x7fd6500cc0d0 msgr2=0x7fd650143760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:31.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.875+0000 7fd63b7fe640 1 -- 172.21.15.2:0/2787369737 shutdown_connections 2024-09-19T05:44:31.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:31.875+0000 7fd63b7fe640 1 -- 172.21.15.2:0/2787369737 wait complete. 2024-09-19T05:44:31.894 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:31.889+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:31.992 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:31.989+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:32.122 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:32.099+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:32.204 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:32.198+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:32.215 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:32.216 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:32.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:32.328 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:32.322+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:32.396 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:32.393+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:32.438 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:32.500 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:32.498+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:32.597 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:32.595+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2140 3.3c 3:3ebd280f:::100000001f8.00000459:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.0899024179999999, 2024-09-19T05:44:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0038021129999999, 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2140 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641694+0000", 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049504800000000002 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646652+0000", 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049580040000000002 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646670+0000", 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646681+0000", 2024-09-19T05:44:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1555e-05 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429025+0000", 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.782343268 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429156+0000", 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013093299999999999 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.435554+0000", 2024-09-19T05:44:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063981860000000002 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549237+0000", 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113683458 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595633+0000", 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046396021000000003 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595950+0000", 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000316514 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595991+0000", 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1430000000000001e-05 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2198 2.14 2:2e2fa760:::200.00000001:head [write 2326139~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.0401908519999998, 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9519298269999998, 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641901+0000", 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.03e-07 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641909+0000", 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5660000000000001e-06 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641917+0000", 2024-09-19T05:44:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9680000000000006e-06 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641923+0000", 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6189999999999996e-06 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.374211+0000", 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7322879150000001 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.374327+0000", 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000115739 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.374472+0000", 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145117 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.475244+0000", 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.100771759 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593639+0000", 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11839486 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593701+0000", 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2432000000000001e-05 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593830+0000", 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012944899999999999 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2199 2.14 2:2e2fa760:::200.00000001:head [write 2328219~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.0401528569999998, 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.951978939, 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641939+0000", 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7199999999999999e-07 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641947+0000", 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6820000000000003e-06 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641954+0000", 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8990000000000004e-06 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641959+0000", 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9779999999999996e-06 2024-09-19T05:44:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.427591+0000", 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.785632256 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.427691+0000", 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9714000000000001e-05 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.427886+0000", 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000194785 2024-09-19T05:44:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.495986+0000", 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068100588000000004 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593839+0000", 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.097852407000000002 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593884+0000", 2024-09-19T05:44:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5476999999999998e-05 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593918+0000", 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3581000000000003e-05 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2200 2.14 2:2e2fa760:::200.00000001:head [write 2330299~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.641968+0000", 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.0401238899999998, 2024-09-19T05:44:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.952084597, 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641968+0000", 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641968+0000", 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683492+0000", 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041524221 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683504+0000", 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2296000000000001e-05 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683517+0000", 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2661e-05 2024-09-19T05:44:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683532+0000", 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4932999999999999e-05 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429218+0000", 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.745686351 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429318+0000", 2024-09-19T05:44:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9476999999999996e-05 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.433587+0000", 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042698240000000002 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.496028+0000", 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.062440233999999997 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593927+0000", 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.097899775999999994 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.593966+0000", 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8668999999999999e-05 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594052+0000", 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6155000000000004e-05 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2201 2.14 2:2e2fa760:::200.00000001:head [write 2332379~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683559+0000", 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.998532363, 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9106536570000001, 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683559+0000", 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683559+0000", 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683560+0000", 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4499999999999997e-07 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683572+0000", 2024-09-19T05:44:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2097e-05 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683579+0000", 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1890000000000001e-06 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683585+0000", 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6509999999999998e-06 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.454583+0000", 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.770998209 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.454762+0000", 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017960299999999999 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.454923+0000", 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016077100000000001 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549115+0000", 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.094191509000000007 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594061+0000", 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044945855 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594130+0000", 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9111999999999998e-05 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594213+0000", 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3115999999999995e-05 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2202 2.14 2:2e2fa760:::200.00000001:head [write 2334459~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.998498782, 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9113874819999999, 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683593+0000", 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4099999999999999e-07 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683597+0000", 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7699999999999999e-06 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683603+0000", 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6729999999999997e-06 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683607+0000", 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6160000000000004e-06 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.455944+0000", 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7723369500000001 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.456039+0000", 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4911e-05 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.456174+0000", 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013522299999999999 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549154+0000", 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092979694000000002 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594220+0000", 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045066348999999999 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594443+0000", 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022309200000000001 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594980+0000", 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000536763 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2203 2.14 2:2e2fa760:::200.00000001:head [write 2336539~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.998477989, 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9114671510000001, 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683614+0000", 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683633+0000", 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9006999999999999e-05 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683640+0000", 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683648+0000", 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4449999999999996e-06 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.458415+0000", 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7747667919999999 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459139+0000", 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072384999999999995 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459255+0000", 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000116623 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549197+0000", 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.089941199999999999 2024-09-19T05:44:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.594988+0000", 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045791599000000002 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595041+0000", 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2757000000000003e-05 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595081+0000", 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9755000000000001e-05 2024-09-19T05:44:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2204 2.14 2:2e2fa760:::200.00000001:head [write 2338619~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683655+0000", 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.9984360859999999, 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9116514599999999, 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683655+0000", 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683655+0000", 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683656+0000", 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2300000000000002e-07 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683659+0000", 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4070000000000002e-06 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683665+0000", 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5609999999999998e-06 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683670+0000", 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.78e-06 2024-09-19T05:44:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.475287+0000", 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.791617797 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.475398+0000", 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000110288 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.475585+0000", 2024-09-19T05:44:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018697599999999999 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549286+0000", 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073701254999999993 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595163+0000", 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045876685 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595248+0000", 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5588999999999997e-05 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595307+0000", 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8699000000000002e-05 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2205 2.14 2:2e2fa760:::200.00000001:head [write 2340699~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.998416454, 2024-09-19T05:44:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.911749621, 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683675+0000", 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1699999999999999e-07 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683682+0000", 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9249999999999998e-06 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683688+0000", 2024-09-19T05:44:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6579999999999999e-06 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683692+0000", 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5079999999999999e-06 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.496991+0000", 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.813298211 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.497225+0000", 2024-09-19T05:44:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023393599999999999 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.497520+0000", 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000295612 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549339+0000", 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051818528000000003 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595316+0000", 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045976969999999999 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595375+0000", 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9265000000000003e-05 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595425+0000", 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9691000000000002e-05 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2206 2.14 2:2e2fa760:::200.00000001:head [write 2342779~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683698+0000", 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.998393197, 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.911830396, 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683698+0000", 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683698+0000", 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683699+0000", 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3000000000000002e-07 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683705+0000", 2024-09-19T05:44:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6760000000000001e-06 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683723+0000", 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7589e-05 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683728+0000", 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0379999999999999e-06 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.499256+0000", 2024-09-19T05:44:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8155281699999999 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.499367+0000", 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111128 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.499571+0000", 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204045 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549387+0000", 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049815832999999997 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595432+0000", 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046045057 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595491+0000", 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8666999999999999e-05 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595529+0000", 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7863e-05 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2207 2.14 2:2e2fa760:::200.00000001:head [write 2344859~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683735+0000", 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.998356392, 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9118896840000001, 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683735+0000", 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683735+0000", 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683736+0000", 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0900000000000002e-07 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683739+0000", 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5860000000000001e-06 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683745+0000", 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.694e-06 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683750+0000", 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2549999999999997e-06 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.503451+0000", 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.819700799 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.503549+0000", 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7696999999999997e-05 2024-09-19T05:44:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.503709+0000", 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016090599999999999 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549427+0000", 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045717704999999997 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595536+0000", 2024-09-19T05:44:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046108364999999998 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595576+0000", 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0448999999999998e-05 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595625+0000", 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8819000000000001e-05 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2208 2.14 2:2e2fa760:::200.00000001:head [write 2346939~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.9983358070000001, 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.912371281, 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683756+0000", 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9999999999999999e-07 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683773+0000", 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6860000000000001e-05 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683779+0000", 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3249999999999996e-06 2024-09-19T05:44:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683784+0000", 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9660000000000004e-06 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.504953+0000", 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8211683839999999 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.505044+0000", 2024-09-19T05:44:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0997999999999998e-05 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.505174+0000", 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013062699999999999 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549505+0000", 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044330983999999997 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.596002+0000", 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046496559 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.596080+0000", 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8485999999999994e-05 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.596127+0000", 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6792000000000002e-05 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2209 2.14 2:2e2fa760:::200.00000001:head [write 2349019~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.683791+0000", 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.998300433, 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9124404310000001, 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683791+0000", 2024-09-19T05:44:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.683791+0000", 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721534+0000", 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037743092999999998 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721554+0000", 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0052e-05 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721595+0000", 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0649000000000003e-05 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721619+0000", 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3777e-05 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.506128+0000", 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7845095560000002 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.506219+0000", 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0513000000000006e-05 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.506355+0000", 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013663999999999999 2024-09-19T05:44:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549589+0000", 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043233764000000001 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.596134+0000", 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046544901 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.596196+0000", 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2169999999999996e-05 2024-09-19T05:44:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.596232+0000", 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5315999999999999e-05 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2217 2.14 2:2e2fa760:::200.00000001:head [write 2365659~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721898+0000", 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.9601931690000001, 2024-09-19T05:44:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.902679191, 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721898+0000", 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721898+0000", 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721899+0000", 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6700000000000003e-07 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721908+0000", 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6289999999999999e-06 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721914+0000", 2024-09-19T05:44:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5740000000000001e-06 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721920+0000", 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8359999999999997e-06 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.581935+0000", 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8600152890000001 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.582037+0000", 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101674 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.582209+0000", 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017197600000000001 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.597019+0000", 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014810076 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.624468+0000", 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027449121999999999 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.624525+0000", 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6920999999999999e-05 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.624578+0000", 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2527e-05 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2218 2.14 2:2e2fa760:::200.00000001:head [write 2367739~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.960164671, 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9040992079999999, 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721927+0000", 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4099999999999999e-07 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721936+0000", 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4800000000000001e-06 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721941+0000", 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6300000000000003e-06 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721946+0000", 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2320000000000003e-06 2024-09-19T05:44:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586390+0000", 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.864444134 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586476+0000", 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5921000000000006e-05 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.591549+0000", 2024-09-19T05:44:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050733740000000003 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.611941+0000", 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020391790999999999 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.625894+0000", 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013952884 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.625964+0000", 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9876999999999995e-05 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.626026+0000", 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2444e-05 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2219 2.14 2:2e2fa760:::200.00000001:head [write 2369819~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.9601398570000002, 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9291401420000001, 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721952+0000", 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7399999999999999e-07 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721960+0000", 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3760000000000008e-06 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721967+0000", 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1469999999999998e-06 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721971+0000", 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.597052+0000", 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8750810439999999 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.597206+0000", 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000154365 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.597398+0000", 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019206599999999999 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.611994+0000", 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014595861999999999 2024-09-19T05:44:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.650967+0000", 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038972551000000001 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651034+0000", 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7005e-05 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651092+0000", 2024-09-19T05:44:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7952999999999999e-05 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2220 2.14 2:2e2fa760:::200.00000001:head [write 2371899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.960114892, 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9292026259999999, 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721977+0000", 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7300000000000002e-07 2024-09-19T05:44:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721988+0000", 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1056999999999999e-05 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.721994+0000", 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1310000000000003e-06 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.722000+0000", 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7910000000000001e-06 2024-09-19T05:44:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.610241+0000", 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.888241289 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.610456+0000", 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000214446 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.610703+0000", 2024-09-19T05:44:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024688200000000003 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.622527+0000", 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011824612999999999 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651100+0000", 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028573221999999999 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651141+0000", 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0689000000000003e-05 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651179+0000", 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8133000000000002e-05 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2221 2.14 2:2e2fa760:::200.00000001:head [write 2373979~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.722007+0000", 2024-09-19T05:44:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.9600846000000001, 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.929274699, 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.722007+0000", 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.722007+0000", 2024-09-19T05:44:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732818+0000", 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010811368999999999 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732829+0000", 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1021e-05 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732847+0000", 2024-09-19T05:44:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7357e-05 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732858+0000", 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1234e-05 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.612025+0000", 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.879166825 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.612130+0000", 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010501 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.612292+0000", 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000161776 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.622574+0000", 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010282060000000001 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651185+0000", 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028611817000000001 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651225+0000", 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9564999999999998e-05 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651282+0000", 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6665e-05 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2222 2.14 2:2e2fa760:::200.00000001:head [write 2376059~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.732878+0000", 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.949213291, 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.918487587, 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732878+0000", 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.732878+0000", 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756284+0000", 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023406138999999999 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756314+0000", 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9529000000000001e-05 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756344+0000", 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9998999999999999e-05 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756364+0000", 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9956e-05 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.618583+0000", 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8622194649999999 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.620203+0000", 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001620051 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.620346+0000", 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014213799999999999 2024-09-19T05:44:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.650953+0000", 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030607321 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651289+0000", 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033624600000000001 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651333+0000", 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4313000000000001e-05 2024-09-19T05:44:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651366+0000", 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.243e-05 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2223 2.14 2:2e2fa760:::200.00000001:head [write 2378139~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.756404+0000", 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.9256872839999999, 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.903939713, 2024-09-19T05:44:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756404+0000", 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.756404+0000", 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769113+0000", 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01270903 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769143+0000", 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9624999999999998e-05 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769160+0000", 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7255e-05 2024-09-19T05:44:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.769173+0000", 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3079e-05 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.622625+0000", 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8534515900000001 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.622763+0000", 2024-09-19T05:44:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000137708 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.622921+0000", 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015850599999999999 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.651372+0000", 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028451071000000001 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.652613+0000", 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012412949999999999 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.660252+0000", 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076389240000000001 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.660344+0000", 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1630000000000002e-05 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:32.798 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:32.799 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.767+0000 7f9f4ffff640 1 Processor -- start 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.767+0000 7f9f4ffff640 1 -- start start 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.768+0000 7f9f4ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9f48045dd0 0x7f9f48035b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.768+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f480042b0 0x7f9f480461b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.768+0000 7f9f4ffff640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4804bec0 con 0x7f9f48036210 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.768+0000 7f9f4ffff640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f9f4804b000 con 0x7f9f48045dd0 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.768+0000 7f9f4ffff640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4804b2e0 con 0x7f9f480042b0 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.770+0000 7f9f4e7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f480042b0 0x7f9f480461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:32.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.770+0000 7f9f4f7fe640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9f48036210 0x7f9f480e2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:47688/0 (socket says 172.21.15.2:47688) 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.770+0000 7f9f4f7fe640 1 -- 172.21.15.2:0/2050145372 learned_addr learned my addr 172.21.15.2:0/2050145372 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.771+0000 7f9f4effd640 1 --2- 172.21.15.2:0/2050145372 >> v2:172.21.15.2:3300/0 conn(0x7f9f48045dd0 0x7f9f48035b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.773+0000 7f9f4effd640 1 -- 172.21.15.2:0/2050145372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f480042b0 msgr2=0x7f9f480461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.773+0000 7f9f4effd640 1 --2- 172.21.15.2:0/2050145372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f480042b0 0x7f9f480461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.773+0000 7f9f4effd640 1 -- 172.21.15.2:0/2050145372 >> v1:172.21.15.123:6789/0 conn(0x7f9f48036210 legacy=0x7f9f480e2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.773+0000 7f9f4effd640 1 -- 172.21.15.2:0/2050145372 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4804b870 con 0x7f9f48045dd0 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.775+0000 7f9f4effd640 1 --2- 172.21.15.2:0/2050145372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f48045dd0 0x7f9f48035b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f50051220 tx=0x7f9f500a6b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb43b8384914686 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.775+0000 7f9f4dffb640 1 -- 172.21.15.2:0/2050145372 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f500a8920 con 0x7f9f48045dd0 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.775+0000 7f9f4dffb640 1 -- 172.21.15.2:0/2050145372 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f500a8b00 con 0x7f9f48045dd0 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.776+0000 7f9f4ffff640 1 -- 172.21.15.2:0/2050145372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f48045dd0 msgr2=0x7f9f48035b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.776+0000 7f9f4ffff640 1 --2- 172.21.15.2:0/2050145372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f48045dd0 0x7f9f48035b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f50051220 tx=0x7f9f500a6b40 comp rx=0 tx=0).stop 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.781+0000 7f9f4ffff640 1 -- 172.21.15.2:0/2050145372 shutdown_connections 2024-09-19T05:44:32.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.781+0000 7f9f4ffff640 1 --2- 172.21.15.2:0/2050145372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f480042b0 0x7f9f480461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.781+0000 7f9f4ffff640 1 --2- 172.21.15.2:0/2050145372 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f48045dd0 0x7f9f48035b40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.781+0000 7f9f4ffff640 1 -- 172.21.15.2:0/2050145372 >> 172.21.15.2:0/2050145372 conn(0x7f9f480504e0 msgr2=0x7f9f480508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.790+0000 7f9f4ffff640 1 -- 172.21.15.2:0/2050145372 shutdown_connections 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.790+0000 7f9f4ffff640 1 -- 172.21.15.2:0/2050145372 wait complete. 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.792+0000 7f9f4ffff640 1 Processor -- start 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.792+0000 7f9f4ffff640 1 -- start start 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f480042b0 0x7f9f480e0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f48036210 0x7f9f480cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f48045dd0 0x7f9f48049c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4ffff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9f480e3480 con 0x7f9f480042b0 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4ffff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f480511b0 con 0x7f9f48045dd0 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4ffff640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9f480e3600 con 0x7f9f48036210 2024-09-19T05:44:32.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4e7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f48036210 0x7f9f480cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4e7fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f48036210 0x7f9f480cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:45454/0 (socket says 172.21.15.2:45454) 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4e7fc640 1 -- 172.21.15.2:0/3818745075 learned_addr learned my addr 172.21.15.2:0/3818745075 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.794+0000 7f9f4f7fe640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f48045dd0 0x7f9f48049c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.795+0000 7f9f4e7fc640 1 -- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f48045dd0 msgr2=0x7f9f48049c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.795+0000 7f9f4e7fc640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f48045dd0 0x7f9f48049c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.795+0000 7f9f4e7fc640 1 -- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f480042b0 msgr2=0x7f9f480e0f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.795+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f480042b0 0x7f9f480e0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.796+0000 7f9f4e7fc640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f480042b0 0x7f9f480e0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.796+0000 7f9f4e7fc640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f480e14c0 con 0x7f9f48036210 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.796+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f480042b0 0x7f9f480e0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:32.806 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.796+0000 7f9f4e7fc640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9f48036210 0x7f9f480cb8b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000c820 tx=0x7f9f4000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:32.807 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:32.797+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.797+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40015070 con 0x7f9f48036210 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.798+0000 7f9f4ffff640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f480e1a70 con 0x7f9f48036210 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.798+0000 7f9f4ffff640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f48049480 con 0x7f9f48036210 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.801+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f9f48035b40 con 0x7f9f48036210 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.801+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f40004220 con 0x7f9f48036210 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.801+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000ce10 con 0x7f9f48036210 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.802+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9f40007830 con 0x7f9f48036210 2024-09-19T05:44:32.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.804+0000 7f9f37fff640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f9f3c06c330 0x7f9f3c06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.808+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 5799+0+0 (secure 0 0 0) 0x7f9f4008b2a0 con 0x7f9f48036210 2024-09-19T05:44:32.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.808+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f9f3c06c330 0x7f9f3c06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:32.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.809+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v71) ==== 54+0+16 (secure 0 0 0) 0x7f9f4008d830 con 0x7f9f48036210 2024-09-19T05:44:32.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.809+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f9f3c06c330 0x7f9f3c06e7f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009fc0 tx=0x7f9f5006e030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:32.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.811+0000 7f9f4ffff640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f9f48049680 0x7f9f481967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:32.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:32.812+0000 7f9f4ffff640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f48045bb0 con 0x7f9f48049680 2024-09-19T05:44:32.900 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:32.898+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:33.005 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:33.000+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:33.103 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:33.099+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:33.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.142+0000 7f9f4f7fe640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f9f48049680 0x7f9f481967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:33.151 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3186 3.37 3.abb70277 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.838547+0000", 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.29317051799999999, 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29322351099999999, 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3186 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838547+0000", 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838547+0000", 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838550+0000", 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.914e-06 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.842958+0000", 2024-09-19T05:44:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00440822 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.842982+0000", 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3570999999999998e-05 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.842997+0000", 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5194999999999999e-05 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3295 3.11 3:895c8f5c:::100000001f8.000008dc:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.990687+0000", 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.14103112900000001, 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14120279199999999, 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3295 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.990687+0000", 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.990687+0000", 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.990689+0000", 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6790000000000001e-06 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.993761+0000", 2024-09-19T05:44:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003071528 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.993780+0000", 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.878e-05 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.993790+0000", 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9089999999999995e-06 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.993887+0000", 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7887000000000007e-05 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994048+0000", 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016076900000000001 2024-09-19T05:44:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994216+0000", 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016772999999999999 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.029767+0000", 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035551435999999999 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3389 3.a 3.7d557b8a (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.088460+0000", 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.043257891999999999, 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043544616000000001, 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3389 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.088460+0000", 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.088460+0000", 2024-09-19T05:44:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.088461+0000", 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5379999999999999e-06 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.089751+0000", 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012894670000000001 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.089766+0000", 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4837999999999999e-05 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.089777+0000", 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0713999999999999e-05 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2877 3.9 3:91dc4dd5:::100000001f8.0000073a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.79946070800000002, 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.79982571599999996, 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2877 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332260+0000", 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9689999999999999e-06 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337127+0000", 2024-09-19T05:44:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048665890000000002 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337155+0000", 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8632e-05 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337203+0000", 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7228000000000003e-05 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586056+0000", 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24885332199999999 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586595+0000", 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053866000000000005 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586768+0000", 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017356199999999999 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786605+0000", 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19983721300000001 2024-09-19T05:44:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3083 3.14 3:2b3bae94:::100000001f8.00000808:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.723573+0000", 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.40814439699999999, 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.408631101, 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3083 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.723573+0000", 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.723573+0000", 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.723576+0000", 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6390000000000001e-06 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.729726+0000", 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061495719999999999 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730580+0000", 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00085463499999999997 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730775+0000", 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019437400000000001 2024-09-19T05:44:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730836+0000", 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1624999999999994e-05 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730987+0000", 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151132 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.731150+0000", 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016210199999999999 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.777949+0000", 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046799008000000003 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3188 3.37 3.55ea23f7 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.843025+0000", 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.28869331500000001, 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.28929852, 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3188 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.843025+0000", 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.843025+0000", 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.843028+0000", 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1879999999999998e-06 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.846426+0000", 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033983669999999998 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.846445+0000", 2024-09-19T05:44:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8564e-05 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.846457+0000", 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2782000000000001e-05 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3296 3.13 3:ca1e5f65:::100000001f8.000008dd:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.994549+0000", 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.137168439, 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13782929299999999, 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3296 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994549+0000", 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994549+0000", 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994552+0000", 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.997734+0000", 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031824010000000001 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.997753+0000", 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8919999999999998e-05 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.997765+0000", 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2179e-05 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.998302+0000", 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053613499999999998 2024-09-19T05:44:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.998469+0000", 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000167558 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.998635+0000", 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166076 2024-09-19T05:44:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.023334+0000", 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024698905 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3394 3.14 3:2869d2dd:::100000001f8.0000093f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.089797+0000", 2024-09-19T05:44:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.041921142000000002, 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042666018, 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3394 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.089797+0000", 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.089797+0000", 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.089800+0000", 2024-09-19T05:44:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.937e-06 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092533+0000", 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027336489999999999 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092546+0000", 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2996999999999999e-05 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092556+0000", 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7529999999999996e-06 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.105385+0000", 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012829126999999999 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.105516+0000", 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013042800000000001 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.105746+0000", 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023046500000000001 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2994 3.2f 3:f4d09ad1:::100000001f8.000007af:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.481336+0000", 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.65038180700000003, 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65121195799999998, 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2994 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.481336+0000", 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.481336+0000", 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.481339+0000", 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.819e-06 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.492526+0000", 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011186890999999999 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.492553+0000", 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7182000000000002e-05 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.492570+0000", 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6799e-05 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.063729+0000", 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57115902500000004 2024-09-19T05:44:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.063880+0000", 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015076299999999999 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.064389+0000", 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00050961300000000001 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3397 3.13 3:c8b931af:::100000001f8.00000942:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.092581+0000", 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.039136969000000001, 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040053067999999997, 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3397 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092581+0000", 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092581+0000", 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092584+0000", 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6819999999999999e-06 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.095959+0000", 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033750579999999998 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.095983+0000", 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4666999999999999e-05 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.095997+0000", 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4100999999999999e-05 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.096116+0000", 2024-09-19T05:44:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118477 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.096289+0000", 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000173019 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.096457+0000", 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000168574 2024-09-19T05:44:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3084 3.2f 3:f436aed2:::100000001f8.00000809:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.736672+0000", 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.39504561999999999, 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.396042169, 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3084 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.736672+0000", 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.736672+0000", 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.736675+0000", 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.83e-06 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.742653+0000", 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059781139999999997 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.742678+0000", 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5225e-05 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.742691+0000", 2024-09-19T05:44:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2509e-05 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092351+0000", 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34965979000000003 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092543+0000", 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019210900000000001 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092760+0000", 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000217321 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3198 3.2f 3.25d2c16f (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.856727+0000", 2024-09-19T05:44:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.27499084200000001, 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.27606863999999998, 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3198 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.856727+0000", 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.856727+0000", 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.856730+0000", 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7149999999999998e-06 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.859820+0000", 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030904790000000001 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.859856+0000", 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5654000000000001e-05 2024-09-19T05:44:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.859888+0000", 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2347e-05 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3399 3.13 3.57a41a53 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.098709+0000", 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.033008598, 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034148220999999999, 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3399 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.098709+0000", 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.098709+0000", 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.098712+0000", 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4619999999999999e-06 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.101284+0000", 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002572193 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.101298+0000", 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4311000000000001e-05 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.101307+0000", 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5199999999999997e-06 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3301 3.d 3:b0b70bf4:::100000001f8.000008e2:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.002017+0000", 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.12970051799999999, 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.130953602, 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3301 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.002017+0000", 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.002017+0000", 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.002021+0000", 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.518e-06 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.005547+0000", 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035259390000000001 2024-09-19T05:44:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.014490+0000", 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.008942864 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.014502+0000", 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1994000000000001e-05 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.078083+0000", 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063581430999999994 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.078229+0000", 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145439 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.078373+0000", 2024-09-19T05:44:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014413300000000001 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103259+0000", 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024886537 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3400 3.11 3:8b6ee1ca:::100000001f8.00000945:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.101326+0000", 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.030392301, 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031753313999999998, 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3400 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.101326+0000", 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.101326+0000", 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.101327+0000", 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9319999999999999e-06 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103860+0000", 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002532215 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103871+0000", 2024-09-19T05:44:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1222e-05 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103879+0000", 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4570000000000005e-06 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.104868+0000", 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00098865999999999993 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.105062+0000", 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000193857 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.105248+0000", 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018594099999999999 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3403 3.30 3.37abc0b0 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.103901+0000", 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.027816875000000001, 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029263443, 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3403 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103901+0000", 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103901+0000", 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103903+0000", 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6130000000000001e-06 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.106752+0000", 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028493059999999998 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.106767+0000", 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5158e-05 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.106777+0000", 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5349999999999993e-06 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2700 3.30 3:0e790ae4:::100000001f8.00000689:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.1881538309999999, 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1896709169999999, 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2700 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943567+0000", 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.083e-06 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946402+0000", 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028349439999999998 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946419+0000", 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7105999999999999e-05 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946429+0000", 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8709999999999992e-06 2024-09-19T05:44:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.947951+0000", 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015219960000000001 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948153+0000", 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020153400000000001 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948415+0000", 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026250200000000002 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.064312+0000", 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115897369 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3209 3.1d 3:baf92435:::100000001f8.00000886:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.865533+0000", 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.26618531099999998, 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26779482700000001, 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3209 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.865533+0000", 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.865533+0000", 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.865535+0000", 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6120000000000001e-06 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.873558+0000", 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080231030000000002 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.873592+0000", 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3442999999999998e-05 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.873607+0000", 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4825e-05 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.950529+0000", 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076922083000000002 2024-09-19T05:44:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.950700+0000", 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000170946 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.950880+0000", 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018034600000000001 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.081599+0000", 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13071944099999999 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3306 3.11 3:8a5bd816:::100000001f8.000008e7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.014605+0000", 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.117112984, 2024-09-19T05:44:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.118801253, 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3306 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.014605+0000", 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.014605+0000", 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.014608+0000", 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3720000000000001e-06 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.016828+0000", 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022197060000000001 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.016847+0000", 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9364e-05 2024-09-19T05:44:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.016857+0000", 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9979999999999998e-06 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.017231+0000", 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037368699999999999 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.017389+0000", 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158209 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.017577+0000", 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018740599999999999 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.031795+0000", 2024-09-19T05:44:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014218374000000001 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3424 3.28 3.8221a5e8 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.106797+0000", 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024920388000000002, 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026728609, 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3424 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.106797+0000", 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.106797+0000", 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.106800+0000", 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1550000000000001e-06 2024-09-19T05:44:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.109862+0000", 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030621609999999999 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.109885+0000", 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2802e-05 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.109899+0000", 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4027e-05 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2894 3.14 3:2bba2ff8:::100000001f8.0000074b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.349763+0000", 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.78195509200000002, 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78382892400000004, 2024-09-19T05:44:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2894 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.349763+0000", 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.349763+0000", 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.349766+0000", 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9859999999999999e-06 2024-09-19T05:44:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353828+0000", 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040618019999999998 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353854+0000", 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6281000000000001e-05 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353870+0000", 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5743000000000002e-05 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.357875+0000", 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040050540000000001 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358048+0000", 2024-09-19T05:44:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017327 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358248+0000", 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019979899999999999 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.434330+0000", 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076082178 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3211 3.37 3.4acf78f7 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.873637+0000", 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.25808087800000001, 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.26004358300000002, 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3211 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.873637+0000", 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.873637+0000", 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.873640+0000", 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.785e-06 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.884646+0000", 2024-09-19T05:44:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011005769 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.884670+0000", 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4187000000000001e-05 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.884682+0000", 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2057e-05 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3322 3.1d 3:b8199b58:::100000001f8.000008f7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.018217+0000", 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.113500797, 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115527582, 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3322 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.018217+0000", 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.018217+0000", 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.018223+0000", 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5169999999999999e-06 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.022489+0000", 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042666340000000001 2024-09-19T05:44:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.022501+0000", 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1454000000000001e-05 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.022510+0000", 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1099999999999992e-06 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103460+0000", 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.080950039000000001 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103595+0000", 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013523499999999999 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103749+0000", 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015376800000000001 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4534.0:3152 3.1d e71/20)", 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.110293+0000", 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.02142486, 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023517753999999998, 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110293+0000", 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110293+0000", 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110295+0000", 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.646e-06 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110304+0000", 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4599999999999992e-06 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110319+0000", 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5000999999999999e-05 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110327+0000", 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9209999999999992e-06 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2895 3.9 3:939d0220:::100000001f8.0000074c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.77781686299999997, 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77996984300000005, 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2895 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353903+0000", 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356485+0000", 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025819390000000001 2024-09-19T05:44:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356507+0000", 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2109000000000001e-05 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356521+0000", 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3560999999999999e-05 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.696659+0000", 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34013843500000002 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.696845+0000", 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185873 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.698377+0000", 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015312240000000001 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.805489+0000", 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10711206199999999 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3213 3.17 3.fc4773d7 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.884707+0000", 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24701121500000001, 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24925641700000001, 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3213 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.884707+0000", 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.884707+0000", 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.884709+0000", 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.261e-06 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.890070+0000", 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053608090000000002 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.890096+0000", 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6401e-05 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.890110+0000", 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3769e-05 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3323 3.1d 3.ae5a789d (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.022535+0000", 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.10918327899999999, 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11149300500000001, 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3323 2024-09-19T05:44:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.022535+0000", 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.022535+0000", 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.022537+0000", 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4549999999999998e-06 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.025167+0000", 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026296869999999999 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.025187+0000", 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0222000000000001e-05 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.025202+0000", 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4598e-05 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4534.0:2900 3.2f e71/20)", 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.110362+0000", 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.021356242000000001, 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023758537999999999, 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110362+0000", 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110362+0000", 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110362+0000", 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1899999999999998e-07 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110364+0000", 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4729999999999999e-06 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110371+0000", 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7150000000000001e-06 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110377+0000", 2024-09-19T05:44:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4450000000000004e-06 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2900 3.2f 3:f407a56c:::100000001f8.00000751:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.356552+0000", 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.77516631700000005, 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77765571899999997, 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2900 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356552+0000", 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356552+0000", 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356555+0000", 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2679999999999999e-06 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358406+0000", 2024-09-19T05:44:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00185126 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358420+0000", 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3579e-05 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358428+0000", 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2199999999999992e-06 2024-09-19T05:44:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.909104+0000", 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55067599499999997 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.909283+0000", 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000179555 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.910367+0000", 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001083606 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038891+0000", 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12852424600000001 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3328 3.37 3.4701c477 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.025229+0000", 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.106488654, 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.109071341, 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3328 2024-09-19T05:44:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.025229+0000", 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.025229+0000", 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.025232+0000", 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9849999999999998e-06 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.029531+0000", 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042992079999999997 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.029546+0000", 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4739999999999999e-05 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.029558+0000", 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1782e-05 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4534.0:2942 3.2f e71/20)", 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.110386+0000", 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.021331536000000002, 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024001010999999999, 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110386+0000", 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110386+0000", 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110387+0000", 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.22e-07 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110389+0000", 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4509999999999999e-06 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110394+0000", 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.412e-06 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.110398+0000", 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0949999999999998e-06 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2796 3.9 3:9157a25d:::100000001f8.000006e9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.90201738099999995, 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90474715900000002, 2024-09-19T05:44:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2796 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229704+0000", 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1860000000000001e-06 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234027+0000", 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043232710000000001 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234040+0000", 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3485999999999999e-05 2024-09-19T05:44:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234051+0000", 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1022999999999999e-05 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364098+0000", 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.130046151 2024-09-19T05:44:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364248+0000", 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150793 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364444+0000", 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195735 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534144+0000", 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16970036399999999 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2908 3.11 3:8a4f5bd7:::100000001f8.00000759:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.77327458900000001, 2024-09-19T05:44:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77609243500000002, 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2908 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358445+0000", 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.08e-06 2024-09-19T05:44:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362030+0000", 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035843250000000002 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362050+0000", 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9860999999999998e-05 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362063+0000", 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3145999999999999e-05 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.715923+0000", 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35386083800000001 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.734420+0000", 2024-09-19T05:44:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018496133000000001 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.734712+0000", 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029234500000000001 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852483+0000", 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11777127499999999 2024-09-19T05:44:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3215 3.14 3:2978b787:::100000001f8.0000088c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.890138+0000", 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24158031899999999, 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24449427900000001, 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3215 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.890138+0000", 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.890138+0000", 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.890141+0000", 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2100000000000002e-06 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.892526+0000", 2024-09-19T05:44:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023855069999999998 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.892542+0000", 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5415000000000001e-05 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.892552+0000", 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0703999999999999e-05 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.892776+0000", 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022373799999999999 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.892933+0000", 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000156947 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.893105+0000", 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017146699999999999 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.961790+0000", 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068685528999999995 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3436 3.30 3.355ba070 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.109926+0000", 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.021792110999999999, 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024789723, 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3436 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.109926+0000", 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.109926+0000", 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.109928+0000", 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5799999999999999e-06 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.111748+0000", 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018197899999999999 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.111768+0000", 2024-09-19T05:44:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0202000000000001e-05 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.111778+0000", 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8700000000000004e-06 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2711 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.180636222, 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.183700682, 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2711 2024-09-19T05:44:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951084+0000", 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.553e-06 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956906+0000", 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058222400000000002 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956926+0000", 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9619e-05 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956937+0000", 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0594999999999999e-05 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958546+0000", 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001609369 2024-09-19T05:44:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958792+0000", 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024587600000000002 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.959831+0000", 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010394079999999999 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293509+0000", 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33367725100000001 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3438 3.11 3:88aee16f:::100000001f8.0000096b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.111798+0000", 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0199195, 2024-09-19T05:44:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023083672999999999, 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3438 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.111798+0000", 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.111798+0000", 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.111801+0000", 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.396e-06 2024-09-19T05:44:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.114402+0000", 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026012679999999999 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.114427+0000", 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4715999999999998e-05 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.114438+0000", 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1365999999999999e-05 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.130406+0000", 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015967898000000001 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.130600+0000", 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019357700000000001 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.130778+0000", 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178328 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3008 3.17 3:e9a16377:::100000001f8.000007bd:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.498533+0000", 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.63318460099999996, 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63643915799999995, 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3008 2024-09-19T05:44:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.498533+0000", 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.498533+0000", 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.498536+0000", 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.402e-06 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.519611+0000", 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021075856 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.519643+0000", 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1126e-05 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.519658+0000", 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5846e-05 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.068734+0000", 2024-09-19T05:44:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54907548699999997 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.069408+0000", 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067442399999999997 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3339 3.22 3:45333549:::100000001f8.00000908:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.032674+0000", 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.099044153999999995, 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.102389322, 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3339 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.032674+0000", 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.032674+0000", 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.032676+0000", 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5229999999999998e-06 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.036011+0000", 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033344669999999998 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.036025+0000", 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4259e-05 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.036038+0000", 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3448000000000001e-05 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.074138+0000", 2024-09-19T05:44:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.038099149999999998 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.074290+0000", 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015245600000000001 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3440 3.2f 3.82d598ef (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.114462+0000", 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017255467, 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020682663, 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3440 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.114462+0000", 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.114462+0000", 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.114465+0000", 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5689999999999999e-06 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116498+0000", 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020326440000000001 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116519+0000", 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1914999999999999e-05 2024-09-19T05:44:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116530+0000", 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.042e-05 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2808 3.11 3:8bd5b656:::100000001f8.000006f5:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.891590306, 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89508513199999995, 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2808 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240131+0000", 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244285+0000", 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041546919999999998 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244303+0000", 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7646999999999999e-05 2024-09-19T05:44:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244316+0000", 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2931999999999999e-05 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373296+0000", 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12897989000000001 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373463+0000", 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166847 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373621+0000", 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015846400000000001 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.585995+0000", 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.212374013 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2921 3.11 3:88e2f4e7:::100000001f8.00000766:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.76137685700000002, 2024-09-19T05:44:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76505641099999999, 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2921 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370344+0000", 2024-09-19T05:44:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373083+0000", 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027388270000000001 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373102+0000", 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9789e-05 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373116+0000", 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3456e-05 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778465+0000", 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.405348808 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778616+0000", 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015118400000000001 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778791+0000", 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017549300000000001 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852488+0000", 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073696686999999997 2024-09-19T05:44:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3112 3.9 3:92578395:::100000001f8.00000825:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.770688+0000", 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.36102972300000002, 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.36482813200000003, 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3112 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.770688+0000", 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.770688+0000", 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.770691+0000", 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4140000000000001e-06 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.774593+0000", 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039020249999999999 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.774612+0000", 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9599e-05 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.774622+0000", 2024-09-19T05:44:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9990000000000003e-06 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792618+0000", 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017996253 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792782+0000", 2024-09-19T05:44:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016351000000000001 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792958+0000", 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017623199999999999 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852494+0000", 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059536089 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3446 3.1d 3.ecceaf5d (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.116559+0000", 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015159196, 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019047611999999998, 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3446 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116559+0000", 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116559+0000", 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116562+0000", 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.281e-06 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.118090+0000", 2024-09-19T05:44:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015277979999999999 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.118103+0000", 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3033999999999999e-05 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.118112+0000", 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6779999999999995e-06 2024-09-19T05:44:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3344 3.17 3.a41fbc17 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.040536+0000", 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.091182286000000001, 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.095130351000000002, 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3344 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.040536+0000", 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.040536+0000", 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.040539+0000", 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1259999999999998e-06 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.044367+0000", 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038279920000000001 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.044390+0000", 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3249000000000001e-05 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.044403+0000", 2024-09-19T05:44:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2656e-05 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3452 3.a 3.eda9198a (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.118133+0000", 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013585105, 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017598099999999998, 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3452 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.118133+0000", 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.118133+0000", 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.118135+0000", 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8029999999999999e-06 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.122270+0000", 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041352990000000003 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.122289+0000", 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8697000000000001e-05 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.122300+0000", 2024-09-19T05:44:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1292e-05 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3115 3.30 3:0d9a3020:::100000001f8.00000828:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.774640+0000", 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.35707784100000001, 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.361158971, 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3115 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.774640+0000", 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.774640+0000", 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.774642+0000", 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778938+0000", 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042959590000000002 2024-09-19T05:44:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778951+0000", 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2432000000000001e-05 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778970+0000", 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8847000000000001e-05 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.779051+0000", 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1286999999999996e-05 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.779234+0000", 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018342799999999999 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.779458+0000", 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022387000000000001 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816463+0000", 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037004519999999999 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3454 3.14 3:2990efe9:::100000001f8.0000097b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.122326+0000", 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0093920210000000004, 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013567469, 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3454 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.122326+0000", 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.122326+0000", 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.122328+0000", 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.123984+0000", 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016552509999999999 2024-09-19T05:44:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.123999+0000", 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5634e-05 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.124008+0000", 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2869999999999993e-06 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.126594+0000", 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025855840000000001 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.126772+0000", 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000177801 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.126990+0000", 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021823200000000001 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3218 3.13 3:ca464668:::100000001f8.0000088f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.906645+0000", 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.22507317600000001, 2024-09-19T05:44:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22931553399999999, 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3218 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.906645+0000", 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.906645+0000", 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.906648+0000", 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2430000000000001e-06 2024-09-19T05:44:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.936943+0000", 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030295328 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.936961+0000", 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7382999999999999e-05 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.936970+0000", 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6190000000000006e-06 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.937731+0000", 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076095000000000004 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.937928+0000", 2024-09-19T05:44:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001964 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.938137+0000", 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020925199999999999 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.956175+0000", 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018038480999999999 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3462 3.13 3.bab16953 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.124033+0000", 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0076847740000000001, 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011990878999999999, 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3462 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.124033+0000", 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.124033+0000", 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.124035+0000", 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1610000000000001e-06 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.129398+0000", 2024-09-19T05:44:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053631929999999996 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.129424+0000", 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5707e-05 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.129439+0000", 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4966e-05 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2732 3.9 3:92ca8380:::100000001f8.000006a9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.1237666179999999, 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.128120215, 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2732 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007954+0000", 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018132+0000", 2024-09-19T05:44:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010177734000000001 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018148+0000", 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6555999999999999e-05 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018187+0000", 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8760000000000002e-05 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020347+0000", 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021597589999999998 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020514+0000", 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016697899999999999 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020695+0000", 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018172199999999999 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.375158+0000", 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35446218200000001 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3220 3.2f 3.5298a6af (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.936994+0000", 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.19472418, 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.199137657, 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3220 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.936994+0000", 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.936994+0000", 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.936996+0000", 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2450000000000001e-06 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.939767+0000", 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002770743 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.939787+0000", 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0126999999999999e-05 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.939799+0000", 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2058e-05 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3355 3.22 3:4498199b:::100000001f8.00000918:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.044427+0000", 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.087291162000000005, 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091752944000000003, 2024-09-19T05:44:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3355 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.044427+0000", 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.044427+0000", 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.044431+0000", 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3579999999999996e-06 2024-09-19T05:44:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.047484+0000", 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030532390000000001 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.047501+0000", 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6447e-05 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.047511+0000", 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0773e-05 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092920+0000", 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045408769000000002 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.093046+0000", 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012570800000000001 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3468 3.14 3.d3c3754 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.129466+0000", 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0022514509999999998, 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067651550000000001, 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3468 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.129466+0000", 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.129466+0000", 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.129470+0000", 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0740000000000003e-06 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.134334+0000", 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048638520000000001 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.134350+0000", 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6021000000000001e-05 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.134362+0000", 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2004000000000001e-05 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2821 3.9 3:9040d340:::100000001f8.00000702:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.87458805500000003, 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87916814899999995, 2024-09-19T05:44:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2821 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257133+0000", 2024-09-19T05:44:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.146e-06 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266617+0000", 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0094844509999999996 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266644+0000", 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6942999999999999e-05 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266660+0000", 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5379e-05 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459066+0000", 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19240588 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459208+0000", 2024-09-19T05:44:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014251699999999999 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459363+0000", 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000154535 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786586+0000", 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32722325099999999 2024-09-19T05:44:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3029 3.17 3:ebfd80ec:::100000001f8.000007d2:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.531477+0000", 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.60024077600000003, 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60501993399999998, 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3029 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.531477+0000", 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.531477+0000", 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.531480+0000", 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2610000000000002e-06 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.536028+0000", 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004547242 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.536048+0000", 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0489999999999999e-05 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.536072+0000", 2024-09-19T05:44:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3601e-05 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075359+0000", 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53928714099999997 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075515+0000", 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015659499999999999 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3121 3.37 3.264e08f7 (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.782354+0000", 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34936402799999999, 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.354220651, 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3121 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.782354+0000", 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.782354+0000", 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.782356+0000", 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4710000000000001e-06 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786220+0000", 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038637530000000002 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786242+0000", 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2067000000000001e-05 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786257+0000", 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4902000000000001e-05 2024-09-19T05:44:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3357 3.22 3.ae606462 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.047530+0000", 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.084187709999999999, 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.089116294999999998, 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3357 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.047530+0000", 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.047530+0000", 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.047532+0000", 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1689999999999999e-06 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.050050+0000", 2024-09-19T05:44:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025173600000000002 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.050062+0000", 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2379e-05 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.050070+0000", 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8439999999999998e-06 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3127 3.37 3.d7581e37 (undecoded) ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.786313+0000", 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34540526999999999, 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35039980599999998, 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3127 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786313+0000", 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786313+0000", 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786318+0000", 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9690000000000002e-06 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.789245+0000", 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002927439 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.789273+0000", 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8102e-05 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.789296+0000", 2024-09-19T05:44:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3388e-05 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3227 3.30 3:0ee2eef2:::100000001f8.00000898:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.944362+0000", 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.18735550000000001, 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.192418969, 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3227 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.944362+0000", 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.944362+0000", 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.944365+0000", 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7989999999999998e-06 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947458+0000", 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030929600000000001 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947478+0000", 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9942999999999999e-05 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947490+0000", 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2252000000000001e-05 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947603+0000", 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112802 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947741+0000", 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013760700000000001 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947900+0000", 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000159634 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.992987+0000", 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045086198000000001 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3360 3.2d 3.f345daad (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.050085+0000", 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.081633321999999994, 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.086791698, 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3360 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.050085+0000", 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.050085+0000", 2024-09-19T05:44:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.050086+0000", 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9800000000000001e-06 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.055819+0000", 2024-09-19T05:44:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.005732394 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.055838+0000", 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8629000000000001e-05 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.055850+0000", 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1986e-05 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2746 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.096229406, 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1014641350000001, 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2746 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035491+0000", 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043360+0000", 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0078693240000000005 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043383+0000", 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2853e-05 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.045231+0000", 2024-09-19T05:44:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001847476 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.122884+0000", 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077653244999999996 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123275+0000", 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039126200000000002 2024-09-19T05:44:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123895+0000", 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061987200000000004 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293518+0000", 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.169623147 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3035 3.2d 3:b47285d1:::100000001f8.000007d8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.536928+0000", 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.59479000999999998, 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60009844899999998, 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3035 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.536928+0000", 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.536928+0000", 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.536961+0000", 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2814999999999997e-05 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.541016+0000", 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00405522 2024-09-19T05:44:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.541036+0000", 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0069999999999999e-05 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.541046+0000", 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7049999999999994e-06 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670286+0000", 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.129240828 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670445+0000", 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158825 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670586+0000", 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000140748 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.727154+0000", 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056568089000000002 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3133 3.9 3:936cb4f9:::100000001f8.0000083a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.789323+0000", 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.342394529, 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34777883599999998, 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3133 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.789323+0000", 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.789323+0000", 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.789326+0000", 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.954e-06 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.791390+0000", 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002063938 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795067+0000", 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036765360000000002 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795084+0000", 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7489000000000001e-05 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.805524+0000", 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010439634999999999 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.805671+0000", 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014724599999999999 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.805834+0000", 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016301099999999999 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.889237+0000", 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.083403273999999999 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3237 3.37 3.9c86dcf7 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.947524+0000", 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.18419359900000001, 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18965042400000001, 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3237 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947524+0000", 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947524+0000", 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.947527+0000", 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4600000000000002e-06 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.951017+0000", 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034907240000000002 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.951033+0000", 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5549e-05 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.951044+0000", 2024-09-19T05:44:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0892e-05 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3363 3.d 3.affa668d (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.055874+0000", 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.075844079999999994, 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.081351100999999995, 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3363 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.055874+0000", 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.055874+0000", 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.055877+0000", 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0589999999999998e-06 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.058789+0000", 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029119179999999999 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.058807+0000", 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8255999999999999e-05 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.058819+0000", 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1953999999999999e-05 2024-09-19T05:44:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2942 3.2f 3:f4ea050d:::100000001f8.0000077b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.425635+0000", 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.70608254400000003, 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71165804700000002, 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2942 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.425635+0000", 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.425635+0000", 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.425639+0000", 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.349e-06 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.430175+0000", 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045359570000000002 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.430198+0000", 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3266999999999999e-05 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.430210+0000", 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2469e-05 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963090+0000", 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.532879667 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963282+0000", 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000191924 2024-09-19T05:44:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963433+0000", 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015078699999999999 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039100+0000", 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075667643000000007 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3037 3.14 3:29880457:::100000001f8.000007da:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.541064+0000", 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.59065426499999996, 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59629939600000004, 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3037 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.541064+0000", 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.541064+0000", 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.541066+0000", 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.615e-06 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549140+0000", 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0080733820000000005 2024-09-19T05:44:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549155+0000", 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4936999999999999e-05 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549165+0000", 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0190999999999999e-05 2024-09-19T05:44:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549363+0000", 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000198774 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549529+0000", 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165076 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549708+0000", 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000179519 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.618596+0000", 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068887444000000006 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3242 3.1d 3:bb8d9892:::100000001f8.000008a7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.951061+0000", 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.18065682199999999, 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18639201599999999, 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3242 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.951061+0000", 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.951061+0000", 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.951064+0000", 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7460000000000001e-06 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.953727+0000", 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026634229999999998 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.953745+0000", 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.789e-05 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.953756+0000", 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1416e-05 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994288+0000", 2024-09-19T05:44:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040531788999999999 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994395+0000", 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106952 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994519+0000", 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012342899999999999 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103281+0000", 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.108762436 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3364 3.14 3:29f1438c:::100000001f8.00000921:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.058843+0000", 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.072874688000000007, 2024-09-19T05:44:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.078705608999999996, 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3364 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.058843+0000", 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.058843+0000", 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.058846+0000", 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8210000000000002e-06 2024-09-19T05:44:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.062854+0000", 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040083280000000002 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.062869+0000", 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5107e-05 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.062880+0000", 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101999999999999e-05 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.063058+0000", 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017799600000000001 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.063255+0000", 2024-09-19T05:44:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019786200000000001 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.063456+0000", 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020063899999999999 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.074393+0000", 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010936899 2024-09-19T05:44:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2666 3.9 3:92ca8d9b:::100000001f8.00000667:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.2369932079999999, 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2429040119999999, 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2666 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894727+0000", 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8439999999999998e-06 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897799+0000", 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030718199999999998 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897817+0000", 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7847999999999999e-05 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897831+0000", 2024-09-19T05:44:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3845999999999999e-05 2024-09-19T05:44:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897938+0000", 2024-09-19T05:44:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010742399999999999 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898111+0000", 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017279099999999999 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898360+0000", 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024885800000000002 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956644+0000", 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058284217999999999 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2956 3.30 3:0f38f4c8:::100000001f8.00000789:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.430235+0000", 2024-09-19T05:44:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.70148258900000005, 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70746900400000001, 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2956 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.430235+0000", 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.430235+0000", 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.430238+0000", 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.439601+0000", 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093628419999999997 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.439639+0000", 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8664000000000001e-05 2024-09-19T05:44:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.439660+0000", 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0506e-05 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.442565+0000", 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029054549999999999 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.442773+0000", 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000207894 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.442979+0000", 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000206178 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.465387+0000", 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022407844999999999 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3368 3.13 3:c9fd95ca:::100000001f8.00000925:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.062906+0000", 2024-09-19T05:44:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.068811507999999993, 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074879326999999996, 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3368 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.062906+0000", 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.062906+0000", 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.062909+0000", 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.198e-06 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.066500+0000", 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003591772 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.066517+0000", 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6728e-05 2024-09-19T05:44:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.066526+0000", 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8210000000000008e-06 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.068127+0000", 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016015980000000001 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.068312+0000", 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018451700000000001 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.068483+0000", 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017061299999999999 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082754+0000", 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014270972999999999 2024-09-19T05:44:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2667 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.2338533599999999, 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2400038179999999, 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2667 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897867+0000", 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9299999999999999e-06 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902567+0000", 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004699356 2024-09-19T05:44:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902588+0000", 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1631e-05 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902602+0000", 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3478e-05 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.906894+0000", 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042925760000000002 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907078+0000", 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183067 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907289+0000", 2024-09-19T05:44:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021123499999999999 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956671+0000", 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049382078000000003 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2959 3.37 3:ecfba76b:::100000001f8.0000078c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.439706+0000", 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.69201185099999996, 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.69823824899999998, 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2959 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.439706+0000", 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.439706+0000", 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.439709+0000", 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2380000000000002e-06 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.443945+0000", 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004236096 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.443979+0000", 2024-09-19T05:44:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3546e-05 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.443997+0000", 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8216e-05 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.974609+0000", 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53061193699999998 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.974770+0000", 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016069399999999999 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.974919+0000", 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014974000000000001 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.040071+0000", 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065151797999999997 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3152 3.1d 3:b9e29808:::100000001f8.0000084d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.803481+0000", 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.32823690100000003, 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33454451899999998, 2024-09-19T05:44:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3152 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.803481+0000", 2024-09-19T05:44:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.803481+0000", 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.803483+0000", 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.526e-06 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806428+0000", 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00294415 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806446+0000", 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7920000000000001e-05 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806457+0000", 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1891e-05 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.874224+0000", 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067766181999999994 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.874462+0000", 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000238175 2024-09-19T05:44:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.874689+0000", 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000226794 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.994133+0000", 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.119444492 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3375 3.17 3.659daad7 (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.066552+0000", 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.065166026000000002, 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.071575810000000004, 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3375 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.066552+0000", 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.066552+0000", 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.066554+0000", 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2570000000000002e-06 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.069124+0000", 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025695829999999998 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.069142+0000", 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8046e-05 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.069153+0000", 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1569000000000001e-05 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2753 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.070206569, 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0766884480000001, 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2753 2024-09-19T05:44:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061513+0000", 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.871e-06 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078405+0000", 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016891461 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078433+0000", 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8436e-05 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078447+0000", 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3607e-05 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137285+0000", 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058837911 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137457+0000", 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017199399999999999 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137637+0000", 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180173 2024-09-19T05:44:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.460405+0000", 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.322768426 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3153 3.9 3:90608b28:::100000001f8.0000084e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.806480+0000", 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.32523767100000001, 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.331821211, 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3153 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806480+0000", 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806480+0000", 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806484+0000", 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.394e-06 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.809620+0000", 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031366469999999998 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.809635+0000", 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4808e-05 2024-09-19T05:44:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.809646+0000", 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0648999999999999e-05 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852525+0000", 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042879665999999997 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852652+0000", 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012682899999999999 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852797+0000", 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145277 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.892336+0000", 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039538624000000001 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3376 3.2d 3.4d8206ed (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.069185+0000", 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.062533321000000003, 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069203611999999998, 2024-09-19T05:44:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3376 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.069185+0000", 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.069185+0000", 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.069188+0000", 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2849999999999999e-06 2024-09-19T05:44:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.072201+0000", 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003013417 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.072217+0000", 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6152000000000001e-05 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.072229+0000", 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1471000000000001e-05 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2965 3.17 3:e89dd327:::100000001f8.00000792:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.444041+0000", 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.68767726600000001, 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.69441848399999995, 2024-09-19T05:44:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2965 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.444041+0000", 2024-09-19T05:44:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.444041+0000", 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.444046+0000", 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0130000000000001e-06 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.448933+0000", 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048875139999999999 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.448952+0000", 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9224000000000001e-05 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.448965+0000", 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.226e-05 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.981919+0000", 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53295421300000001 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.982169+0000", 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025024999999999998 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3051 3.30 3:0fd3b3ba:::100000001f8.000007e8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.572191+0000", 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55952660700000001, 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.566351983, 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3051 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.572191+0000", 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.572191+0000", 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.572195+0000", 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7500000000000001e-06 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.598318+0000", 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026123091000000001 2024-09-19T05:44:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.639428+0000", 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041109973000000001 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.639449+0000", 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0971999999999998e-05 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.646141+0000", 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066920089999999996 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.646278+0000", 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013660500000000001 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.646462+0000", 2024-09-19T05:44:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018390299999999999 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.726221+0000", 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.079759098 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3381 3.30 3:0e5a7e5b:::100000001f8.00000932:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.072255+0000", 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.059463027000000002, 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066383881000000006, 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3381 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.072255+0000", 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.072255+0000", 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.072258+0000", 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9160000000000001e-06 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075331+0000", 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030730990000000001 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075347+0000", 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6129000000000001e-05 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075357+0000", 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0471e-05 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075453+0000", 2024-09-19T05:44:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5946000000000006e-05 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075583+0000", 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000129694 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075724+0000", 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014091099999999999 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2673 3.2d 3:b45a3f48:::100000001f8.0000066e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.2208559329999999, 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2278674330000001, 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2673 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910864+0000", 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915100+0000", 2024-09-19T05:44:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042352259999999999 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915123+0000", 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3334999999999999e-05 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915137+0000", 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4178999999999999e-05 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915220+0000", 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2693000000000003e-05 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915395+0000", 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017551099999999999 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915574+0000", 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178357 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951064+0000", 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035490116000000002 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2858 3.9 3:92f6ecc1:::100000001f8.00000727:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.82539041999999996, 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.83249638100000001, 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2858 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306329+0000", 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0719999999999998e-06 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311583+0000", 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052534000000000001 2024-09-19T05:44:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311607+0000", 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3716000000000001e-05 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311625+0000", 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8681999999999998e-05 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.528241+0000", 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.216615688 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.528918+0000", 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067717000000000005 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.529096+0000", 2024-09-19T05:44:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017772600000000001 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786593+0000", 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25749734800000001 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3053 3.17 3:eae4145f:::100000001f8.000007ea:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.639474+0000", 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.49224379499999998, 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49944154000000002, 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3053 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.639474+0000", 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.639474+0000", 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.639493+0000", 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8468999999999999e-05 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.649843+0000", 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010350933999999999 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.649863+0000", 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9967999999999999e-05 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.649874+0000", 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0339999999999999e-05 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092757+0000", 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.442883733 2024-09-19T05:44:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.092895+0000", 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013773799999999999 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3176 3.30 3:0dfe1bb6:::100000001f8.00000865:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.822291+0000", 2024-09-19T05:44:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.30942678499999998, 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.316708347, 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3176 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.822291+0000", 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.822291+0000", 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.822294+0000", 2024-09-19T05:44:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0180000000000002e-06 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.824975+0000", 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026807189999999998 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826163+0000", 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011880879999999999 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826177+0000", 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4007e-05 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826259+0000", 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2225000000000006e-05 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826549+0000", 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028957899999999999 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826802+0000", 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000252854 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.866872+0000", 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040070924000000001 2024-09-19T05:44:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3279 3.2f 3.419109ef (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.978869+0000", 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.152848599, 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.160224273, 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3279 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978869+0000", 2024-09-19T05:44:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978869+0000", 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978872+0000", 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7010000000000001e-06 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.983296+0000", 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044244089999999998 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.983328+0000", 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2044e-05 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.983353+0000", 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4533999999999999e-05 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2862 3.9 3:907f8708:::100000001f8.0000072b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.82005014399999998, 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.82748607799999996, 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2862 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311670+0000", 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316240+0000", 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045697189999999999 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316271+0000", 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1341999999999999e-05 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316324+0000", 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2984e-05 2024-09-19T05:44:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534179+0000", 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21785516099999999 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534368+0000", 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018821699999999999 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534552+0000", 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018435599999999999 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786599+0000", 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25204730600000003 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3384 3.30 3:0c844521:::100000001f8.00000935:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.075919+0000", 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.055798748000000002, 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063319464000000006, 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3384 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075919+0000", 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075919+0000", 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.075921+0000", 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0940000000000002e-06 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.080334+0000", 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044126479999999999 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.080361+0000", 2024-09-19T05:44:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7348999999999999e-05 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.080374+0000", 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2489e-05 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.117653+0000", 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037279633 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.117915+0000", 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026161000000000001 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.118126+0000", 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021152700000000001 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2769 3.11 3:8be9d197:::100000001f8.000006ce:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 1.0289540749999999, 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0365620390000001, 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2769 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102767+0000", 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.021e-06 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136259+0000", 2024-09-19T05:44:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033491981999999997 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136291+0000", 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1773999999999998e-05 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136302+0000", 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1154e-05 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.239524+0000", 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103221854 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240715+0000", 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001191042 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.242825+0000", 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002110289 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534133+0000", 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29130808400000002 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3387 3.14 3:29f10644:::100000001f8.00000938:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.080402+0000", 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.051315771000000003, 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.059004929999999997, 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3387 2024-09-19T05:44:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.080402+0000", 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.080402+0000", 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.080405+0000", 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8329999999999998e-06 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082535+0000", 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021301359999999999 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082556+0000", 2024-09-19T05:44:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0959e-05 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082570+0000", 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3539e-05 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082736+0000", 2024-09-19T05:44:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166167 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082921+0000", 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018530200000000001 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.083097+0000", 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000175778 2024-09-19T05:44:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2991 3.2f 3:f7814f15:::100000001f8.000007ac:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.479294+0000", 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.65242398300000004, 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66018824899999995, 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2991 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.479294+0000", 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.479294+0000", 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.479297+0000", 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.728e-06 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.481286+0000", 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019892690000000001 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.481306+0000", 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9786e-05 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.481315+0000", 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8300000000000008e-06 2024-09-19T05:44:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.040109+0000", 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55879357900000004 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.040239+0000", 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013022899999999999 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.040385+0000", 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014575800000000001 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3065 3.13 3:ca2e1d21:::100000001f8.000007f6:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.705331+0000", 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.42638727999999998, 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.434221994, 2024-09-19T05:44:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3065 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.705331+0000", 2024-09-19T05:44:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.705331+0000", 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.705335+0000", 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8400000000000002e-06 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.716472+0000", 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011136392 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.716488+0000", 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5750999999999999e-05 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.716500+0000", 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2047e-05 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.718104+0000", 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001604706 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.718362+0000", 2024-09-19T05:44:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025760699999999997 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.718564+0000", 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020195400000000001 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.760080+0000", 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041516005000000002 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3288 3.2f 3.880100af (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.983408+0000", 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.14831006599999999, 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15621950700000001, 2024-09-19T05:44:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3288 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.983408+0000", 2024-09-19T05:44:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.983408+0000", 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.983415+0000", 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7519999999999999e-06 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.988904+0000", 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054890340000000003 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.988927+0000", 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3600000000000001e-05 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.988941+0000", 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.38e-05 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2773 3.9 3:9102af4b:::100000001f8.000006d2:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.98577360800000002, 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.99373966000000002, 2024-09-19T05:44:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2773 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145947+0000", 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.162e-06 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149477+0000", 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035300800000000001 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149524+0000", 2024-09-19T05:44:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6264999999999998e-05 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149539+0000", 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5169e-05 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234044+0000", 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.084504863999999999 2024-09-19T05:44:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234273+0000", 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022875599999999999 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234509+0000", 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023624799999999999 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534140+0000", 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29963108999999999 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3067 3.11 3:8909406e:::100000001f8.000007f8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.716567+0000", 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.415150927, 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42319553599999998, 2024-09-19T05:44:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3067 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.716567+0000", 2024-09-19T05:44:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.716567+0000", 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.716570+0000", 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2219999999999998e-06 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.723516+0000", 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069459789999999997 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.723534+0000", 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8142e-05 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.723545+0000", 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1225e-05 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786656+0000", 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.063110543000000005 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786822+0000", 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165866 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.787025+0000", 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020317600000000001 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.865620+0000", 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.078594949999999997 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3180 3.14 3:2a46c135:::100000001f8.00000869:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.835230+0000", 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.296487377, 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.30462125699999998, 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3180 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.835230+0000", 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.835230+0000", 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.835234+0000", 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4369999999999999e-06 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838484+0000", 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032501629999999999 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838508+0000", 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4110000000000001e-05 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838519+0000", 2024-09-19T05:44:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0628000000000001e-05 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838741+0000", 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022191100000000001 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.838998+0000", 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025689499999999999 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.839181+0000", 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183769 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.865845+0000", 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026663744999999999 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3290 3.1d 3:ba9c4a52:::100000001f8.000008d7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.988971+0000", 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.142747132, 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15097998600000001, 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3290 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.988971+0000", 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.988971+0000", 2024-09-19T05:44:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.988975+0000", 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4009999999999999e-06 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.990595+0000", 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016203649999999999 2024-09-19T05:44:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.990637+0000", 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1539000000000003e-05 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.990654+0000", 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6575999999999999e-05 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.042411+0000", 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051757230000000001 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.042583+0000", 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000172256 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.042744+0000", 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000161096 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.103433+0000", 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.060689130000000001 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3388 3.2d 3.4d63ee6d (undecoded) ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:33.082606+0000", 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.049111464000000001, 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057433367999999999, 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3388 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082606+0000", 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082606+0000", 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.082610+0000", 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0040000000000001e-06 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.088381+0000", 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057706429999999998 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.088420+0000", 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8518e-05 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.088434+0000", 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4337e-05 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 100 2024-09-19T05:44:33.661 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:33.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.171+0000 7f9f4f7fe640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f9f48049680 0x7f9f481967b0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:33.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:33.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.230+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f48045bb0 con 0x7f9f48049680 2024-09-19T05:44:33.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.262+0000 7f9f4ffff640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9f480e2f00 con 0x7f9f48049680 2024-09-19T05:44:33.664 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.424+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9f480e2f00 con 0x7f9f48049680 2024-09-19T05:44:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.425+0000 7f9f35ffb640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f9f480cfcc0 0x7f9f480d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.425+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f9f4804b5b0 con 0x7f9f480cfcc0 2024-09-19T05:44:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.425+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f9f480cfcc0 0x7f9f480d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.426+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f9f480cfcc0 0x7f9f480d00a0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:33.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.428+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f4804b5b0 con 0x7f9f480cfcc0 2024-09-19T05:44:33.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.469+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9f4804c0f0 con 0x7f9f480cfcc0 2024-09-19T05:44:33.666 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.474+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9f4804c0f0 con 0x7f9f480cfcc0 2024-09-19T05:44:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.476+0000 7f9f35ffb640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f9f48002dd0 0x7f9f480031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.476+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f9f4804eca0 con 0x7f9f48002dd0 2024-09-19T05:44:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.485+0000 7f9f4e7fc640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f9f48002dd0 0x7f9f480031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:33.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.486+0000 7f9f4e7fc640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f9f48002dd0 0x7f9f480031b0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:33.668 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:33.662+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:33.738 INFO:teuthology.orchestra.run.smithi123.stdout:get_pg_num_history pg_num_history pg_num_history(e67 pg_nums {1={16=1},2={19=32},3={20=64}} deleted_pools ) 2024-09-19T05:44:33.738 INFO:teuthology.orchestra.run.smithi123.stdout:Importing pgid 3.18 2024-09-19T05:44:33.738 INFO:teuthology.orchestra.run.smithi123.stdout:Write #3:186b5309:::100000001f7.000000fd:head# 2024-09-19T05:44:33.740 INFO:teuthology.orchestra.run.smithi123.stdout:snapset 1=[]:{} 2024-09-19T05:44:33.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.739+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f4804eca0 con 0x7f9f48002dd0 2024-09-19T05:44:33.743 INFO:teuthology.orchestra.run.smithi123.stdout:Write #3:191fe66d:::100000001f7.000000a0:head# 2024-09-19T05:44:33.744 INFO:teuthology.orchestra.run.smithi123.stdout:snapset 1=[]:{} 2024-09-19T05:44:33.746 INFO:teuthology.orchestra.run.smithi123.stdout:Write #3:19643448:::100000001f7.00000065:head# 2024-09-19T05:44:33.746 INFO:teuthology.orchestra.run.smithi123.stdout:snapset 1=[]:{} 2024-09-19T05:44:33.748 INFO:teuthology.orchestra.run.smithi123.stdout:Write #3:19a9fe8d:::100000001f7.000000b4:head# 2024-09-19T05:44:33.749 INFO:teuthology.orchestra.run.smithi123.stdout:snapset 1=[]:{} 2024-09-19T05:44:33.753 INFO:teuthology.orchestra.run.smithi123.stdout:write_pg epoch 62 info 3.18( v 62'23 (0'0,62'23] local-lis/les=20/21 n=4 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-19T05:44:33.755 INFO:teuthology.orchestra.run.smithi123.stdout:Import successful 2024-09-19T05:44:33.762 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:33.760+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:33.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:44:33.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:44:33.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.775+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9f48034c80 con 0x7f9f48002dd0 2024-09-19T05:44:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.840+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9f48034c80 con 0x7f9f48002dd0 2024-09-19T05:44:33.842 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:33.843 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:33.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.840+0000 7f9f35ffb640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f9f480ce1d0 0x7f9f480ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:33.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.840+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9f480e0370 con 0x7f9f480ce1d0 2024-09-19T05:44:33.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.840+0000 7f9f4f7fe640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f9f480ce1d0 0x7f9f480ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:33.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.841+0000 7f9f4f7fe640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f9f480ce1d0 0x7f9f480ce5b0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:33.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.842+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f480e0370 con 0x7f9f480ce1d0 2024-09-19T05:44:33.864 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:33.861+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:33.887 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:33.887 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:33.887 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.885+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9f480349a0 con 0x7f9f480ce1d0 2024-09-19T05:44:33.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.888+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9f480349a0 con 0x7f9f480ce1d0 2024-09-19T05:44:33.891 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:33.891 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:33.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.889+0000 7f9f35ffb640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f9f480df700 0x7f9f480dfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:33.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.889+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f9f480e00a0 con 0x7f9f480df700 2024-09-19T05:44:33.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.889+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f9f480df700 0x7f9f480dfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:33.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.890+0000 7f9f4effd640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f9f480df700 0x7f9f480dfb60 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:33.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.890+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f480e00a0 con 0x7f9f480df700 2024-09-19T05:44:33.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.916+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9f480e00a0 con 0x7f9f480df700 2024-09-19T05:44:33.920 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:44:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.918+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9f480e00a0 con 0x7f9f480df700 2024-09-19T05:44:33.921 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.919+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=72}) -- 0x7f9f480cbdf0 con 0x7f9f48036210 2024-09-19T05:44:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.919+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f480e00a0 con 0x7f9f48036210 2024-09-19T05:44:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.920+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f9f4001e040 con 0x7f9f48036210 2024-09-19T05:44:33.925 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:33.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.921+0000 7f9f35ffb640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f9f4819de40 0x7f9f4819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:33.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.922+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f9f4819e940 con 0x7f9f4819de40 2024-09-19T05:44:33.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.922+0000 7f9f4e7fc640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f9f4819de40 0x7f9f4819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:33.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.923+0000 7f9f4e7fc640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f9f4819de40 0x7f9f4819e2c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:33.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.924+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f4819e940 con 0x7f9f4819de40 2024-09-19T05:44:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.955+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9f4819e890 con 0x7f9f4819de40 2024-09-19T05:44:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.957+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9f4819e890 con 0x7f9f4819de40 2024-09-19T05:44:33.960 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:33.960 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:33.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.958+0000 7f9f35ffb640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f9f481ab1f0 0x7f9f481ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:33.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.958+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f9f481b0520 con 0x7f9f481ab1f0 2024-09-19T05:44:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.958+0000 7f9f4f7fe640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f9f481ab1f0 0x7f9f481ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.958+0000 7f9f4f7fe640 1 --2- 172.21.15.2:0/3818745075 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f9f481ab1f0 0x7f9f481ad5e0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:33.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.959+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f481b0520 con 0x7f9f481ab1f0 2024-09-19T05:44:33.965 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:33.963+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:33.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.987+0000 7f9f35ffb640 1 -- 172.21.15.2:0/3818745075 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9f481b04f0 con 0x7f9f481ab1f0 2024-09-19T05:44:33.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:33.989+0000 7f9f37fff640 1 -- 172.21.15.2:0/3818745075 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9f481b04f0 con 0x7f9f481ab1f0 2024-09-19T05:44:33.992 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:33.992 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:33.996 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:33.997 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:34.065 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-19T05:44:34.065 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:34.064+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:34.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.109+0000 7f1ce07f6640 1 Processor -- start 2024-09-19T05:44:34.112 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.109+0000 7f1ce07f6640 1 -- start start 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1ce07f6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cd8070f50 0x7f1cd8073cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1ce07f6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8071330 0x7f1cd8072c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1ce07f6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd8073610 con 0x7f1cd8073180 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1ce07f6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd8074200 con 0x7f1cd8070f50 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1ce07f6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd8162480 con 0x7f1cd8071330 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1cde592640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cd8070f50 0x7f1cd8073cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1cded93640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1cd8073180 0x7f1cd815ee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:47702/0 (socket says 172.21.15.2:47702) 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1cded93640 1 -- 172.21.15.2:0/1797734807 learned_addr learned my addr 172.21.15.2:0/1797734807 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.112+0000 7f1cde592640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cd8070f50 0x7f1cd8073cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44248/0 (socket says 172.21.15.2:44248) 2024-09-19T05:44:34.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.113+0000 7f1cddd91640 1 --2- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8071330 0x7f1cd8072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.113+0000 7f1cde592640 1 -- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8071330 msgr2=0x7f1cd8072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.113+0000 7f1cde592640 1 --2- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8071330 0x7f1cd8072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.113+0000 7f1cdd590640 1 -- 172.21.15.2:0/1797734807 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3631037410 0 0) 0x7f1cd8073610 con 0x7f1cd8073180 2024-09-19T05:44:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.113+0000 7f1cde592640 1 -- 172.21.15.2:0/1797734807 >> v1:172.21.15.123:6789/0 conn(0x7f1cd8073180 legacy=0x7f1cd815ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.113+0000 7f1cde592640 1 -- 172.21.15.2:0/1797734807 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd8162760 con 0x7f1cd8070f50 2024-09-19T05:44:34.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.114+0000 7f1cde592640 1 --2- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8070f50 0x7f1cd8073cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40097d0 tx=0x7f1cd402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b963e45834bc1670 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.116+0000 7f1cdd590640 1 -- 172.21.15.2:0/1797734807 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4037070 con 0x7f1cd8070f50 2024-09-19T05:44:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.116+0000 7f1cdd590640 1 -- 172.21.15.2:0/1797734807 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd4032440 con 0x7f1cd8070f50 2024-09-19T05:44:34.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.116+0000 7f1cdd590640 1 -- 172.21.15.2:0/1797734807 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4032810 con 0x7f1cd8070f50 2024-09-19T05:44:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.117+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8070f50 msgr2=0x7f1cd8073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:34.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.117+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8070f50 0x7f1cd8073cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40097d0 tx=0x7f1cd402a1c0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.117+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1797734807 shutdown_connections 2024-09-19T05:44:34.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.118+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8071330 0x7f1cd8072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.118+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1797734807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd8070f50 0x7f1cd8073cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.120 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.118+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1797734807 >> 172.21.15.2:0/1797734807 conn(0x7f1cd806cd20 msgr2=0x7f1cd806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:34.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.119+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1797734807 shutdown_connections 2024-09-19T05:44:34.121 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.119+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1797734807 wait complete. 2024-09-19T05:44:34.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.122+0000 7f1ce07f6640 1 Processor -- start 2024-09-19T05:44:34.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.122+0000 7f1ce07f6640 1 -- start start 2024-09-19T05:44:34.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.123+0000 7f1ce07f6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8070f50 0x7f1cd8161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.124+0000 7f1ce07f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cd8071330 0x7f1cd815e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:34.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.124+0000 7f1cddd91640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cd8071330 0x7f1cd815e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:34.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.124+0000 7f1cddd91640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cd8071330 0x7f1cd815e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60198/0 (socket says 172.21.15.2:60198) 2024-09-19T05:44:34.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.124+0000 7f1cddd91640 1 -- 172.21.15.2:0/1043647780 learned_addr learned my addr 172.21.15.2:0/1043647780 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:34.127 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:34.127 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.125+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd81f4ac0 0x7f1cd81f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.125+0000 7f1cde592640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8070f50 0x7f1cd8161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.125+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8163000 con 0x7f1cd8071330 2024-09-19T05:44:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.125+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8162d20 con 0x7f1cd81f4ac0 2024-09-19T05:44:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.125+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd815f5a0 con 0x7f1cd8070f50 2024-09-19T05:44:34.128 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.125+0000 7f1cded93640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd81f4ac0 0x7f1cd81f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.126+0000 7f1cddd91640 1 -- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8070f50 msgr2=0x7f1cd8161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.126+0000 7f1cddd91640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8070f50 0x7f1cd8161ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.126+0000 7f1cddd91640 1 -- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd81f4ac0 msgr2=0x7f1cd81f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.126+0000 7f1cddd91640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd81f4ac0 0x7f1cd81f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.126+0000 7f1cddd91640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd81f5480 con 0x7f1cd8071330 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.127+0000 7f1cddd91640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cd8071330 0x7f1cd815e560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800d8f0 tx=0x7f1cc800ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.127+0000 7f1cceffd640 1 -- 172.21.15.2:0/1043647780 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800b6c0 con 0x7f1cd8071330 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.128+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd8155a00 con 0x7f1cd8071330 2024-09-19T05:44:34.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.128+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81618a0 con 0x7f1cd8071330 2024-09-19T05:44:34.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.128+0000 7f1cceffd640 1 -- 172.21.15.2:0/1043647780 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc8019070 con 0x7f1cd8071330 2024-09-19T05:44:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.130+0000 7f1cceffd640 1 -- 172.21.15.2:0/1043647780 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800ba80 con 0x7f1cd8071330 2024-09-19T05:44:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.130+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cc0005290 con 0x7f1cd8071330 2024-09-19T05:44:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.131+0000 7f1cceffd640 1 -- 172.21.15.2:0/1043647780 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1cc800bc20 con 0x7f1cd8071330 2024-09-19T05:44:34.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.131+0000 7f1cceffd640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cbc06c380 0x7f1cbc06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:34.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.132+0000 7f1cceffd640 1 -- 172.21.15.2:0/1043647780 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5799+0+0 (secure 0 0 0) 0x7f1cc808b8d0 con 0x7f1cd8071330 2024-09-19T05:44:34.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.133+0000 7f1cde592640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cbc06c380 0x7f1cbc06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:34.135 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.133+0000 7f1cde592640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cbc06c380 0x7f1cbc06e840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4034000 tx=0x7f1cd402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:34.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.141+0000 7f1cceffd640 1 -- 172.21.15.2:0/1043647780 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1cd81f5480 con 0x7f1cd8071330 2024-09-19T05:44:34.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:34.188 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:34.186+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:34.267 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:34.264+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:34.273 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:34.274 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:34.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /var/log/ceph/exp.3.18.5 2024-09-19T05:44:34.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:34.397 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:34.394+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:34.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.432+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f1cc0005580 con 0x7f1cd8071330 2024-09-19T05:44:34.457 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:34.457 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:34.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:34.469 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:34.467+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:34.552 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:34.552 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 initial writers = 1269057.25 kB/sec 2024-09-19T05:44:34.552 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 initial writers = 1269028.42 kB/sec 2024-09-19T05:44:34.553 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 1269057.25 kB/sec 2024-09-19T05:44:34.553 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 1269057.25 kB/sec 2024-09-19T05:44:34.553 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 1269057.25 kB/sec 2024-09-19T05:44:34.553 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:44:34.570 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:34.567+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:34.644 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:34.644 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:34.655 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:34.697 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:34.695+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:34.772 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:34.770+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:34.799 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:34.799 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:34.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:34.872 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:34.870+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:34.911 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:34.911 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:34.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.909+0000 7f1cceffd640 1 -- 172.21.15.2:0/1043647780 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v72) ==== 101+0+0 (secure 0 0 0) 0x7f1cc8059f30 con 0x7f1cd8071330 2024-09-19T05:44:34.911 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cbc06c380 msgr2=0x7f1cbc06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cbc06c380 0x7f1cbc06e840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4034000 tx=0x7f1cd402e000 comp rx=0 tx=0).stop 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cd8071330 msgr2=0x7f1cd815e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cd8071330 0x7f1cd815e560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800d8f0 tx=0x7f1cc800ddb0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 shutdown_connections 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1cbc06c380 0x7f1cbc06e840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd81f4ac0 0x7f1cd81f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cd8071330 0x7f1cd815e560 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 --2- 172.21.15.2:0/1043647780 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1cd8070f50 0x7f1cd8161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 >> 172.21.15.2:0/1043647780 conn(0x7f1cd806cd20 msgr2=0x7f1cd80cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 shutdown_connections 2024-09-19T05:44:34.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:34.912+0000 7f1ce07f6640 1 -- 172.21.15.2:0/1043647780 wait complete. 2024-09-19T05:44:34.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:34.973 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:34.971+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:35.024 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:35.024 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:35.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:35.077 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:35.076+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:35.136 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:35.136 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:35.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:35.174 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:35.172+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:35.275 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:35.273+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:35.289 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:35.289 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:35.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:35.375 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:35.373+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:35.400 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:35.400 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:35.409 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:35.476 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:35.474+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:35.540 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:35.581 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:35.579+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:35.677 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:35.675+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:35.689 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:35.689 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:35.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:35.784 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:35.782+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:35.847 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:35.847 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:35.856 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:35.897 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:35.895+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:35.980 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:35.978+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:35.997 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:36.004 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:36.004 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:36.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:36.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.097+0000 7fbbb35ef640 1 Processor -- start 2024-09-19T05:44:36.099 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:36.097+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:36.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.097+0000 7fbbb35ef640 1 -- start start 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb35ef640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd4e0 0x7fbbac0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb35ef640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbbac15aa90 0x7fbbac0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb35ef640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbbac0cd8c0 0x7fbbac15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb35ef640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fbbac15eec0 con 0x7fbbac15aa90 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb35ef640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbbac15ebe0 con 0x7fbbac0cd8c0 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb35ef640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbbac0b2e00 con 0x7fbbac0cd4e0 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbbac15aa90 0x7fbbac0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbbac15aa90 0x7fbbac0b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60208/0 (socket says 172.21.15.2:60208) 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 -- 172.21.15.2:0/394850505 learned_addr learned my addr 172.21.15.2:0/394850505 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb138b640 1 --2- 172.21.15.2:0/394850505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd4e0 0x7fbbac0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/394850505 >> v2:172.21.15.123:3300/0 conn(0x7fbbac0cd8c0 0x7fbbac15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 -- 172.21.15.2:0/394850505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd4e0 msgr2=0x7fbbac0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/394850505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd4e0 0x7fbbac0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 -- 172.21.15.2:0/394850505 >> v2:172.21.15.123:3300/0 conn(0x7fbbac0cd8c0 msgr2=0x7fbbac15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/394850505 >> v2:172.21.15.123:3300/0 conn(0x7fbbac0cd8c0 0x7fbbac15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.098+0000 7fbbb0b8a640 1 -- 172.21.15.2:0/394850505 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac0b30e0 con 0x7fbbac15aa90 2024-09-19T05:44:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.099+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/394850505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 0x7fbbac0b4100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbac15edf0 tx=0x7fbb9c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d07bfd1bb680343 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.099+0000 7fbba3fff640 1 -- 172.21.15.2:0/394850505 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c037070 con 0x7fbbac15aa90 2024-09-19T05:44:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.099+0000 7fbba3fff640 1 -- 172.21.15.2:0/394850505 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb9c032440 con 0x7fbbac15aa90 2024-09-19T05:44:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.099+0000 7fbba3fff640 1 -- 172.21.15.2:0/394850505 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c032810 con 0x7fbbac15aa90 2024-09-19T05:44:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 -- 172.21.15.2:0/394850505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 msgr2=0x7fbbac0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/394850505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 0x7fbbac0b4100 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbbac15edf0 tx=0x7fbb9c02a1c0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 -- 172.21.15.2:0/394850505 shutdown_connections 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/394850505 >> v2:172.21.15.123:3300/0 conn(0x7fbbac0cd8c0 0x7fbbac15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/394850505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 0x7fbbac0b4100 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/394850505 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd4e0 0x7fbbac0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 -- 172.21.15.2:0/394850505 >> 172.21.15.2:0/394850505 conn(0x7fbbac067b30 msgr2=0x7fbbac067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 -- 172.21.15.2:0/394850505 shutdown_connections 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.100+0000 7fbbb35ef640 1 -- 172.21.15.2:0/394850505 wait complete. 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.101+0000 7fbbb35ef640 1 Processor -- start 2024-09-19T05:44:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.101+0000 7fbbb35ef640 1 -- start start 2024-09-19T05:44:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb35ef640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbac0cd4e0 0x7fbbac1e6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb35ef640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd8c0 0x7fbbac1e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb35ef640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 0x7fbbac1f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb35ef640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac15e470 con 0x7fbbac15aa90 2024-09-19T05:44:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb35ef640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac0b36a0 con 0x7fbbac0cd4e0 2024-09-19T05:44:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb35ef640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbbac15e5f0 con 0x7fbbac0cd8c0 2024-09-19T05:44:36.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 0x7fbbac1f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 0x7fbbac1f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60220/0 (socket says 172.21.15.2:60220) 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 -- 172.21.15.2:0/2682096870 learned_addr learned my addr 172.21.15.2:0/2682096870 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd8c0 0x7fbbac1e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbac0cd4e0 0x7fbbac1e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 -- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd8c0 msgr2=0x7fbbac1e6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd8c0 0x7fbbac1e6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 -- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbac0cd4e0 msgr2=0x7fbbac1e6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbac0cd4e0 0x7fbbac1e6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb1b8c640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac1e4c50 con 0x7fbbac15aa90 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbac0cd4e0 0x7fbbac1e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.102+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbbac0cd8c0 0x7fbbac1e6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.103+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbac15aa90 0x7fbbac1f1c30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbba400efc0 tx=0x7fbba400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.103+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4018020 con 0x7fbbac15aa90 2024-09-19T05:44:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.103+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba4014480 con 0x7fbbac15aa90 2024-09-19T05:44:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.103+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac1e3f10 con 0x7fbbac15aa90 2024-09-19T05:44:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.103+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba4014870 con 0x7fbbac15aa90 2024-09-19T05:44:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.103+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1e51a0 con 0x7fbbac15aa90 2024-09-19T05:44:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.104+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbbac15df30 con 0x7fbbac15aa90 2024-09-19T05:44:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.105+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbba4005d10 con 0x7fbbac15aa90 2024-09-19T05:44:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.105+0000 7fbba1ffb640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbb9006c2e0 0x7fbb9006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5799+0+0 (secure 0 0 0) 0x7fbba408c120 con 0x7fbbac15aa90 2024-09-19T05:44:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v72) ==== 54+0+16 (secure 0 0 0) 0x7fbba4022030 con 0x7fbbac15aa90 2024-09-19T05:44:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbb9006c2e0 0x7fbb9006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fbbac1f1670 0x7fbbac0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbac0c83c0 con 0x7fbbac1f1670 2024-09-19T05:44:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fbbac1f1670 0x7fbbac0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbb9006c2e0 0x7fbb9006e7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009eb0 tx=0x7fbba8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.106+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fbbac1f1670 0x7fbbac0c99e0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.107+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbac0c83c0 con 0x7fbbac1f1670 2024-09-19T05:44:36.123 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.121+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbbac0008d0 con 0x7fbbac1f1670 2024-09-19T05:44:36.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.123+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbbac0008d0 con 0x7fbbac1f1670 2024-09-19T05:44:36.125 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.123+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fbbac156ad0 0x7fbbac156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.123+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbbac1e53e0 con 0x7fbbac156ad0 2024-09-19T05:44:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.123+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fbbac156ad0 0x7fbbac156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.124+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fbbac156ad0 0x7fbbac156eb0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.124+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbac1e53e0 con 0x7fbbac156ad0 2024-09-19T05:44:36.141 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.139+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbbac157640 con 0x7fbbac156ad0 2024-09-19T05:44:36.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.141+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbbac157640 con 0x7fbbac156ad0 2024-09-19T05:44:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:36.144 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:36.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.141+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fbbac1f3ed0 0x7fbbac062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.141+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbbac1f42f0 con 0x7fbbac1f3ed0 2024-09-19T05:44:36.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.141+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fbbac1f3ed0 0x7fbbac062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.144 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.142+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fbbac1f3ed0 0x7fbbac062630 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.145 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.142+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbac1f42f0 con 0x7fbbac1f3ed0 2024-09-19T05:44:36.158 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.156+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbbac1f42f0 con 0x7fbbac1f3ed0 2024-09-19T05:44:36.159 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.158+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbbac1f42f0 con 0x7fbbac1f3ed0 2024-09-19T05:44:36.160 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:36.161 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:36.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.158+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fbbac15bb00 0x7fbbac15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.158+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbbac1f42f0 con 0x7fbbac15bb00 2024-09-19T05:44:36.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.158+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fbbac15bb00 0x7fbbac15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.158+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fbbac15bb00 0x7fbbac15bf20 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.162 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.159+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbac1f42f0 con 0x7fbbac15bb00 2024-09-19T05:44:36.164 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:36.164 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:36.165 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:36.165 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:36.165 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:36.173 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:36.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.173+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbbac1f42f0 con 0x7fbbac15bb00 2024-09-19T05:44:36.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.174+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbbac1f42f0 con 0x7fbbac15bb00 2024-09-19T05:44:36.177 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.174+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fbbac02c640 0x7fbbac02ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.174+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbbac15c970 con 0x7fbbac02c640 2024-09-19T05:44:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.175+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fbbac02c640 0x7fbbac02ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.175+0000 7fbbb138b640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fbbac02c640 0x7fbbac02ca60 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.176+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbac15c970 con 0x7fbbac02c640 2024-09-19T05:44:36.181 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:36.179+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:36.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.190+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbbac15c970 con 0x7fbbac02c640 2024-09-19T05:44:36.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.192+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbbac15c970 con 0x7fbbac02c640 2024-09-19T05:44:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:36.194 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.192+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7fbbac02cfa0 con 0x7fbbac15aa90 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.192+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbbac15c970 con 0x7fbbac15aa90 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.192+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fbba4093070 con 0x7fbbac15aa90 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.192+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fbbac204af0 0x7fbbac206ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.192+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fbbac209d30 con 0x7fbbac204af0 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.193+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fbbac204af0 0x7fbbac206ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.193+0000 7fbbb0b8a640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fbbac204af0 0x7fbbac206ee0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.194+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbac209d30 con 0x7fbbac204af0 2024-09-19T05:44:36.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.207+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbbac207420 con 0x7fbbac204af0 2024-09-19T05:44:36.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.209+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbbac207420 con 0x7fbbac204af0 2024-09-19T05:44:36.212 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:36.213 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:36.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.210+0000 7fbbb35ef640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fbbac207b40 0x7fbbac207f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:36.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.210+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fbbac20d0f0 con 0x7fbbac207b40 2024-09-19T05:44:36.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.210+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fbbac207b40 0x7fbbac207f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:36.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.211+0000 7fbbb1b8c640 1 --2- 172.21.15.2:0/2682096870 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fbbac207b40 0x7fbbac207f80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:36.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.211+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbac20d0f0 con 0x7fbbac207b40 2024-09-19T05:44:36.227 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.225+0000 7fbbb35ef640 1 -- 172.21.15.2:0/2682096870 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbbac2085d0 con 0x7fbbac207b40 2024-09-19T05:44:36.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:36.227+0000 7fbba1ffb640 1 -- 172.21.15.2:0/2682096870 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbbac2085d0 con 0x7fbbac207b40 2024-09-19T05:44:36.230 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:36.230 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:36.233 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:36.233 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:36.283 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:36.281+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:36.284 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4172.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 133.40277769400001, 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58043602000000005, 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4172", 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.123:0/311463449", 2024-09-19T05:44:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879045+0000", 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879047+0000", 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879061+0000", 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3636e-05 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879085+0000", 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3699000000000001e-05 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879105+0000", 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879240+0000", 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013564 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:22.879255+0000", 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4586999999999999e-05 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458252+0000", 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57899701100000001 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458410+0000", 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158615 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.458561+0000", 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015053199999999999 2024-09-19T05:44:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459238+0000", 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000677153 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459404+0000", 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016631099999999999 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459450+0000", 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5522999999999998e-05 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:42:23.459481+0000", 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0719000000000002e-05 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:315 3.1c 3:394f8a26:::10000000002.00000038:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.650428511000001, 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55644120600000002, 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 315 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631394+0000", 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.631396+0000", 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5490000000000001e-06 2024-09-19T05:44:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636257+0000", 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048607859999999998 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636331+0000", 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3992000000000006e-05 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636355+0000", 2024-09-19T05:44:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3813999999999999e-05 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636445+0000", 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0180999999999996e-05 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636569+0000", 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123991 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.636849+0000", 2024-09-19T05:44:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000279664 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.659414+0000", 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022565053000000002 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187715+0000", 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.52830067000000003 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187783+0000", 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8509000000000004e-05 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187835+0000", 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1996999999999997e-05 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:348 3.1c 3:3aa14ed1:::10000000002.00000059:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.635453194999997, 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54159870300000001, 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 348 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646369+0000", 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.646372+0000", 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9110000000000002e-06 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.678772+0000", 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032399869999999997 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681572+0000", 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002799714 2024-09-19T05:44:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.681592+0000", 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0298000000000001e-05 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682777+0000", 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011847190000000001 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.682909+0000", 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000132146 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.683102+0000", 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019280499999999999 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:58.702631+0000", 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019529271000000001 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187923+0000", 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48529252 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187948+0000", 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5004999999999999e-05 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:43:59.187968+0000", 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9443999999999999e-05 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3033 3.1f 3:fa1c525e:::100000001f8.000007d6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.825765+0000", 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4560571250000001, 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54453675899999998, 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3033 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.825765+0000", 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.825765+0000", 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.825767+0000", 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2050000000000001e-06 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829926+0000", 2024-09-19T05:44:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041588329999999998 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829951+0000", 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4719000000000002e-05 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829971+0000", 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9901000000000001e-05 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.178316+0000", 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34834517300000001 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.178459+0000", 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142581 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370111+0000", 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19165188899999999 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370185+0000", 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4379000000000001e-05 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370302+0000", 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011707899999999999 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3076 3.27 3:e7ca339f:::100000001f8.00000801:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.830005+0000", 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4518170000000001, 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56816617999999997, 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3076 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.830005+0000", 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.830005+0000", 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.830008+0000", 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4949999999999998e-06 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834860+0000", 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048524659999999997 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834884+0000", 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3502e-05 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834897+0000", 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2756e-05 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.250516+0000", 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41561989100000002 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.258826+0000", 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083093249999999994 2024-09-19T05:44:36.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.398002+0000", 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13917624300000001 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.398071+0000", 2024-09-19T05:44:36.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9383999999999998e-05 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.398172+0000", 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010011800000000001 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3303 3.27 3:e5e0177a:::100000001f8.000008e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.042798+0000", 2024-09-19T05:44:36.315 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2390246549999997, 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59290563399999996, 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3303 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.042798+0000", 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.042798+0000", 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.042801+0000", 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9900000000000002e-06 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.044718+0000", 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00191742 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.044738+0000", 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9539999999999999e-05 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.044751+0000", 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3298999999999999e-05 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.500641+0000", 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.45588980499999998 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.501303+0000", 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00066218099999999997 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.635396+0000", 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13409341899999999 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.635616+0000", 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000219275 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.635703+0000", 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7705000000000001e-05 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.320 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3316 3.27 3:e58ad412:::100000001f8.000008f1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.044779+0000", 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2370431960000001, 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59109450100000005, 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3316 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.044779+0000", 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.044779+0000", 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.044782+0000", 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4710000000000001e-06 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.051833+0000", 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0070516019999999997 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.051849+0000", 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6229e-05 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.051860+0000", 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1037e-05 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.524350+0000", 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47248954100000001 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.524591+0000", 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000240652 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.635762+0000", 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11117093 2024-09-19T05:44:36.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.635805+0000", 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3050000000000003e-05 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.635874+0000", 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8989e-05 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3358 3.27 3:e7c6e76b:::100000001f8.0000091b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.053179+0000", 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2286433949999997, 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61705209900000002, 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3358 2024-09-19T05:44:36.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.053179+0000", 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.053179+0000", 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.053181+0000", 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.131e-06 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.054935+0000", 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017541480000000001 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.060330+0000", 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053951240000000003 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.060341+0000", 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0698e-05 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.546328+0000", 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.485986786 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.546459+0000", 2024-09-19T05:44:36.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000131354 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670050+0000", 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.123591038 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670113+0000", 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2965000000000004e-05 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670231+0000", 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000117855 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3365 3.1f 3:f886b5f6:::100000001f8.00000922:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.060858+0000", 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2209638949999997, 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59499797899999995, 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3365 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.060858+0000", 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.060858+0000", 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.060861+0000", 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5330000000000001e-06 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.065221+0000", 2024-09-19T05:44:36.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043601059999999999 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.074319+0000", 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0090977999999999996 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.074341+0000", 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1926999999999999e-05 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.546069+0000", 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.471728076 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.546220+0000", 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015117700000000001 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.655321+0000", 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.109101051 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.655757+0000", 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00043539099999999998 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.655856+0000", 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9918000000000001e-05 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3372 3.1f 3:f86d9493:::100000001f8.00000929:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.074393+0000", 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.2074296689999997, 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59598563000000004, 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3372 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.074393+0000", 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.074393+0000", 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.074396+0000", 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1099999999999999e-06 2024-09-19T05:44:36.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.077504+0000", 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031080869999999998 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.081804+0000", 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043001280000000003 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.081821+0000", 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7056999999999999e-05 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.590497+0000", 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.50867563500000001 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.591533+0000", 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010364790000000001 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670247+0000", 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.078713886999999996 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670301+0000", 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3593e-05 2024-09-19T05:44:36.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670378+0000", 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7653999999999999e-05 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3451 3.27 3:e774c081:::100000001f8.00000978:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.127016+0000", 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.154806529, 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54359890799999999, 2024-09-19T05:44:36.342 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3451 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.127016+0000", 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.127016+0000", 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.127018+0000", 2024-09-19T05:44:36.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5560000000000002e-06 2024-09-19T05:44:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.128252+0000", 2024-09-19T05:44:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012334259999999999 2024-09-19T05:44:36.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.128275+0000", 2024-09-19T05:44:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3149000000000001e-05 2024-09-19T05:44:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.128284+0000", 2024-09-19T05:44:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2720000000000003e-06 2024-09-19T05:44:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.600552+0000", 2024-09-19T05:44:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47226760600000001 2024-09-19T05:44:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.603563+0000", 2024-09-19T05:44:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030112519999999999 2024-09-19T05:44:36.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670505+0000", 2024-09-19T05:44:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066941731000000004 2024-09-19T05:44:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670547+0000", 2024-09-19T05:44:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2670000000000003e-05 2024-09-19T05:44:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670615+0000", 2024-09-19T05:44:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7246e-05 2024-09-19T05:44:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3459 3.1f 3:f909e746:::100000001f8.00000980:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.137878+0000", 2024-09-19T05:44:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1439446350000004, 2024-09-19T05:44:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53261951399999996, 2024-09-19T05:44:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3459 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137878+0000", 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137878+0000", 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137882+0000", 2024-09-19T05:44:36.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2069999999999997e-06 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.141966+0000", 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040836400000000004 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.141992+0000", 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6115e-05 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.142023+0000", 2024-09-19T05:44:36.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1405000000000002e-05 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.615417+0000", 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.47339371400000002 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.615609+0000", 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019211200000000001 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670387+0000", 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054778483000000003 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670430+0000", 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2648000000000002e-05 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670497+0000", 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7189999999999999e-05 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.357 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3488 3.1f 3:fb82176e:::100000001f8.0000099d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.155029+0000", 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1267933619999999, 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57281516399999999, 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3488 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.155029+0000", 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.155029+0000", 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.155032+0000", 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7889999999999999e-06 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.157114+0000", 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020827150000000002 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.157136+0000", 2024-09-19T05:44:36.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1316000000000001e-05 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.157146+0000", 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0429e-05 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.655912+0000", 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49876582400000002 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.656092+0000", 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017948199999999999 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.727580+0000", 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.071488595000000002 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.727744+0000", 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016385299999999999 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.727844+0000", 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010016100000000001 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3492 3.27 3:e7b93799:::100000001f8.000009a1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.157166+0000", 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1246566580000001, 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58584006200000005, 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3492 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.157166+0000", 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.157166+0000", 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.157168+0000", 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.39e-06 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.159829+0000", 2024-09-19T05:44:36.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002660993 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.159849+0000", 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0092e-05 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.159862+0000", 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2442999999999999e-05 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670790+0000", 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51092874700000002 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.670962+0000", 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000171812 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.742719+0000", 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.071757099000000005 2024-09-19T05:44:36.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.742887+0000", 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016769500000000001 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743006+0000", 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011879099999999999 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3493 3.27 3:e77fb6c3:::100000001f8.000009a2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.169105+0000", 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1127173829999997, 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.62450148000000005, 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3493 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.169105+0000", 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.169105+0000", 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.170607+0000", 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015016960000000001 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.173822+0000", 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032156400000000001 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.173849+0000", 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6730999999999999e-05 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.173861+0000", 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1477e-05 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.712770+0000", 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53890961599999998 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.712931+0000", 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016092999999999999 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793403+0000", 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.080471645999999994 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793477+0000", 2024-09-19T05:44:36.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4791999999999998e-05 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793606+0000", 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012895199999999999 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3501 3.27 3:e4baa440:::100000001f8.000009aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.173883+0000", 2024-09-19T05:44:36.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1079393309999999, 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61984682499999999, 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3501 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.173883+0000", 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.173883+0000", 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.173885+0000", 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4310000000000001e-06 2024-09-19T05:44:36.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.177848+0000", 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039622360000000001 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.177868+0000", 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9995000000000001e-05 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.177881+0000", 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.293e-05 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.726667+0000", 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.548785949 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.726835+0000", 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016859199999999999 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793624+0000", 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.066788611999999997 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793666+0000", 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2329000000000001e-05 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793730+0000", 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3751000000000007e-05 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.379 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3549 3.1f 3:f99124f5:::100000001f8.000009da:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.202800+0000", 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0790223540000001, 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59105411100000005, 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3549 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.202800+0000", 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.202800+0000", 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.202802+0000", 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4380000000000002e-06 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.211296+0000", 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084937160000000001 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240672+0000", 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029375801999999999 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240689+0000", 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6954000000000001e-05 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.727948+0000", 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.48725898200000001 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.728097+0000", 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014919600000000001 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793746+0000", 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065648675000000004 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793787+0000", 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1075999999999998e-05 2024-09-19T05:44:36.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793854+0000", 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7272000000000002e-05 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3554 3.1f 3:f86e50aa:::100000001f8.000009df:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.240725+0000", 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0410973339999998, 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55323707899999996, 2024-09-19T05:44:36.385 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3554 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240725+0000", 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240725+0000", 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240729+0000", 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8240000000000001e-06 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.251166+0000", 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010437352 2024-09-19T05:44:36.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.251178+0000", 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2252000000000001e-05 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.251187+0000", 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9069999999999996e-06 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.742781+0000", 2024-09-19T05:44:36.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49159392600000001 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743148+0000", 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036638300000000002 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793863+0000", 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.050715231 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793901+0000", 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7978999999999997e-05 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793962+0000", 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1224999999999997e-05 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3557 3.1f 3:fa9fa4f6:::100000001f8.000009e2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.251208+0000", 2024-09-19T05:44:36.390 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0306143560000001, 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54286458999999998, 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3557 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.251208+0000", 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.251208+0000", 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.251210+0000", 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8560000000000001e-06 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.261422+0000", 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010212129 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.261449+0000", 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7138e-05 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.261465+0000", 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5430999999999998e-05 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.753257+0000", 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.49179292099999999 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.753428+0000", 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017048200000000001 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.793970+0000", 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040542474000000002 2024-09-19T05:44:36.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.794009+0000", 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9059e-05 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.794073+0000", 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3100000000000002e-05 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:3574 3.1f 3:fb77f77c:::100000001f8.000009f3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.268013+0000", 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0138096189999999, 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54493317600000002, 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 3574 2024-09-19T05:44:36.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.268013+0000", 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.268013+0000", 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.268074+0000", 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0912000000000002e-05 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.271952+0000", 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003878233 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.271973+0000", 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1228000000000001e-05 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.271987+0000", 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4270999999999999e-05 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.785540+0000", 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.51355232900000003 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.785699+0000", 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015947800000000001 2024-09-19T05:44:36.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.809586+0000", 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023886599000000001 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.812846+0000", 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032597899999999998 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.812946+0000", 2024-09-19T05:44:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100336 2024-09-19T05:44:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.401 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.401 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:36.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:36.403 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:36.400+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:36.503 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:36.501+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:36.551 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:36.551 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:36.551 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:36.551 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:36.560 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:36.604 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:36.603+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:36.667 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:36.667 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:36.667 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:36.675 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:36.721 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:36.719+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:36.786 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2140 3.3c 3:3ebd280f:::100000001f8.00000459:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.1903980389999997, 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0038021129999999, 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2140 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641694+0000", 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049504800000000002 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646652+0000", 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049580040000000002 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646670+0000", 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646681+0000", 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1555e-05 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429025+0000", 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.782343268 2024-09-19T05:44:36.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429156+0000", 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013093299999999999 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.435554+0000", 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063981860000000002 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549237+0000", 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113683458 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595633+0000", 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046396021000000003 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595950+0000", 2024-09-19T05:44:36.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000316514 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595991+0000", 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1430000000000001e-05 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2232 2.14 2:2e2fa760:::200.00000001:head [write 2396859~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.793 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9796836769999997, 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9689986130000001, 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802904+0000", 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-07 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802912+0000", 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9489999999999996e-06 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802920+0000", 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5970000000000003e-06 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802925+0000", 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5040000000000002e-06 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730249+0000", 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9273239709999999 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730352+0000", 2024-09-19T05:44:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102648 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730514+0000", 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016198499999999999 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755135+0000", 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024621601 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771776+0000", 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016640773000000001 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771841+0000", 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5178000000000005e-05 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771902+0000", 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0825999999999997e-05 2024-09-19T05:44:36.799 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2233 2.14 2:2e2fa760:::200.00000001:head [write 2398939~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.979653581, 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969058076, 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807300+0000", 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043666679999999998 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807312+0000", 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2199e-05 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807327+0000", 2024-09-19T05:44:36.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4260999999999999e-05 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807339+0000", 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737540+0000", 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9302006619999998 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737660+0000", 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119881 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737878+0000", 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218614 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755190+0000", 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017311756000000001 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771913+0000", 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016722954000000002 2024-09-19T05:44:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771958+0000", 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123000000000001e-05 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771992+0000", 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3657e-05 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2234 2.14 2:2e2fa760:::200.00000001:head [write 2401019~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9751998290000001, 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9661180190000001, 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807388+0000", 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-07 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807399+0000", 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1338999999999999e-05 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807409+0000", 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:44:36.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807416+0000", 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1180000000000003e-06 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743599+0000", 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9361828380000001 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743702+0000", 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103303 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743868+0000", 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771998+0000", 2024-09-19T05:44:36.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028130420999999999 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773391+0000", 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013930699999999999 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773448+0000", 2024-09-19T05:44:36.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6672999999999998e-05 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773505+0000", 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7445000000000003e-05 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.813 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2235 2.14 2:2e2fa760:::200.00000001:head [write 2403099~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9751606150000001, 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9662057229999998, 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:36.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4000000000000002e-07 2024-09-19T05:44:36.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807435+0000", 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.072e-06 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807478+0000", 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2460999999999999e-05 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807486+0000", 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5350000000000004e-06 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.745914+0000", 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9384274700000002 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.747669+0000", 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017549239999999999 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.748294+0000", 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00062547500000000001 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773338+0000", 2024-09-19T05:44:36.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025043950999999998 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773551+0000", 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021271199999999999 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773599+0000", 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7876e-05 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773632+0000", 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3707000000000001e-05 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2236 2.14 2:2e2fa760:::200.00000001:head [write 2405179~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:36.820 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9750904980000001, 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9899550850000001, 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807502+0000", 2024-09-19T05:44:36.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4850000000000003e-06 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807509+0000", 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0859999999999996e-06 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807515+0000", 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9179999999999999e-06 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755230+0000", 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9477156679999998 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755342+0000", 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111703 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755522+0000", 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000179937 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.783617+0000", 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028095472 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797341+0000", 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013723506999999999 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797404+0000", 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797452+0000", 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7567e-05 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2237 2.14 2:2e2fa760:::200.00000001:head [write 2407259~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9750658249999997, 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0092068749999998, 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:36.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807522+0000", 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807530+0000", 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8930000000000005e-06 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807577+0000", 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7064999999999997e-05 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807606+0000", 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8963e-05 2024-09-19T05:44:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761580+0000", 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.953974901 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761685+0000", 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010435800000000001 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761827+0000", 2024-09-19T05:44:36.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142313 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792591+0000", 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030764073999999999 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816632+0000", 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024040371000000001 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816688+0000", 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5951e-05 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816728+0000", 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0636000000000002e-05 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2238 2.14 2:2e2fa760:::200.00000001:head [write 2409339~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9749679699999998, 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0122208439999998, 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834275+0000", 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026656197999999999 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834287+0000", 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1691e-05 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834301+0000", 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3974e-05 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834310+0000", 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8659999999999995e-06 2024-09-19T05:44:36.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772040+0000", 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9377302950000002 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772148+0000", 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108136 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772314+0000", 2024-09-19T05:44:36.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016526400000000001 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792597+0000", 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020283483000000001 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819691+0000", 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027094025000000001 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819771+0000", 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0022000000000001e-05 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819840+0000", 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8889999999999999e-05 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.838 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2239 2.14 2:2e2fa760:::200.00000001:head [write 2411419~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9482579869999999, 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.985646596, 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851429+0000", 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017099436999999999 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851444+0000", 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5449000000000001e-05 2024-09-19T05:44:36.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851460+0000", 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.556e-05 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851470+0000", 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0001e-05 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773666+0000", 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9221966369999999 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773757+0000", 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1160000000000001e-05 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773902+0000", 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014419500000000001 2024-09-19T05:44:36.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795500+0000", 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021598346000000001 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819898+0000", 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024398474 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819942+0000", 2024-09-19T05:44:36.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3151000000000001e-05 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819976+0000", 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4186000000000001e-05 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2240 2.14 2:2e2fa760:::200.00000001:head [write 2413499~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:36.844 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9310992650000003, 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9685921560000001, 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862188+0000", 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010699774 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862213+0000", 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5536000000000001e-05 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862225+0000", 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1328000000000001e-05 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862231+0000", 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.55e-06 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.780995+0000", 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.918763854 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781115+0000", 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120538 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781261+0000", 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145785 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795505+0000", 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014244158 2024-09-19T05:44:36.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820012+0000", 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024506607 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820051+0000", 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8597000000000003e-05 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820080+0000", 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9428999999999999e-05 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2245 2.14 2:2e2fa760:::200.00000001:head [write 2423899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9202233499999997, 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969609441, 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.08e-07 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862368+0000", 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7929999999999999e-06 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862375+0000", 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6680000000000004e-06 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862380+0000", 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797491+0000", 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9351107490000001 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797581+0000", 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9559000000000001e-05 2024-09-19T05:44:36.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797723+0000", 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014230199999999999 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820590+0000", 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022867386 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827763+0000", 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071727190000000001 2024-09-19T05:44:36.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.828797+0000", 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010343570000000001 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831973+0000", 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003175925 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2246 2.14 2:2e2fa760:::200.00000001:head [write 2425979~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9201994359999999, 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9696936549999999, 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.96e-07 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862397+0000", 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-19T05:44:36.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862404+0000", 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862409+0000", 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1320000000000002e-06 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804733+0000", 2024-09-19T05:44:36.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.94232443 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804840+0000", 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106724 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804998+0000", 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015801500000000001 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826045+0000", 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021046664999999999 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831983+0000", 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059387570000000002 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832038+0000", 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4475999999999998e-05 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832081+0000", 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3591000000000003e-05 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2247 2.14 2:2e2fa760:::200.00000001:head [write 2428059~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9201723550000001, 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969738048, 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862426+0000", 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1090999999999999e-05 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862433+0000", 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8070000000000002e-06 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862438+0000", 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1599999999999997e-06 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806006+0000", 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.943567885 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806105+0000", 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8782000000000004e-05 2024-09-19T05:44:36.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806289+0000", 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183678 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827739+0000", 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021450258999999999 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832087+0000", 2024-09-19T05:44:36.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043483280000000003 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832124+0000", 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6816000000000001e-05 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832153+0000", 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.887e-05 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2248 2.14 2:2e2fa760:::200.00000001:head [write 2430139~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9201423569999996, 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9698627229999999, 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877451+0000", 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015005956000000001 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877461+0000", 2024-09-19T05:44:36.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0034000000000001e-05 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.896137+0000", 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018675967000000002 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909217+0000", 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013080629999999999 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807553+0000", 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8983351339999999 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807647+0000", 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4289000000000005e-05 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807794+0000", 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000147467 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832160+0000", 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024365753 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832224+0000", 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3689000000000006e-05 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832274+0000", 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0649000000000002e-05 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832308+0000", 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3154999999999997e-05 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2250 2.14 2:2e2fa760:::200.00000001:head [write 2434299~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8727015070000004, 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9595307040000001, 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939724+0000", 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029838587 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939743+0000", 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9077e-05 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941640+0000", 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018968310000000001 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941651+0000", 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0379e-05 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814230+0000", 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.872579934 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814430+0000", 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199481 2024-09-19T05:44:36.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814571+0000", 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014074399999999999 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832173+0000", 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017602199999999998 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869325+0000", 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037152223999999998 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869374+0000", 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9149e-05 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869416+0000", 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2098000000000002e-05 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2337 3.3c 3:3eebf982:::100000001f8.0000051e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.770555367, 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.014058033, 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2337 2024-09-19T05:44:36.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.018343+0000", 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063114770000000002 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030374+0000", 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012031086999999999 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030396+0000", 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.156e-05 2024-09-19T05:44:36.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030411+0000", 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4875000000000001e-05 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897205+0000", 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8667940299999999 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897418+0000", 2024-09-19T05:44:36.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021300400000000001 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897573+0000", 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015481300000000001 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.943251+0000", 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045678424000000002 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026021+0000", 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082769669000000004 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026059+0000", 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8296000000000001e-05 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026090+0000", 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0797999999999999e-05 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2267 2.14 2:2e2fa760:::200.00000001:head [write 2469659~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.713913099, 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9703313549999999, 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076283+0000", 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076092460000000001 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076295+0000", 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1857e-05 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076309+0000", 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3661e-05 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076319+0000", 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0589e-05 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963409+0000", 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.887089915 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963490+0000", 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1137000000000006e-05 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963615+0000", 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012412999999999999 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012633+0000", 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049018171999999999 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038886+0000", 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026253216999999999 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038950+0000", 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3876999999999998e-05 2024-09-19T05:44:36.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039005+0000", 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5553999999999997e-05 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2268 2.14 2:2e2fa760:::200.00000001:head [write 2471739~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.7062376009999998, 2024-09-19T05:44:36.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962740785, 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076361+0000", 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076368+0000", 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4240000000000004e-06 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076374+0000", 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0179999999999998e-06 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975236+0000", 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.898862345 2024-09-19T05:44:36.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975332+0000", 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5217999999999995e-05 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975458+0000", 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125986 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012673+0000", 2024-09-19T05:44:36.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037215224999999998 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039014+0000", 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026341256 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039056+0000", 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1944999999999999e-05 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039090+0000", 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4301999999999998e-05 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2269 2.14 2:2e2fa760:::200.00000001:head [write 2473819~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.7062040429999996, 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9627868230000001, 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076384+0000", 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076392+0000", 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.174e-06 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076398+0000", 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4309999999999999e-06 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076403+0000", 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6399999999999996e-06 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976688+0000", 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9002852180000001 2024-09-19T05:44:36.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976785+0000", 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6963e-05 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976951+0000", 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016594799999999999 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012708+0000", 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035757376 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039097+0000", 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026388551 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039137+0000", 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0025999999999999e-05 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039170+0000", 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3065000000000003e-05 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2270 2.14 2:2e2fa760:::200.00000001:head [write 2475899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.7061780779999998, 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962839365, 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:36.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102351+0000", 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025941703999999999 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102359+0000", 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102372+0000", 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3749e-05 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:36.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102382+0000", 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0229999999999998e-06 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978315+0000", 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.875933294 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978422+0000", 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107007 2024-09-19T05:44:36.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978575+0000", 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015296099999999999 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012746+0000", 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034170849000000003 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039176+0000", 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026430282999999999 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039215+0000", 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9368e-05 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039248+0000", 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3157000000000001e-05 2024-09-19T05:44:36.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:36.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:36.908 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:36.910 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:36.910 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:36.908+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:36.928 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:44:37.010 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:37.008+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:37.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.054+0000 7fb4bd677640 1 Processor -- start 2024-09-19T05:44:37.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.055+0000 7fb4bd677640 1 -- start start 2024-09-19T05:44:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.055+0000 7fb4bd677640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b81584c0 0x7fb4b80c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.055+0000 7fb4bd677640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb4b80c86f0 0x7fb4b815e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.055+0000 7fb4bd677640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b80725c0 con 0x7fb4b8073660 2024-09-19T05:44:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.055+0000 7fb4bd677640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b8072370 con 0x7fb4b80c86f0 2024-09-19T05:44:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.055+0000 7fb4bd677640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b80728a0 con 0x7fb4b81584c0 2024-09-19T05:44:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4b77fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb4b80c86f0 0x7fb4b815e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:37.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4affff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb4b8073660 0x7fb4b806bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:37544/0 (socket says 172.21.15.2:37544) 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4affff640 1 -- 172.21.15.2:0/3441949910 learned_addr learned my addr 172.21.15.2:0/3441949910 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4b6ffd640 1 --2- 172.21.15.2:0/3441949910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b81584c0 0x7fb4b80c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4b77fe640 1 -- 172.21.15.2:0/3441949910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b81584c0 msgr2=0x7fb4b80c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4b77fe640 1 --2- 172.21.15.2:0/3441949910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b81584c0 0x7fb4b80c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4b77fe640 1 -- 172.21.15.2:0/3441949910 >> v1:172.21.15.2:6789/0 conn(0x7fb4b8073660 legacy=0x7fb4b806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.056+0000 7fb4b77fe640 1 -- 172.21.15.2:0/3441949910 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b8072b80 con 0x7fb4b80c86f0 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4b77fe640 1 --2- 172.21.15.2:0/3441949910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b80c86f0 0x7fb4b815e330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009d90 tx=0x7fb4b002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522965bc6f0838dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4b67fc640 1 -- 172.21.15.2:0/3441949910 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0007740 con 0x7fb4b80c86f0 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4b67fc640 1 -- 172.21.15.2:0/3441949910 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b00078e0 con 0x7fb4b80c86f0 2024-09-19T05:44:37.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4b67fc640 1 -- 172.21.15.2:0/3441949910 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0007cb0 con 0x7fb4b80c86f0 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4bd677640 1 -- 172.21.15.2:0/3441949910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b80c86f0 msgr2=0x7fb4b815e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4bd677640 1 --2- 172.21.15.2:0/3441949910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b80c86f0 0x7fb4b815e330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009d90 tx=0x7fb4b002a530 comp rx=0 tx=0).stop 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4bd677640 1 -- 172.21.15.2:0/3441949910 shutdown_connections 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4bd677640 1 --2- 172.21.15.2:0/3441949910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b80c86f0 0x7fb4b815e330 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4bd677640 1 --2- 172.21.15.2:0/3441949910 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b81584c0 0x7fb4b80c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.057+0000 7fb4bd677640 1 -- 172.21.15.2:0/3441949910 >> 172.21.15.2:0/3441949910 conn(0x7fb4b80cd470 msgr2=0x7fb4b80c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.058+0000 7fb4bd677640 1 -- 172.21.15.2:0/3441949910 shutdown_connections 2024-09-19T05:44:37.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.058+0000 7fb4bd677640 1 -- 172.21.15.2:0/3441949910 wait complete. 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 Processor -- start 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 -- start start 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b8073660 0x7fb4b8155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b80c86f0 0x7fb4b814b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b81584c0 0x7fb4b81464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b81588a0 con 0x7fb4b81584c0 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8074120 con 0x7fb4b80c86f0 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4bd677640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b8158a20 con 0x7fb4b8073660 2024-09-19T05:44:37.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.059+0000 7fb4b77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b81584c0 0x7fb4b81464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:37.064 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:37.064 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b81584c0 0x7fb4b81464b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60244/0 (socket says 172.21.15.2:60244) 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 -- 172.21.15.2:0/2605982100 learned_addr learned my addr 172.21.15.2:0/2605982100 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b6ffd640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b8073660 0x7fb4b8155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4affff640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b80c86f0 0x7fb4b814b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 -- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b8073660 msgr2=0x7fb4b8155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b8073660 0x7fb4b8155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 -- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b80c86f0 msgr2=0x7fb4b814b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b80c86f0 0x7fb4b814b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 -- 172.21.15.2:0/2605982100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b8147000 con 0x7fb4b81584c0 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4b77fe640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b81584c0 0x7fb4b81464b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002ab20 tx=0x7fb4b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4af7fe640 1 -- 172.21.15.2:0/2605982100 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0040070 con 0x7fb4b81584c0 2024-09-19T05:44:37.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4af7fe640 1 -- 172.21.15.2:0/2605982100 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b0031bc0 con 0x7fb4b81584c0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b81f3d10 con 0x7fb4b81584c0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4af7fe640 1 -- 172.21.15.2:0/2605982100 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0045460 con 0x7fb4b81584c0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.060+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b8154980 con 0x7fb4b81584c0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.061+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b80c4400 con 0x7fb4b81584c0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.062+0000 7fb4af7fe640 1 -- 172.21.15.2:0/2605982100 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb4b0039040 con 0x7fb4b81584c0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.062+0000 7fb4af7fe640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb49806c2b0 0x7fb49806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.062+0000 7fb4af7fe640 1 -- 172.21.15.2:0/2605982100 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5799+0+0 (secure 0 0 0) 0x7fb4b00acbd0 con 0x7fb4b81584c0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.062+0000 7fb4b6ffd640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb49806c2b0 0x7fb49806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:37.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.063+0000 7fb4b6ffd640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb49806c2b0 0x7fb49806e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80079a0 tx=0x7fb4a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:37.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.067+0000 7fb4af7fe640 1 -- 172.21.15.2:0/2605982100 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fb4b007ae40 con 0x7fb4b81584c0 2024-09-19T05:44:37.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:37.113 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:37.111+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:37.212 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:37.210+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:37.219 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:37.220 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:37.228 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:37.256 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.254+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fb4b8155890 con 0x7fb4b81584c0 2024-09-19T05:44:37.313 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:37.311+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:37.364 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:37.364 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2653 3.9 3:916fb10b:::100000001f8.0000065a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.799828+0000", 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.5618331159999999, 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1159312100000001, 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2653 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.799828+0000", 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.799828+0000", 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.799830+0000", 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4439999999999998e-06 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.841194+0000", 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041363977000000003 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.841211+0000", 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7343e-05 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.841223+0000", 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1669e-05 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.851903+0000", 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010679953000000001 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.853016+0000", 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011128010000000001 2024-09-19T05:44:37.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.853207+0000", 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019156500000000001 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.879156+0000", 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025949014999999999 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.915650+0000", 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0364931740000001 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.915707+0000", 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7129999999999997e-05 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.915759+0000", 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2138999999999997e-05 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2666 3.9 3:92ca8d9b:::100000001f8.00000667:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.4669362079999999, 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3293169570000001, 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2666 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894727+0000", 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8439999999999998e-06 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897799+0000", 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030718199999999998 2024-09-19T05:44:37.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897817+0000", 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7847999999999999e-05 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897831+0000", 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3845999999999999e-05 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897938+0000", 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010742399999999999 2024-09-19T05:44:37.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898111+0000", 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017279099999999999 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898360+0000", 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024885800000000002 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956644+0000", 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058284217999999999 2024-09-19T05:44:37.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223960+0000", 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.267316119 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224004+0000", 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4039000000000003e-05 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224042+0000", 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7150000000000002e-05 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2667 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.377 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.4637963599999999, 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3258909560000001, 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2667 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897867+0000", 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9299999999999999e-06 2024-09-19T05:44:37.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902567+0000", 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004699356 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902588+0000", 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1631e-05 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902602+0000", 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3478e-05 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.906894+0000", 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042925760000000002 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907078+0000", 2024-09-19T05:44:37.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183067 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907289+0000", 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021123499999999999 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956671+0000", 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049382078000000003 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223652+0000", 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.266981122 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223704+0000", 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2244999999999999e-05 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223755+0000", 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1238e-05 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2673 3.2d 3:b45a3f48:::100000001f8.0000066e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.4507989329999997, 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.373980822, 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2673 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910864+0000", 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915100+0000", 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042352259999999999 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915123+0000", 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3334999999999999e-05 2024-09-19T05:44:37.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915137+0000", 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4178999999999999e-05 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915220+0000", 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2693000000000003e-05 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915395+0000", 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017551099999999999 2024-09-19T05:44:37.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915574+0000", 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178357 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951064+0000", 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035490116000000002 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284764+0000", 2024-09-19T05:44:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.333700259 2024-09-19T05:44:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284808+0000", 2024-09-19T05:44:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3813999999999998e-05 2024-09-19T05:44:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284843+0000", 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4807000000000001e-05 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2700 3.30 3:0e790ae4:::100000001f8.00000689:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.4180968309999997, 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.25937777, 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2700 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943567+0000", 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.083e-06 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946402+0000", 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028349439999999998 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946419+0000", 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7105999999999999e-05 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946429+0000", 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8709999999999992e-06 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.947951+0000", 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015219960000000001 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948153+0000", 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020153400000000001 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948415+0000", 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026250200000000002 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.064312+0000", 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115897369 2024-09-19T05:44:37.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202811+0000", 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.138498891 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202872+0000", 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0529000000000003e-05 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202942+0000", 2024-09-19T05:44:37.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9944999999999995e-05 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2711 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.410579222, 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.27282242, 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2711 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951084+0000", 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.553e-06 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956906+0000", 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058222400000000002 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956926+0000", 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9619e-05 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956937+0000", 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0594999999999999e-05 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958546+0000", 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001609369 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958792+0000", 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024587600000000002 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.959831+0000", 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010394079999999999 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293509+0000", 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33367725100000001 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223817+0000", 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93030886999999995 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223864+0000", 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6465000000000003e-05 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223904+0000", 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0173999999999999e-05 2024-09-19T05:44:37.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2732 3.9 3:92ca8380:::100000001f8.000006a9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3537096179999999, 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2619013319999999, 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2732 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007954+0000", 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018132+0000", 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010177734000000001 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018148+0000", 2024-09-19T05:44:37.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6555999999999999e-05 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018187+0000", 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8760000000000002e-05 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020347+0000", 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021597589999999998 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020514+0000", 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016697899999999999 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020695+0000", 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018172199999999999 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.375158+0000", 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35446218200000001 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269697+0000", 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89453943599999997 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269768+0000", 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0619999999999998e-05 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269853+0000", 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4876000000000005e-05 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.410 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2746 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3261724060000004, 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1890195509999999, 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2746 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035491+0000", 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043360+0000", 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0078693240000000005 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043383+0000", 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2853e-05 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.045231+0000", 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001847476 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.122884+0000", 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077653244999999996 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123275+0000", 2024-09-19T05:44:37.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039126200000000002 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123895+0000", 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061987200000000004 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293518+0000", 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.169623147 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224430+0000", 2024-09-19T05:44:37.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93091173699999996 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224474+0000", 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3816000000000002e-05 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224508+0000", 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4369000000000002e-05 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2753 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.3001495690000002, 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.188208317, 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2753 2024-09-19T05:44:37.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061513+0000", 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.871e-06 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078405+0000", 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016891461 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078433+0000", 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8436e-05 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078447+0000", 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3607e-05 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137285+0000", 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058837911 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137457+0000", 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017199399999999999 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137637+0000", 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180173 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.460405+0000", 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.322768426 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249610+0000", 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78920498100000003 2024-09-19T05:44:37.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249664+0000", 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3668000000000002e-05 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249720+0000", 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5788999999999997e-05 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.422 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2769 3.11 3:8be9d197:::100000001f8.000006ce:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2588970750000001, 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1472854269999999, 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2769 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102767+0000", 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.021e-06 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136259+0000", 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033491981999999997 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136291+0000", 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1773999999999998e-05 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136302+0000", 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1154e-05 2024-09-19T05:44:37.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.239524+0000", 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103221854 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240715+0000", 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001191042 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.242825+0000", 2024-09-19T05:44:37.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002110289 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534133+0000", 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29130808400000002 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249969+0000", 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71583560999999996 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.250018+0000", 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9459000000000002e-05 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.250049+0000", 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1158000000000002e-05 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2773 3.9 3:9102af4b:::100000001f8.000006d2:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:37.428 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.2157166080000001, 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124067851, 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2773 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145947+0000", 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.162e-06 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149477+0000", 2024-09-19T05:44:37.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035300800000000001 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149524+0000", 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6264999999999998e-05 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149539+0000", 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5169e-05 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234044+0000", 2024-09-19T05:44:37.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.084504863999999999 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234273+0000", 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022875599999999999 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234509+0000", 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023624799999999999 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534140+0000", 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29963108999999999 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269911+0000", 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73577157699999995 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269966+0000", 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4512000000000003e-05 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.270012+0000", 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6128000000000003e-05 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2796 3.9 3:9157a25d:::100000001f8.000006e9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:37.434 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1319603809999998, 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0747865430000001, 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2796 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:37.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229704+0000", 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1860000000000001e-06 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234027+0000", 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043232710000000001 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234040+0000", 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3485999999999999e-05 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234051+0000", 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1022999999999999e-05 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364098+0000", 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.130046151 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364248+0000", 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150793 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364444+0000", 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195735 2024-09-19T05:44:37.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534144+0000", 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16970036399999999 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.304323+0000", 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77017807800000004 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.304400+0000", 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7881999999999997e-05 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.304487+0000", 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6574000000000002e-05 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2808 3.11 3:8bd5b656:::100000001f8.000006f5:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:37.440 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1215333059999999, 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.160580451, 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2808 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:37.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240131+0000", 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244285+0000", 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041546919999999998 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244303+0000", 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7646999999999999e-05 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244316+0000", 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2931999999999999e-05 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373296+0000", 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12897989000000001 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373463+0000", 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166847 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373621+0000", 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015846400000000001 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.585995+0000", 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.212374013 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400581+0000", 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81458587900000001 2024-09-19T05:44:37.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400646+0000", 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5036999999999994e-05 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400708+0000", 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1937e-05 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2821 3.9 3:9040d340:::100000001f8.00000702:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.1045310549999998, 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.06449839, 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2821 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257133+0000", 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.146e-06 2024-09-19T05:44:37.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266617+0000", 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0094844509999999996 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266644+0000", 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6942999999999999e-05 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266660+0000", 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5379e-05 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459066+0000", 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19240588 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459208+0000", 2024-09-19T05:44:37.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014251699999999999 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459363+0000", 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000154535 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786586+0000", 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32722325099999999 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.321507+0000", 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53492102799999997 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.321586+0000", 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8886000000000003e-05 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.321628+0000", 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2373999999999998e-05 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2858 3.9 3:92f6ecc1:::100000001f8.00000727:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0553334200000002, 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0285565109999999, 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2858 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306329+0000", 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0719999999999998e-06 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311583+0000", 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052534000000000001 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311607+0000", 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3716000000000001e-05 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311625+0000", 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8681999999999998e-05 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.528241+0000", 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.216615688 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.528918+0000", 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067717000000000005 2024-09-19T05:44:37.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.529096+0000", 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017772600000000001 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786593+0000", 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25749734800000001 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.334738+0000", 2024-09-19T05:44:37.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54814497299999998 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.334814+0000", 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6044000000000006e-05 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.334884+0000", 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9691999999999996e-05 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2862 3.9 3:907f8708:::100000001f8.0000072b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0499931440000001, 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.023459962, 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2862 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311670+0000", 2024-09-19T05:44:37.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316240+0000", 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045697189999999999 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316271+0000", 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1341999999999999e-05 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316324+0000", 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2984e-05 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534179+0000", 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21785516099999999 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534368+0000", 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018821699999999999 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534552+0000", 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018435599999999999 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786599+0000", 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25204730600000003 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335039+0000", 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54843970500000006 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335084+0000", 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5235999999999999e-05 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335128+0000", 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3479999999999997e-05 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2877 3.9 3:91dc4dd5:::100000001f8.0000073a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0294037080000003, 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0029870350000001, 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2877 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332260+0000", 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9689999999999999e-06 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337127+0000", 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048665890000000002 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337155+0000", 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8632e-05 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337203+0000", 2024-09-19T05:44:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7228000000000003e-05 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586056+0000", 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24885332199999999 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586595+0000", 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053866000000000005 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586768+0000", 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017356199999999999 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786605+0000", 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19983721300000001 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335170+0000", 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54856438100000005 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335211+0000", 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1696000000000002e-05 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335244+0000", 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2783000000000001e-05 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2895 3.9 3:939d0220:::100000001f8.0000074c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0077598630000004, 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0565064120000001, 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2895 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353903+0000", 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356485+0000", 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025819390000000001 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356507+0000", 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2109000000000001e-05 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356521+0000", 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3560999999999999e-05 2024-09-19T05:44:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.696659+0000", 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34013843500000002 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.696845+0000", 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185873 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.698377+0000", 2024-09-19T05:44:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015312240000000001 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.805489+0000", 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10711206199999999 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.410285+0000", 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60479590599999999 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.410350+0000", 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5244000000000001e-05 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.410407+0000", 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7589e-05 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2908 3.11 3:8a4f5bd7:::100000001f8.00000759:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.0032175890000001, 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1380135730000001, 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2908 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358445+0000", 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.08e-06 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362030+0000", 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035843250000000002 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362050+0000", 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9860999999999998e-05 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362063+0000", 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3145999999999999e-05 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.715923+0000", 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35386083800000001 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.734420+0000", 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018496133000000001 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.734712+0000", 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029234500000000001 2024-09-19T05:44:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852483+0000", 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11777127499999999 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.496362+0000", 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.643879058 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.496412+0000", 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9428999999999997e-05 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.496457+0000", 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5083000000000002e-05 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2921 3.11 3:88e2f4e7:::100000001f8.00000766:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.9913198569999997, 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1412101940000001, 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2921 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370344+0000", 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373083+0000", 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027388270000000001 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373102+0000", 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9789e-05 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373116+0000", 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3456e-05 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778465+0000", 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.405348808 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778616+0000", 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015118400000000001 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778791+0000", 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017549300000000001 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852488+0000", 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073696686999999997 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.511418+0000", 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65892995200000004 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.511495+0000", 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6765000000000001e-05 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.511551+0000", 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6366999999999998e-05 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:37.488 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:37.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:37.490 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:37.487+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:37.590 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:37.588+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:37.605 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:37.605 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:37.605 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:37.605 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:37.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:37.691 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:37.690+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:37.758 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:37.758 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:37.758 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:37.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:37.792 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:37.790+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:13 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 44.556650955000002, 2024-09-19T05:44:37.873 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41922241599999999, 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 13 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.874 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313313+0000", 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.313315+0000", 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.978e-06 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329516+0000", 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016201601999999999 2024-09-19T05:44:37.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329535+0000", 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9157e-05 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329545+0000", 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9909999999999997e-06 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329609+0000", 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3931999999999997e-05 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329764+0000", 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015456700000000001 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.329895+0000", 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013116700000000001 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.877 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.336041+0000", 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061460580000000003 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732421+0000", 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.396379484 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732484+0000", 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3100000000000002e-05 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.878 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.732535+0000", 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1379999999999999e-05 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:269 3.2 3:438a76aa:::10000000002.0000000a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 39.276846014999997, 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.44962057300000002, 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 269 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.880 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593118+0000", 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.593119+0000", 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5099999999999999e-06 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614489+0000", 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.021369545 2024-09-19T05:44:37.881 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614511+0000", 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2353000000000001e-05 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614525+0000", 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4311000000000001e-05 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614691+0000", 2024-09-19T05:44:37.882 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016586800000000001 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614811+0000", 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000119552 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.614956+0000", 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014476499999999999 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.883 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.619618+0000", 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046627020000000003 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042637+0000", 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.42301865999999999 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042689+0000", 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1876000000000003e-05 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.884 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042738+0000", 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9431000000000002e-05 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:320 3.2 3:40abce4f:::10000000002.0000003d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 39.132422067, 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30531098200000001, 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.885 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 320 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.886 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737542+0000", 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.737544+0000", 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6539999999999999e-06 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760554+0000", 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.023009943000000001 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.887 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760577+0000", 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2538e-05 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760585+0000", 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9219999999999997e-06 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760691+0000", 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106606 2024-09-19T05:44:37.888 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760791+0000", 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100239 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.760966+0000", 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017421299999999999 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.766414+0000", 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.005448067 2024-09-19T05:44:37.889 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042779+0000", 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.27636495599999999 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042822+0000", 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3359000000000003e-05 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.042853+0000", 2024-09-19T05:44:37.890 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0485e-05 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2513 3.1b 3:db5bc09b:::100000001f8.000005ce:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.447123+0000", 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.4228406600000003, 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.41167335100000002, 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.891 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2513 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.447123+0000", 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.447123+0000", 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.892 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.447125+0000", 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8840000000000001e-06 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.451101+0000", 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039764709999999997 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.451112+0000", 2024-09-19T05:44:37.893 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0587e-05 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.451117+0000", 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9930000000000003e-06 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.451175+0000", 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8375999999999998e-05 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.894 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.451304+0000", 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012885800000000001 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.451461+0000", 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015667200000000001 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.455352+0000", 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038910419999999999 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.895 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.858695+0000", 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.40334271500000002 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.858750+0000", 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5294000000000003e-05 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.858796+0000", 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6458999999999997e-05 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.896 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:2645 3.2d e70/20 3:b48c1fbb:::100000001f8.00000652:head v 70'65, mlcod=70'63)", 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.885407+0000", 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.9845562509999999, 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34688133999999998, 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.897 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.885407+0000", 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.885407+0000", 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.885409+0000", 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4610000000000001e-06 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.898 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.228153+0000", 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.342743823 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.228168+0000", 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5332000000000002e-05 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.228177+0000", 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.4469999999999995e-06 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.899 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.228245+0000", 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7477999999999993e-05 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.228270+0000", 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5194999999999998e-05 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.232151+0000", 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038807519999999999 2024-09-19T05:44:37.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.232289+0000", 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013785199999999999 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2906 3.1b 3:db0dad13:::100000001f8.00000757:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.018699+0000", 2024-09-19T05:44:37.901 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8512648389999997, 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32638593500000002, 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2906 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.018699+0000", 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.018699+0000", 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.018700+0000", 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6869999999999999e-06 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030600+0000", 2024-09-19T05:44:37.903 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011899796000000001 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030628+0000", 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7597e-05 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030641+0000", 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3154e-05 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.904 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030740+0000", 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9178000000000005e-05 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030885+0000", 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014524399999999999 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.031024+0000", 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013843300000000001 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.905 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.036979+0000", 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059551639999999998 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.344965+0000", 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.307985813 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.345025+0000", 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0043000000000001e-05 2024-09-19T05:44:37.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.345085+0000", 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9825999999999999e-05 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2915 3.1b 3:d80cab9c:::100000001f8.00000760:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.030674+0000", 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8392894680000005, 2024-09-19T05:44:37.907 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53960979899999995, 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2915 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030674+0000", 2024-09-19T05:44:37.908 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030674+0000", 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030677+0000", 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7650000000000002e-06 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.909 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.040963+0000", 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010285608 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041031+0000", 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8497999999999994e-05 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041049+0000", 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7691000000000001e-05 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.910 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041216+0000", 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000167529 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041348+0000", 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013218600000000001 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041483+0000", 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013436500000000001 2024-09-19T05:44:37.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.046423+0000", 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049396969999999998 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.570173+0000", 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52375065499999995 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.912 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.570234+0000", 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0613999999999999e-05 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.570284+0000", 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0191000000000001e-05 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2931 3.2 3:418769c2:::100000001f8.00000770:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.913 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.062079+0000", 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8078847040000001, 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.36921192400000002, 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2931 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.914 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.062079+0000", 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.062079+0000", 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.062082+0000", 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.711e-06 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.915 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072180+0000", 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010098256 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072201+0000", 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0967000000000001e-05 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072211+0000", 2024-09-19T05:44:37.916 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0273e-05 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072312+0000", 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100393 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072432+0000", 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000120451 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.917 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072563+0000", 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013100300000000001 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.078119+0000", 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055556920000000001 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.431157+0000", 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.35303837199999999 2024-09-19T05:44:37.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.431221+0000", 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4397e-05 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.431291+0000", 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9408999999999999e-05 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2937 3.1b 3:d95cf893:::100000001f8.00000776:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.072238+0000", 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.797725378, 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.62993552500000005, 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2937 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072238+0000", 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072238+0000", 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072240+0000", 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1950000000000002e-06 2024-09-19T05:44:37.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081623+0000", 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093823429999999996 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081636+0000", 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3338e-05 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081649+0000", 2024-09-19T05:44:37.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2711000000000001e-05 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081727+0000", 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8064000000000003e-05 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081875+0000", 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148283 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.082099+0000", 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022402300000000001 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.087700+0000", 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056004999999999996 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702066+0000", 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.61436600399999997 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702125+0000", 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9320000000000001e-05 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702174+0000", 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8744e-05 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2946 3.2b 3:d506a813:::100000001f8.0000077f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.925 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.081668+0000", 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7882955029999996, 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.34392292000000002, 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2946 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.926 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081668+0000", 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081668+0000", 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081670+0000", 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.406e-06 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.091397+0000", 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0097274000000000006 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.091417+0000", 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0177999999999999e-05 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.091426+0000", 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8310000000000007e-06 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.091516+0000", 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9622000000000004e-05 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.091677+0000", 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000161712 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.091818+0000", 2024-09-19T05:44:37.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014047500000000001 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.095754+0000", 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0039358650000000002 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.425490+0000", 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.32973633499999999 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.425541+0000", 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0756e-05 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.425591+0000", 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0340000000000003e-05 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2969 3.1b 3:d99b0240:::100000001f8.00000796:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.123365+0000", 2024-09-19T05:44:37.931 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7465986449999997, 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.608564514, 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2969 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.123365+0000", 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.123365+0000", 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.123368+0000", 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.57e-06 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134313+0000", 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010945217 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134336+0000", 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3365000000000001e-05 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134345+0000", 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8599999999999999e-06 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134425+0000", 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0414000000000006e-05 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134597+0000", 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017141699999999999 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134749+0000", 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015180599999999999 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.139528+0000", 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047798290000000002 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731839+0000", 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.59231012500000002 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731890+0000", 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1752000000000002e-05 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731930+0000", 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9159000000000002e-05 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2976 3.1b 3:dba4d56b:::100000001f8.0000079d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.144911+0000", 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7250526959999997, 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.58715358200000001, 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.937 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2976 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.144911+0000", 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.144911+0000", 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.144913+0000", 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.584e-06 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163241+0000", 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018328420000000002 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163276+0000", 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5377999999999999e-05 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163394+0000", 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117496 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163494+0000", 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010056300000000001 2024-09-19T05:44:37.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163643+0000", 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014862400000000001 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163749+0000", 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105504 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169203+0000", 2024-09-19T05:44:37.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054548019999999999 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731983+0000", 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56277920999999997 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732022+0000", 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9869000000000001e-05 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732065+0000", 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2132000000000003e-05 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3057 3.1b 3:da4a88b9:::100000001f8.000007ee:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.225809+0000", 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.6441549330000003, 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.50634741400000005, 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.943 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3057 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.225809+0000", 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.225809+0000", 2024-09-19T05:44:37.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.225811+0000", 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.492e-06 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239326+0000", 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013515070000000001 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.945 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239351+0000", 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5068000000000001e-05 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239364+0000", 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2916e-05 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239489+0000", 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012427799999999999 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239708+0000", 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021967699999999999 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239965+0000", 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000256576 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.245413+0000", 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054483509999999997 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732093+0000", 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48667946600000001 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732128+0000", 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5404000000000002e-05 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732156+0000", 2024-09-19T05:44:37.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8116e-05 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:2666 3.9 e70/20 3:92ca8d9b:::100000001f8.00000667:head v 70'71, mlcod=70'67)", 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.504567+0000", 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3653962399999999, 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.43294840600000001, 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.949 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.504567+0000", 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.504567+0000", 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.504568+0000", 2024-09-19T05:44:37.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.2800000000000005e-07 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.933297+0000", 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.42872838600000002 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.933312+0000", 2024-09-19T05:44:37.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5107e-05 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.933322+0000", 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6889999999999999e-06 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.933392+0000", 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0108e-05 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.933416+0000", 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4550999999999999e-05 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.937481+0000", 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0040649909999999996 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.937516+0000", 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4646e-05 2024-09-19T05:44:37.953 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3235 3.1b 3:d849a4fe:::100000001f8.000008a0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.512061+0000", 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3579028649999998, 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.59435483, 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.954 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3235 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.512061+0000", 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.512061+0000", 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.512066+0000", 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522551+0000", 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010484745 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522563+0000", 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2646999999999999e-05 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522570+0000", 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6379999999999998e-06 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522649+0000", 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9091000000000006e-05 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522752+0000", 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102589 2024-09-19T05:44:37.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522858+0000", 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010684299999999999 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.529143+0000", 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062849569999999999 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.106342+0000", 2024-09-19T05:44:37.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57719904799999999 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.106383+0000", 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0482999999999998e-05 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.106416+0000", 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2706000000000002e-05 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3352 3.1b 3:d821d6cb:::100000001f8.00000915:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.689256+0000", 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.1807074540000002, 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.47540637000000002, 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3352 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.960 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.689256+0000", 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.689256+0000", 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.689259+0000", 2024-09-19T05:44:37.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7030000000000002e-06 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696787+0000", 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075276129999999998 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696819+0000", 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2326000000000002e-05 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696833+0000", 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3791999999999999e-05 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696904+0000", 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0955000000000001e-05 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.697059+0000", 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000155797 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.963 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.697237+0000", 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017717500000000001 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702015+0000", 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047785739999999998 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164557+0000", 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46254178200000001 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164616+0000", 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8789999999999998e-05 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164663+0000", 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6863000000000002e-05 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3359 3.1b 3:d9005dad:::100000001f8.0000091c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:37.965 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.696863+0000", 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.1731008699999999, 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46792231000000001, 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3359 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696863+0000", 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696863+0000", 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696865+0000", 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6400000000000001e-06 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702619+0000", 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057532829999999997 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702631+0000", 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2748000000000001e-05 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702640+0000", 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3699999999999995e-06 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702702+0000", 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702822+0000", 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012001 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702935+0000", 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113186 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.707872+0000", 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049367109999999999 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164710+0000", 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.45683812400000001 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164745+0000", 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.489e-05 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164785+0000", 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0500999999999997e-05 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3383 3.1b 3:dacb2766:::100000001f8.00000934:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:37.971 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.702665+0000", 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.1672983600000002, 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46223359400000003, 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3383 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702665+0000", 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702665+0000", 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702667+0000", 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6610000000000001e-06 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709793+0000", 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071257580000000003 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709812+0000", 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8824000000000001e-05 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709820+0000", 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6770000000000001e-06 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709904+0000", 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3947000000000001e-05 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.710079+0000", 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000175274 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.710249+0000", 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000169877 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.714908+0000", 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046582999999999998 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164834+0000", 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.44992615899999999 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164867+0000", 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3368999999999998e-05 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164899+0000", 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1748000000000002e-05 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3391 3.1b 3:db721795:::100000001f8.0000093c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.718185+0000", 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.1517787970000004, 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.446851212, 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.977 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3391 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.718185+0000", 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.718185+0000", 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.718187+0000", 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.119e-06 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.725910+0000", 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077230379999999998 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.725943+0000", 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2838e-05 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.725957+0000", 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4635e-05 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.726042+0000", 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4227000000000005e-05 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.726210+0000", 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016831300000000001 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.726360+0000", 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015039599999999999 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.730597+0000", 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042367469999999999 2024-09-19T05:44:37.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164988+0000", 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.434391205 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.165017+0000", 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8343e-05 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.165036+0000", 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9351000000000002e-05 2024-09-19T05:44:37.982 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3559 3.1b 3:dae512e2:::100000001f8.000009e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.892388+0000", 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.9775753050000002, 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.31479056900000002, 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:37.983 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3559 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.892388+0000", 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.892388+0000", 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.892391+0000", 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1859999999999999e-06 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.900665+0000", 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0082741080000000005 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.900678+0000", 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3711e-05 2024-09-19T05:44:37.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.900687+0000", 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1349999999999998e-06 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.900738+0000", 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0031999999999998e-05 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.900841+0000", 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000103187 2024-09-19T05:44:37.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.900963+0000", 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122657 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.906047+0000", 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050833789999999998 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.207083+0000", 2024-09-19T05:44:37.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30103582400000001 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.207136+0000", 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3891000000000003e-05 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.207179+0000", 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2459000000000001e-05 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.988 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:37.989 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:37.989 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:37.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:37.991 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:37.988+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.919+0000 7fb4af7fe640 1 -- 172.21.15.2:0/2605982100 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v73) ==== 87+0+0 (secure 0 0 0) 0x7fb4b007f140 con 0x7fb4b81584c0 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb49806c2b0 msgr2=0x7fb49806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb49806c2b0 0x7fb49806e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80079a0 tx=0x7fb4a8008040 comp rx=0 tx=0).stop 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b81584c0 msgr2=0x7fb4b81464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b81584c0 0x7fb4b81464b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002ab20 tx=0x7fb4b0004a40 comp rx=0 tx=0).stop 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 shutdown_connections 2024-09-19T05:44:37.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb49806c2b0 0x7fb49806e770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4b80c86f0 0x7fb4b814b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb4b81584c0 0x7fb4b81464b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 --2- 172.21.15.2:0/2605982100 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb4b8073660 0x7fb4b8155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:37.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 >> 172.21.15.2:0/2605982100 conn(0x7fb4b80cd470 msgr2=0x7fb4b80cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:37.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 shutdown_connections 2024-09-19T05:44:37.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:37.922+0000 7fb4bd677640 1 -- 172.21.15.2:0/2605982100 wait complete. 2024-09-19T05:44:38.091 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:38.089+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:38.134 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:38.134 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:38.134 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:38.134 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:38.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:38.192 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:38.190+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:38.234 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:38.244 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:38.244 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:38.245 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:38.245 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:38.245 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:38.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:38.293 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:38.291+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:38.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.345+0000 7f44a889b640 1 Processor -- start 2024-09-19T05:44:38.347 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.345+0000 7f44a889b640 1 -- start start 2024-09-19T05:44:38.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.345+0000 7f44a889b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f44a4154b30 0x7f44a406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.345+0000 7f44a889b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f44a40cbc00 0x7f44a406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.345+0000 7f44a889b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a40c95f0 con 0x7f44a40c9bc0 2024-09-19T05:44:38.348 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.345+0000 7f44a889b640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f44a40ca220 con 0x7f44a4154b30 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.345+0000 7f44a889b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f44a40ca500 con 0x7f44a40cbc00 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f44a259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f44a4154b30 0x7f44a406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f44a2d9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f44a40c9bc0 0x7f44a415e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:37546/0 (socket says 172.21.15.2:37546) 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f44a2d9d640 1 -- 172.21.15.2:0/1287096191 learned_addr learned my addr 172.21.15.2:0/1287096191 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f449bfff640 1 --2- 172.21.15.2:0/1287096191 >> v2:172.21.15.123:3300/0 conn(0x7f44a40cbc00 0x7f44a406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f44a259c640 1 -- 172.21.15.2:0/1287096191 >> v2:172.21.15.123:3300/0 conn(0x7f44a40cbc00 msgr2=0x7f44a406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f44a259c640 1 --2- 172.21.15.2:0/1287096191 >> v2:172.21.15.123:3300/0 conn(0x7f44a40cbc00 0x7f44a406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f44a259c640 1 -- 172.21.15.2:0/1287096191 >> v1:172.21.15.2:6789/0 conn(0x7f44a40c9bc0 legacy=0x7f44a415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:38.349 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.346+0000 7f44a259c640 1 -- 172.21.15.2:0/1287096191 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a40ca7e0 con 0x7f44a4154b30 2024-09-19T05:44:38.350 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.347+0000 7f44a259c640 1 --2- 172.21.15.2:0/1287096191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a4154b30 0x7f44a406d6e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f44a406e770 tx=0x7f449402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52efd0e33137e649 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.347+0000 7f44a1d9b640 1 -- 172.21.15.2:0/1287096191 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494037070 con 0x7f44a4154b30 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.347+0000 7f44a1d9b640 1 -- 172.21.15.2:0/1287096191 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494032440 con 0x7f44a4154b30 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.347+0000 7f44a1d9b640 1 -- 172.21.15.2:0/1287096191 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449402ae60 con 0x7f44a4154b30 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 -- 172.21.15.2:0/1287096191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a4154b30 msgr2=0x7f44a406d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 --2- 172.21.15.2:0/1287096191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a4154b30 0x7f44a406d6e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f44a406e770 tx=0x7f449402a110 comp rx=0 tx=0).stop 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 -- 172.21.15.2:0/1287096191 shutdown_connections 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 --2- 172.21.15.2:0/1287096191 >> v2:172.21.15.123:3300/0 conn(0x7f44a40cbc00 0x7f44a406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 --2- 172.21.15.2:0/1287096191 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a4154b30 0x7f44a406d6e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 -- 172.21.15.2:0/1287096191 >> 172.21.15.2:0/1287096191 conn(0x7f44a40707f0 msgr2=0x7f44a4070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 -- 172.21.15.2:0/1287096191 shutdown_connections 2024-09-19T05:44:38.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.348+0000 7f44a889b640 1 -- 172.21.15.2:0/1287096191 wait complete. 2024-09-19T05:44:38.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.349+0000 7f44a889b640 1 Processor -- start 2024-09-19T05:44:38.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a889b640 1 -- start start 2024-09-19T05:44:38.353 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a889b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a40c9bc0 0x7f44a41f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a889b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a40cbc00 0x7f44a40c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a889b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44a4154b30 0x7f44a41504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a889b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f44a40c9380 con 0x7f44a40cbc00 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a889b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44a40c9200 con 0x7f44a40c9bc0 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a889b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f44a415ea50 con 0x7f44a4154b30 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f449bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a40cbc00 0x7f44a40c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a40c9bc0 0x7f44a41f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f449bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a40cbc00 0x7f44a40c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60260/0 (socket says 172.21.15.2:60260) 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f449bfff640 1 -- 172.21.15.2:0/1112569448 learned_addr learned my addr 172.21.15.2:0/1112569448 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.350+0000 7f44a2d9d640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44a4154b30 0x7f44a41504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449bfff640 1 -- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44a4154b30 msgr2=0x7f44a41504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449bfff640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f44a4154b30 0x7f44a41504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:38.354 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449bfff640 1 -- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a40c9bc0 msgr2=0x7f44a41f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449bfff640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a40c9bc0 0x7f44a41f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449bfff640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a41f40e0 con 0x7f44a40cbc00 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449bfff640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44a40cbc00 0x7f44a40c6650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f448c00e900 tx=0x7f448c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c017070 con 0x7f44a40cbc00 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a415a6f0 con 0x7f44a40cbc00 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c00ca00 con 0x7f44a40cbc00 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c00cdb0 con 0x7f44a40cbc00 2024-09-19T05:44:38.355 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.351+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a415d340 con 0x7f44a40cbc00 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.352+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f44a415d130 con 0x7f44a40cbc00 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.353+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f448c01a070 con 0x7f44a40cbc00 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.353+0000 7f449affd640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f448406c2e0 0x7f448406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.353+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5799+0+0 (secure 0 0 0) 0x7f448c08b730 con 0x7f44a40cbc00 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.354+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v73) ==== 54+0+16 (secure 0 0 0) 0x7f448c091170 con 0x7f44a40cbc00 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.354+0000 7f44a259c640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f448406c2e0 0x7f448406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.354+0000 7f44a889b640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f44a40c5890 0x7f44a414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.357 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.354+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44a415e750 con 0x7f44a40c5890 2024-09-19T05:44:38.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.354+0000 7f44a2d9d640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f44a40c5890 0x7f44a414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.354+0000 7f44a259c640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f448406c2e0 0x7f448406e7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4494034000 tx=0x7f449402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.354+0000 7f44a2d9d640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f44a40c5890 0x7f44a414af50 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.358 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.355+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44a415e750 con 0x7f44a40c5890 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3 3.12 3:4bec3886:::10000000002.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e52)", 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:38.359 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 45.073817816000002, 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57240116200000002, 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:38.360 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280485+0000", 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.280487+0000", 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.751e-06 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.361 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287574+0000", 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070870530000000003 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287592+0000", 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8241e-05 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287601+0000", 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2800000000000003e-06 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.362 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287664+0000", 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3474999999999998e-05 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287758+0000", 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3577000000000002e-05 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.287882+0000", 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012487199999999999 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.363 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.292606+0000", 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047238860000000001 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852756+0000", 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56015009400000004 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852823+0000", 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6138999999999998e-05 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.364 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852886+0000", 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3794000000000006e-05 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:27 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 45.051283818000002, 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55004442399999998, 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.365 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 27 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.366 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303019+0000", 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.303023+0000", 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4340000000000001e-06 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316366+0000", 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013343634 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.367 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316392+0000", 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6120999999999999e-05 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316402+0000", 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0066e-05 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316514+0000", 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111859 2024-09-19T05:44:38.368 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316686+0000", 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017135699999999999 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.316868+0000", 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018244100000000001 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.324019+0000", 2024-09-19T05:44:38.369 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071506060000000003 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.852956+0000", 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52893702899999995 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853018+0000", 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2178e-05 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.370 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.853064+0000", 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5698999999999997e-05 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:46 3.12 3:489b4446:::10000000002.0000002b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e53)", 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 45.022707951999998, 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.54589824600000003, 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.371 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 46 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.372 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331595+0000", 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.331598+0000", 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5689999999999999e-06 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.373 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347173+0000", 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015575235999999999 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347192+0000", 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9113000000000001e-05 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347202+0000", 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0151e-05 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.374 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347265+0000", 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3163000000000005e-05 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347408+0000", 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014307299999999999 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.375 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.347532+0000", 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012328400000000001 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.351950+0000", 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044183620000000003 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877426+0000", 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52547596500000004 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.376 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877461+0000", 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5309999999999999e-05 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:53.877493+0000", 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2020000000000002e-05 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:283 3.12 3:4b926a6e:::10000000002.00000018:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e54)", 2024-09-19T05:44:38.377 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 39.740340130999996, 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57880034499999999, 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 283 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.378 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613963+0000", 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.613965+0000", 2024-09-19T05:44:38.379 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8759999999999999e-06 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633465+0000", 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019500483999999998 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633485+0000", 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.984e-05 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.380 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633494+0000", 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.5520000000000005e-06 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633589+0000", 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5144999999999998e-05 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633713+0000", 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012382300000000001 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.381 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.633835+0000", 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122705 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:58.640960+0000", 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071243599999999997 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192691+0000", 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.55173159599999999 2024-09-19T05:44:38.382 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192737+0000", 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5487e-05 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:43:59.192763+0000", 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6477000000000001e-05 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.383 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2643 3.12 3:4bb8df49:::100000001f8.00000650:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.706242+0000", 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.6480610430000002, 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.78714695000000001, 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2643 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.384 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.706242+0000", 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.706242+0000", 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.385 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.706270+0000", 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7685999999999999e-05 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.715899+0000", 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0096291250000000005 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.715929+0000", 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0261999999999999e-05 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.386 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.715941+0000", 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1770000000000001e-05 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.716034+0000", 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3560999999999994e-05 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.716171+0000", 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013706300000000001 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.387 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.716279+0000", 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107532 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.721031+0000", 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047521239999999999 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.388 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.493241+0000", 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.77220999099999998 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.493319+0000", 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8291e-05 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.493389+0000", 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9544999999999999e-05 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.389 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2720 3.12 3:4983891e:::100000001f8.0000069d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.796089+0000", 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.558213662, 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.139035335, 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.390 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2720 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.796089+0000", 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.796089+0000", 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.391 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.796092+0000", 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0419999999999998e-06 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807355+0000", 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011262872 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.392 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807383+0000", 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7639000000000001e-05 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807397+0000", 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4405e-05 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807476+0000", 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8950999999999997e-05 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.393 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807739+0000", 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026290900000000003 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807991+0000", 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000251666 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.817503+0000", 2024-09-19T05:44:38.394 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095125000000000001 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.934855+0000", 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1173514680000001 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.934915+0000", 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9877999999999998e-05 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.395 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935125+0000", 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021000500000000001 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2727 3.12 3:4bacb89b:::100000001f8.000006a4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.818067+0000", 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.5362359369999998, 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.117480729, 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.396 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2727 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.818067+0000", 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.818067+0000", 2024-09-19T05:44:38.397 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.818069+0000", 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.345e-06 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829667+0000", 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011597896 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.398 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829711+0000", 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4060999999999997e-05 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829725+0000", 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4073999999999999e-05 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829832+0000", 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106382 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.399 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.830005+0000", 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000173265 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.830175+0000", 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016997299999999999 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.839760+0000", 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095851990000000008 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.400 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935410+0000", 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.095649686 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935473+0000", 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3139999999999995e-05 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935548+0000", 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4708000000000004e-05 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.401 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2809 3.12 3:482c9566:::100000001f8.000006f6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.870050+0000", 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.4842528530000001, 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.335518507, 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.402 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2809 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.870050+0000", 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.870050+0000", 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.403 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.870053+0000", 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.909e-06 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882108+0000", 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012055003 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882128+0000", 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9837999999999999e-05 2024-09-19T05:44:38.404 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882138+0000", 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6169999999999996e-06 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882209+0000", 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1849999999999998e-05 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882310+0000", 2024-09-19T05:44:38.405 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100436 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882406+0000", 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6428999999999994e-05 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.888085+0000", 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056786060000000001 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.406 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.205494+0000", 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.317409576 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.205536+0000", 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1519e-05 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.205569+0000", 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2724e-05 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.407 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2952 3.12 3:49429726:::100000001f8.00000785:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.032287+0000", 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.3220163190000003, 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381939864, 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2952 2024-09-19T05:44:38.408 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.032287+0000", 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.032287+0000", 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.409 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.032291+0000", 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.561e-06 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042630+0000", 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010338905000000001 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042651+0000", 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0492999999999998e-05 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.410 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042660+0000", 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6700000000000006e-06 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042728+0000", 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7884000000000003e-05 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042871+0000", 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142972 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.411 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.043038+0000", 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016656099999999999 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.048005+0000", 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049671289999999998 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414133+0000", 2024-09-19T05:44:38.412 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.366128263 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414176+0000", 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2509000000000002e-05 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414227+0000", 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0917000000000001e-05 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.413 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3007 3.12 3:48356219:::100000001f8.000007bc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.158415+0000", 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.1958881650000004, 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2559814359999999, 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3007 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.414 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.158415+0000", 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.158415+0000", 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.158417+0000", 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8589999999999999e-06 2024-09-19T05:44:38.415 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173428+0000", 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015011648000000001 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173459+0000", 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1161999999999997e-05 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.416 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173475+0000", 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5998000000000002e-05 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173571+0000", 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5579000000000001e-05 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173841+0000", 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026997300000000001 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.417 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.174125+0000", 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028421099999999998 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.180126+0000", 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060009809999999999 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414311+0000", 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2341851130000001 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.418 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414370+0000", 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8899e-05 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414396+0000", 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6013e-05 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3095 3.33 3:cd66a262:::100000001f8.00000814:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.419 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.294970+0000", 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.0593330590000001, 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.82468124399999998, 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3095 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.294970+0000", 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.294970+0000", 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.294973+0000", 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9069999999999999e-06 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.421 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306331+0000", 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011357848 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306348+0000", 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6841e-05 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306358+0000", 2024-09-19T05:44:38.422 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0081000000000001e-05 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306424+0000", 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6806999999999999e-05 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306543+0000", 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118312 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306642+0000", 2024-09-19T05:44:38.423 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8808000000000007e-05 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.312752+0000", 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061104210000000004 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.119572+0000", 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.80681970000000003 2024-09-19T05:44:38.424 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.119615+0000", 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3238999999999997e-05 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.119651+0000", 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6279999999999998e-05 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.425 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3101 3.12 3:4aca477d:::100000001f8.0000081a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.319810+0000", 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.0344928009999999, 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1290244970000001, 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3101 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.426 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.319810+0000", 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.319810+0000", 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.319812+0000", 2024-09-19T05:44:38.427 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8679999999999999e-06 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331254+0000", 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01144163 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331271+0000", 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7013000000000001e-05 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.428 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331281+0000", 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0412e-05 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331356+0000", 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5127000000000001e-05 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331506+0000", 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014979899999999999 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.429 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331664+0000", 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015781399999999999 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.336192+0000", 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045283349999999997 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.448735+0000", 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.112542908 2024-09-19T05:44:38.430 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.448779+0000", 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3634000000000003e-05 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.448835+0000", 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5957e-05 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.431 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3106 3.12 3:48a69cf9:::100000001f8.0000081f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.360705+0000", 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.9935983789999998, 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.119816538, 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3106 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.432 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.360705+0000", 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.360705+0000", 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.360707+0000", 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1950000000000002e-06 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.433 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374370+0000", 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013662818 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374383+0000", 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3203999999999999e-05 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374388+0000", 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5949999999999998e-06 2024-09-19T05:44:38.434 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374454+0000", 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5779999999999997e-05 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374602+0000", 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014766599999999999 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374725+0000", 2024-09-19T05:44:38.435 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012355600000000001 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.381270+0000", 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065449549999999999 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480379+0000", 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0991086860000001 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480447+0000", 2024-09-19T05:44:38.436 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7473999999999998e-05 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480521+0000", 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4609000000000003e-05 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3179 3.12 3:4a7c9af2:::100000001f8.00000868:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.417716+0000", 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.9365873249999996, 2024-09-19T05:44:38.437 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.062994529, 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3179 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.417716+0000", 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.438 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.417716+0000", 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.439 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.417718+0000", 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6759999999999999e-06 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435215+0000", 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017496912999999999 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435233+0000", 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7737e-05 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.440 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435243+0000", 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0045e-05 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435319+0000", 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6029e-05 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435454+0000", 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013470299999999999 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.441 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435589+0000", 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013483399999999999 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.440522+0000", 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049332739999999996 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480598+0000", 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.040076196 2024-09-19T05:44:38.442 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480651+0000", 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3066000000000003e-05 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480710+0000", 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9055999999999998e-05 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.443 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3267 3.12 3:49d1bd5a:::100000001f8.000008c0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.539767+0000", 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.8145364390000003, 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.94104224199999997, 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3267 2024-09-19T05:44:38.444 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.539767+0000", 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.539767+0000", 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.445 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.539768+0000", 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9410000000000001e-06 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.550897+0000", 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011128403 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.550920+0000", 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.338e-05 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.446 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.550937+0000", 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7108999999999998e-05 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.551008+0000", 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0506999999999994e-05 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.447 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.551173+0000", 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016462399999999999 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.551322+0000", 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149498 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.555948+0000", 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046262040000000001 2024-09-19T05:44:38.448 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480757+0000", 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.92480870999999998 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480785+0000", 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8220000000000001e-05 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.449 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480809+0000", 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3646e-05 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3315 3.12 3:4962d06a:::100000001f8.000008f0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.611024+0000", 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7432791840000004, 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.90393573900000002, 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.450 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3315 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.611024+0000", 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.611024+0000", 2024-09-19T05:44:38.451 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.611027+0000", 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5030000000000002e-06 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618757+0000", 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077300169999999996 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618773+0000", 2024-09-19T05:44:38.452 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5778999999999999e-05 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618782+0000", 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.098e-06 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618889+0000", 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106937 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.453 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.619059+0000", 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017009800000000001 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.619224+0000", 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016449600000000001 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.625571+0000", 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006347559 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.454 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.514837+0000", 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.88926545599999995 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.514899+0000", 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2614999999999996e-05 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.514960+0000", 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0180999999999999e-05 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.455 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3443 3.12 3:4be71d45:::100000001f8.00000970:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.749886+0000", 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.6044166100000004, 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.84167001100000005, 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3443 2024-09-19T05:44:38.456 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.749886+0000", 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.749886+0000", 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.457 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.749890+0000", 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0060000000000003e-06 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756769+0000", 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068785080000000002 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756788+0000", 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9143999999999998e-05 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.458 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756799+0000", 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0858e-05 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756862+0000", 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3314999999999999e-05 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756999+0000", 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013695999999999999 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.459 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.757098+0000", 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8987000000000006e-05 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.761307+0000", 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042089789999999998 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591482+0000", 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.83017514999999997 2024-09-19T05:44:38.460 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591522+0000", 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9715999999999997e-05 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591556+0000", 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4387999999999997e-05 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.461 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3455 3.12 3:493add79:::100000001f8.0000097c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.786731+0000", 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5675723210000001, 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.804915995, 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3455 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.462 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.786731+0000", 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.786731+0000", 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.786733+0000", 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3999999999999999e-06 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.463 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828604+0000", 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.041870624000000002 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828630+0000", 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6795999999999999e-05 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828642+0000", 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1823e-05 2024-09-19T05:44:38.464 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828736+0000", 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3364999999999998e-05 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828856+0000", 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012047100000000001 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828979+0000", 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012258599999999999 2024-09-19T05:44:38.465 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.834525+0000", 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055461620000000003 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591590+0000", 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.75706481599999997 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591622+0000", 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2410999999999999e-05 2024-09-19T05:44:38.466 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591647+0000", 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4541000000000001e-05 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3489 3.12 3:4b5b43b4:::100000001f8.0000099e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.875661+0000", 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.478642206, 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.71607116400000004, 2024-09-19T05:44:38.467 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3489 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.875661+0000", 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.875661+0000", 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.875663+0000", 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.012e-06 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.882829+0000", 2024-09-19T05:44:38.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071662870000000004 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.882847+0000", 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8270999999999998e-05 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.882855+0000", 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1270000000000003e-06 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.882931+0000", 2024-09-19T05:44:38.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5362000000000001e-05 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.883070+0000", 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013898599999999999 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.883166+0000", 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6051000000000006e-05 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.888126+0000", 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049603060000000003 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591682+0000", 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.703555657 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591711+0000", 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8943000000000001e-05 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.472 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591732+0000", 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1162000000000002e-05 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3545 3.12 3:4b497566:::100000001f8.000009d6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.928506+0000", 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.4257967980000004, 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.66505559700000005, 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.473 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3545 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.928506+0000", 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.928506+0000", 2024-09-19T05:44:38.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.928509+0000", 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.751e-06 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.938441+0000", 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099319790000000005 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.938453+0000", 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2456999999999999e-05 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.938462+0000", 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7120000000000006e-06 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.938511+0000", 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8565e-05 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.938624+0000", 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113069 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.938728+0000", 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000103834 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.942607+0000", 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038793880000000001 2024-09-19T05:44:38.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.593509+0000", 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.65090250500000002 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.593539+0000", 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9875000000000001e-05 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.593562+0000", 2024-09-19T05:44:38.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2461999999999999e-05 2024-09-19T05:44:38.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.479 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.479 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.479 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.479 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:38.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.370+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f44a415e470 con 0x7f44a40c5890 2024-09-19T05:44:38.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.372+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44a415e470 con 0x7f44a40c5890 2024-09-19T05:44:38.481 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:38.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.372+0000 7f44a889b640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f44a415d510 0x7f44a4158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.372+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f44a4159de0 con 0x7f44a415d510 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.372+0000 7f44a259c640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f44a415d510 0x7f44a4158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.373+0000 7f44a259c640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f44a415d510 0x7f44a4158590 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.374+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44a4159de0 con 0x7f44a415d510 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.389+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f44a40c4b60 con 0x7f44a415d510 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.390+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44a40c4b60 con 0x7f44a415d510 2024-09-19T05:44:38.483 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.391+0000 7f44a889b640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f44a4158fe0 0x7f44a4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.391+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f44a4159420 con 0x7f44a4158fe0 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.391+0000 7f449bfff640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f44a4158fe0 0x7f44a4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.391+0000 7f449bfff640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f44a4158fe0 0x7f44a4062630 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.393+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44a4159420 con 0x7f44a4158fe0 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.419+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f44a4062b70 con 0x7f44a4158fe0 2024-09-19T05:44:38.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.423+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44a4062b70 con 0x7f44a4158fe0 2024-09-19T05:44:38.486 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:38.486 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:38.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.435+0000 7f4498ff9640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f44a414f0b0 0x7f44a414f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.436+0000 7f4498ff9640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f44a414f9f0 con 0x7f44a414f0b0 2024-09-19T05:44:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.438+0000 7f44a2d9d640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f44a414f0b0 0x7f44a414f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.438+0000 7f44a2d9d640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f44a414f0b0 0x7f44a414f4b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.443+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44a414f9f0 con 0x7f44a414f0b0 2024-09-19T05:44:38.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.494+0000 7f4498ff9640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f44a414fe50 con 0x7f44a414f0b0 2024-09-19T05:44:38.497 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:38.482+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:38.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.498+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44a414fe50 con 0x7f44a414f0b0 2024-09-19T05:44:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:38.501 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:38.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.499+0000 7f44a889b640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f44a402cb50 0x7f44a402cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.499+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f44a4150060 con 0x7f44a402cb50 2024-09-19T05:44:38.503 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.501+0000 7f44a259c640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f44a402cb50 0x7f44a402cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.502+0000 7f44a259c640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f44a402cb50 0x7f44a402cf90 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.503+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44a4150060 con 0x7f44a402cb50 2024-09-19T05:44:38.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.542+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f44a4150060 con 0x7f44a402cb50 2024-09-19T05:44:38.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.545+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44a4150060 con 0x7f44a402cb50 2024-09-19T05:44:38.547 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:38.548 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:38.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.546+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=74}) -- 0x7f44a4150000 con 0x7f44a40cbc00 2024-09-19T05:44:38.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.546+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44a402d4d0 con 0x7f44a40cbc00 2024-09-19T05:44:38.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.546+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f448c021020 con 0x7f44a40cbc00 2024-09-19T05:44:38.549 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:38.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.547+0000 7f44a889b640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f44a4206b40 0x7f44a4208f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.547+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f44a402d4d0 con 0x7f44a4206b40 2024-09-19T05:44:38.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.548+0000 7f449bfff640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f44a4206b40 0x7f44a4208f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.548+0000 7f449bfff640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f44a4206b40 0x7f44a4208f30 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.550+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44a402d4d0 con 0x7f44a4206b40 2024-09-19T05:44:38.582 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:38.580+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:38.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.606+0000 7f44a889b640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f44a402d4d0 con 0x7f44a4206b40 2024-09-19T05:44:38.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.612+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44a402d4d0 con 0x7f44a4206b40 2024-09-19T05:44:38.615 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:38.615 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.614+0000 7f4498ff9640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f447c001650 0x7f447c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.615+0000 7f4498ff9640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f447c006bb0 con 0x7f447c001650 2024-09-19T05:44:38.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.615+0000 7f44a2d9d640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f447c001650 0x7f447c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:38.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.621+0000 7f44a2d9d640 1 --2- 172.21.15.2:0/1112569448 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f447c001650 0x7f447c003b10 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:38.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.622+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f447c006bb0 con 0x7f447c001650 2024-09-19T05:44:38.653 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:38.653 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:38.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.653 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3648 3.2e 3:75be0f36:::100000001f8.0000003d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e73)", 2024-09-19T05:44:38.653 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:38.547517+0000", 2024-09-19T05:44:38.653 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.104304781, 2024-09-19T05:44:38.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.104323763, 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3648 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.547517+0000", 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.547517+0000", 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.547519+0000", 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1610000000000001e-06 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.553363+0000", 2024-09-19T05:44:38.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058443820000000004 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.553379+0000", 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5866e-05 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.553387+0000", 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5700000000000004e-06 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.553465+0000", 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8613000000000001e-05 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.553529+0000", 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4185999999999998e-05 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.553620+0000", 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0550999999999994e-05 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.557518+0000", 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038980830000000001 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:3645 3.c e73/20 3:33b8095f:::100000001f8.0000003a:head v 73'64, mlcod=71'62)", 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:38.641402+0000", 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.010419288000000001, 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010487652, 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:38.658 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "started", 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.641402+0000", 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.641402+0000", 2024-09-19T05:44:38.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.641402+0000", 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7500000000000001e-07 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.647084+0000", 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056816419999999998 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.647103+0000", 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8771e-05 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.647113+0000", 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0210000000000001e-05 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.647185+0000", 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2107000000000006e-05 2024-09-19T05:44:38.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:38.647210+0000", 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5168999999999999e-05 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 2 2024-09-19T05:44:38.662 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:38.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.656+0000 7f4498ff9640 1 -- 172.21.15.2:0/1112569448 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f447c0058d0 con 0x7f447c001650 2024-09-19T05:44:38.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:38.658+0000 7f449affd640 1 -- 172.21.15.2:0/1112569448 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f447c0058d0 con 0x7f447c001650 2024-09-19T05:44:38.664 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:38.664 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:38.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:38.676 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:38.677 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:38.725 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:38.717+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:38.789 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:38.783+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:38.871 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:38.871 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:38.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:38.897 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:38.883+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:38.989 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:38.982+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:39.042 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:39.043 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:39.063 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:39.085 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:39.084+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:39.186 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:39.185+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:39.287 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:39.285+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:39.351 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:39.351 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:39.357 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-19T05:44:39.357 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:44:39.357 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-19T05:44:39.357 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T05:44:39.357 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T05:44:39.357 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-19T05:44:39.379 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:39.388 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:39.386+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:39.489 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:39.487+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:39.593 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:39.591+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:39.692 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:39.688+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:39.698 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:39.698 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-19T05:44:39.698 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:39.720 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:39.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:39.801 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:39.792+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:39.892 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:39.890+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:39.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:44:39.995 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:39.993+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:40.099 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:40.099 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:40.114 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:40.097+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:40.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:40.221 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:40.217+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:40.294 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:40.293+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.296+0000 7ff72754b640 1 Processor -- start 2024-09-19T05:44:40.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.296+0000 7ff72754b640 1 -- start start 2024-09-19T05:44:40.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.298+0000 7ff72754b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff718006e00 0x7ff7181001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.298+0000 7ff72754b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff718005af0 con 0x7ff718044b30 2024-09-19T05:44:40.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.298+0000 7ff72754b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff71804b3a0 con 0x7ff7180f64e0 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.298+0000 7ff72754b640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ff718006970 con 0x7ff718006e00 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.299+0000 7ff724ae6640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff718044b30 0x7ff71804bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:37556/0 (socket says 172.21.15.2:37556) 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.299+0000 7ff724ae6640 1 -- 172.21.15.2:0/4274333106 learned_addr learned my addr 172.21.15.2:0/4274333106 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.299+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1187235060 0 0) 0x7ff718005af0 con 0x7ff718044b30 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.300+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff70c003970 con 0x7ff718044b30 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff725ae8640 1 --2- 172.21.15.2:0/4274333106 >> v2:172.21.15.2:3300/0 conn(0x7ff718006e00 0x7ff7181001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1983681332 0 0) 0x7ff70c003970 con 0x7ff718044b30 2024-09-19T05:44:40.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 >> v2:172.21.15.2:3300/0 conn(0x7ff718006e00 msgr2=0x7ff7181001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 --2- 172.21.15.2:0/4274333106 >> v2:172.21.15.2:3300/0 conn(0x7ff718006e00 0x7ff7181001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 >> v1:172.21.15.123:6789/0 conn(0x7ff7180f64e0 legacy=0x7ff71804ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff718005d60 con 0x7ff718044b30 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7ff720050800 con 0x7ff718044b30 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 >> v1:172.21.15.2:6789/0 conn(0x7ff718044b30 legacy=0x7ff71804bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff70c004c50 0x7ff70c005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff70c005590 0x7ff70c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff70c009e00 0x7ff70c004170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff718006970 con 0x7ff70c004c50 2024-09-19T05:44:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c004a40 con 0x7ff70c009e00 2024-09-19T05:44:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff70c00a270 con 0x7ff70c005590 2024-09-19T05:44:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.301+0000 7ff725ae8640 1 --2- 172.21.15.2:0/4274333106 >> v2:172.21.15.2:3300/0 conn(0x7ff718006e00 0x7ff7181001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:44:40.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.302+0000 7ff724ae6640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff70c005590 0x7ff70c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.302+0000 7ff725ae8640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff70c009e00 0x7ff70c004170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.302+0000 7ff724ae6640 1 -- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff70c009e00 msgr2=0x7ff70c004170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.302+0000 7ff724ae6640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff70c009e00 0x7ff70c004170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.302+0000 7ff724ae6640 1 -- 172.21.15.2:0/4274333106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff70c004c50 msgr2=0x7ff70c005050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.302+0000 7ff724ae6640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff70c004c50 0x7ff70c005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.302+0000 7ff724ae6640 1 -- 172.21.15.2:0/4274333106 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff718005d60 con 0x7ff70c005590 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.303+0000 7ff725ae8640 1 -- 172.21.15.2:0/4274333106 reap_dead start 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.303+0000 7ff724ae6640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff70c005590 0x7ff70c003bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff72006b320 tx=0x7ff70c005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49132f1ff0d565c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.304+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7200a2070 con 0x7ff70c005590 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.304+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72009d400 con 0x7ff70c005590 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.304+0000 7ff717fff640 1 -- 172.21.15.2:0/4274333106 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720050070 con 0x7ff70c005590 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.306+0000 7ff72754b640 1 -- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff70c005590 msgr2=0x7ff70c003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.306+0000 7ff72754b640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff70c005590 0x7ff70c003bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff72006b320 tx=0x7ff70c005c80 comp rx=0 tx=0).stop 2024-09-19T05:44:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.306+0000 7ff72754b640 1 -- 172.21.15.2:0/4274333106 shutdown_connections 2024-09-19T05:44:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.306+0000 7ff72754b640 1 --2- 172.21.15.2:0/4274333106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff70c005590 0x7ff70c003bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.306+0000 7ff72754b640 1 -- 172.21.15.2:0/4274333106 >> 172.21.15.2:0/4274333106 conn(0x7ff71804fc50 msgr2=0x7ff718050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.307+0000 7ff72754b640 1 -- 172.21.15.2:0/4274333106 shutdown_connections 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.307+0000 7ff72754b640 1 -- 172.21.15.2:0/4274333106 wait complete. 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.309+0000 7ff72754b640 1 Processor -- start 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.309+0000 7ff72754b640 1 -- start start 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff72754b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718048db0 0x7ff7180f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff72754b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718049190 0x7ff718047900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff72754b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718047e40 0x7ff71804ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff72754b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff718006600 con 0x7ff718048db0 2024-09-19T05:44:40.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff72754b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff718006320 con 0x7ff718047e40 2024-09-19T05:44:40.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff72754b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff7180f68c0 con 0x7ff718049190 2024-09-19T05:44:40.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff7252e7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718048db0 0x7ff7180f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff7252e7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718048db0 0x7ff7180f7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60288/0 (socket says 172.21.15.2:60288) 2024-09-19T05:44:40.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.311+0000 7ff7252e7640 1 -- 172.21.15.2:0/1467127618 learned_addr learned my addr 172.21.15.2:0/1467127618 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:40.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.312+0000 7ff7252e7640 1 -- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718049190 msgr2=0x7ff718047900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.312+0000 7ff7252e7640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718049190 0x7ff718047900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.312+0000 7ff7252e7640 1 -- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718047e40 msgr2=0x7ff71804ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.312+0000 7ff7252e7640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718047e40 0x7ff71804ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.312+0000 7ff7252e7640 1 -- 172.21.15.2:0/1467127618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff718048800 con 0x7ff718048db0 2024-09-19T05:44:40.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.312+0000 7ff7252e7640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718048db0 0x7ff7180f7ab0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff71000c3f0 tx=0x7ff71000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:40.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.313+0000 7ff715ffb640 1 -- 172.21.15.2:0/1467127618 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710011790 con 0x7ff718048db0 2024-09-19T05:44:40.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.313+0000 7ff715ffb640 1 -- 172.21.15.2:0/1467127618 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff710017070 con 0x7ff718048db0 2024-09-19T05:44:40.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.313+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7180f6dd0 con 0x7ff718048db0 2024-09-19T05:44:40.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.313+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff718046ff0 con 0x7ff718048db0 2024-09-19T05:44:40.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.313+0000 7ff715ffb640 1 -- 172.21.15.2:0/1467127618 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710011b50 con 0x7ff718048db0 2024-09-19T05:44:40.317 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.315+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7180071e0 con 0x7ff718048db0 2024-09-19T05:44:40.319 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.317+0000 7ff715ffb640 1 -- 172.21.15.2:0/1467127618 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff710011cf0 con 0x7ff718048db0 2024-09-19T05:44:40.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.318+0000 7ff715ffb640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff70406c2e0 0x7ff70406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.319+0000 7ff724ae6640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff70406c2e0 0x7ff70406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.320+0000 7ff724ae6640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff70406c2e0 0x7ff70406e7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00a440 tx=0x7ff72009a000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:40.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.320+0000 7ff715ffb640 1 -- 172.21.15.2:0/1467127618 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5799+0+0 (secure 0 0 0) 0x7ff71008bc50 con 0x7ff718048db0 2024-09-19T05:44:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.324+0000 7ff715ffb640 1 -- 172.21.15.2:0/1467127618 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff71005a3d0 con 0x7ff718048db0 2024-09-19T05:44:40.395 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:40.394+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:40.452 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:40.452 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:40.494 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:40.555 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:40.535+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:40.600 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:40.595+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:40.677 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:40.697 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:40.695+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:40.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.778+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7ff7180e8d40 con 0x7ff718048db0 2024-09-19T05:44:40.791 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:40.791 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:40.798 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:40.796+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:40.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:40.900 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:40.897+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.929+0000 7f947c4c3640 1 Processor -- start 2024-09-19T05:44:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.929+0000 7f947c4c3640 1 -- start start 2024-09-19T05:44:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.934+0000 7f947c4c3640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94741528e0 0x7f94740c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.934+0000 7f947c4c3640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f947415c020 con 0x7f94740c34d0 2024-09-19T05:44:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.934+0000 7f947c4c3640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f947415c300 con 0x7f9474152cc0 2024-09-19T05:44:40.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.934+0000 7f947c4c3640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f947415cc30 con 0x7f94741528e0 2024-09-19T05:44:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.937+0000 7f947aa60640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9474152cc0 0x7f9474163160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:47714/0 (socket says 172.21.15.2:47714) 2024-09-19T05:44:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.937+0000 7f947aa60640 1 -- 172.21.15.2:0/226301863 learned_addr learned my addr 172.21.15.2:0/226301863 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.937+0000 7f947a25f640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f94741528e0 0x7f94740c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.940+0000 7f947a25f640 1 -- 172.21.15.2:0/226301863 >> v1:172.21.15.123:6789/0 conn(0x7f9474152cc0 legacy=0x7f9474163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.940+0000 7f947a25f640 1 -- 172.21.15.2:0/226301863 >> v1:172.21.15.2:6789/0 conn(0x7f94740c34d0 legacy=0x7f9474158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:40.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.941+0000 7f947a25f640 1 -- 172.21.15.2:0/226301863 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947406ed40 con 0x7f94741528e0 2024-09-19T05:44:40.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.940+0000 7f947925d640 1 -- 172.21.15.2:0/226301863 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 575796227 0 0) 0x7f947415c300 con 0x7f9474152cc0 2024-09-19T05:44:40.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.945+0000 7f947a25f640 1 --2- 172.21.15.2:0/226301863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94741528e0 0x7f94740c3ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94700097d0 tx=0x7f947002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4183482f1de43462 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.950+0000 7f947925d640 1 -- 172.21.15.2:0/226301863 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470037070 con 0x7f94741528e0 2024-09-19T05:44:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.950+0000 7f947925d640 1 -- 172.21.15.2:0/226301863 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9470032440 con 0x7f94741528e0 2024-09-19T05:44:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.951+0000 7f947c4c3640 1 -- 172.21.15.2:0/226301863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94741528e0 msgr2=0x7f94740c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.951+0000 7f947c4c3640 1 --2- 172.21.15.2:0/226301863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94741528e0 0x7f94740c3ee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f94700097d0 tx=0x7f947002a180 comp rx=0 tx=0).stop 2024-09-19T05:44:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.959+0000 7f947c4c3640 1 -- 172.21.15.2:0/226301863 shutdown_connections 2024-09-19T05:44:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.959+0000 7f947c4c3640 1 --2- 172.21.15.2:0/226301863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94741528e0 0x7f94740c3ee0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.961 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.959+0000 7f947c4c3640 1 -- 172.21.15.2:0/226301863 >> 172.21.15.2:0/226301863 conn(0x7f94740707f0 msgr2=0x7f9474070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:40.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.957+0000 7ff715ffb640 1 -- 172.21.15.2:0/1467127618 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v74) ==== 97+0+0 (secure 0 0 0) 0x7ff710059cd0 con 0x7ff718048db0 2024-09-19T05:44:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:44:40.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.961+0000 7f947c4c3640 1 -- 172.21.15.2:0/226301863 shutdown_connections 2024-09-19T05:44:40.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.964+0000 7f947c4c3640 1 -- 172.21.15.2:0/226301863 wait complete. 2024-09-19T05:44:40.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.965+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff70406c2e0 msgr2=0x7ff70406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.965+0000 7ff72754b640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff70406c2e0 0x7ff70406e7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00a440 tx=0x7ff72009a000 comp rx=0 tx=0).stop 2024-09-19T05:44:40.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718048db0 msgr2=0x7ff7180f7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718048db0 0x7ff7180f7ab0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff71000c3f0 tx=0x7ff71000c7f0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 shutdown_connections 2024-09-19T05:44:40.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff70406c2e0 0x7ff70406e7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718047e40 0x7ff71804ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718049190 0x7ff718047900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 --2- 172.21.15.2:0/1467127618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718048db0 0x7ff7180f7ab0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 >> 172.21.15.2:0/1467127618 conn(0x7ff71804fc50 msgr2=0x7ff71804b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 shutdown_connections 2024-09-19T05:44:40.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.966+0000 7ff72754b640 1 -- 172.21.15.2:0/1467127618 wait complete. 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.970+0000 7f947c4c3640 1 Processor -- start 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.972+0000 7f947c4c3640 1 -- start start 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.973+0000 7f947c4c3640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94740c34d0 0x7f94740c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.973+0000 7f947c4c3640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94741528e0 0x7f947415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.973+0000 7f947c4c3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9474152cc0 0x7f94741f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.973+0000 7f947c4c3640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f947415c8c0 con 0x7f94741528e0 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.973+0000 7f947c4c3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f947415bdd0 con 0x7f9474152cc0 2024-09-19T05:44:40.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.973+0000 7f947c4c3640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f94740c8f20 con 0x7f94740c34d0 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.974+0000 7f947aa60640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9474152cc0 0x7f94741f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.974+0000 7f947aa60640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9474152cc0 0x7f94741f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:44332/0 (socket says 172.21.15.2:44332) 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.974+0000 7f947aa60640 1 -- 172.21.15.2:0/751288451 learned_addr learned my addr 172.21.15.2:0/751288451 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.974+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94741528e0 0x7f947415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.975+0000 7f9479a5e640 1 -- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94740c34d0 msgr2=0x7f94740c82b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.975+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94740c34d0 0x7f94740c82b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.975+0000 7f9479a5e640 1 -- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9474152cc0 msgr2=0x7f94741f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.975+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9474152cc0 0x7f94741f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:40.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.975+0000 7f9479a5e640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94740b25c0 con 0x7f94741528e0 2024-09-19T05:44:40.979 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.975+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94741528e0 0x7f947415a2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f946400c440 tx=0x7f946400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:40.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.983+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9464017070 con 0x7f94741528e0 2024-09-19T05:44:40.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.983+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94740c87f0 con 0x7f94741528e0 2024-09-19T05:44:40.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.983+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94740b2840 con 0x7f94741528e0 2024-09-19T05:44:40.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.983+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94640035f0 con 0x7f94741528e0 2024-09-19T05:44:40.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.983+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94640039e0 con 0x7f94741528e0 2024-09-19T05:44:40.994 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:40.985+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f945c005270 con 0x7f94741528e0 2024-09-19T05:44:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.001+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f946401a070 con 0x7f94741528e0 2024-09-19T05:44:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.002+0000 7f946affd640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f9458074bc0 0x7f9458077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.004 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.002+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5799+0+0 (secure 0 0 0) 0x7f946408a780 con 0x7f94741528e0 2024-09-19T05:44:41.007 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.002+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v74) ==== 54+0+16 (secure 0 0 0) 0x7f94740b25c0 con 0x7f94741528e0 2024-09-19T05:44:41.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.005+0000 7f947a25f640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f9458074bc0 0x7f9458077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.008 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.005+0000 7f947c4c3640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f945c000fa0 0x7f945c0045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.006+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f945c004b20 con 0x7f945c000fa0 2024-09-19T05:44:41.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.007+0000 7f947a25f640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f9458074bc0 0x7f9458077080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9470034000 tx=0x7f947002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.013 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:41.002+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:41.106 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:41.099+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:41.118 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:41.118 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:41.151 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:41.201 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:41.200+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:41.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.201+0000 7f947aa60640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f945c000fa0 0x7f945c0045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.206+0000 7f947aa60640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f945c000fa0 0x7f945c0045e0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.225 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.219+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f945c004b20 con 0x7f945c000fa0 2024-09-19T05:44:41.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.262+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f945c005090 con 0x7f945c000fa0 2024-09-19T05:44:41.307 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:41.302+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:41.345 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.341+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f945c005090 con 0x7f945c000fa0 2024-09-19T05:44:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.343+0000 7f947c4c3640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f945c001ca0 0x7f945c002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.346 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.343+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f945c0022b0 con 0x7f945c001ca0 2024-09-19T05:44:41.351 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.347+0000 7f947a25f640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f945c001ca0 0x7f945c002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.357+0000 7f947a25f640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f945c001ca0 0x7f945c002990 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.360+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f945c0022b0 con 0x7f945c001ca0 2024-09-19T05:44:41.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.408+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f945c0031a0 con 0x7f945c001ca0 2024-09-19T05:44:41.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.412+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f945c0031a0 con 0x7f945c001ca0 2024-09-19T05:44:41.418 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:41.418 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:41.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.416+0000 7f9468ff9640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f9450001650 0x7f9450003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.416+0000 7f9468ff9640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f9450006bb0 con 0x7f9450001650 2024-09-19T05:44:41.422 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.420+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f9450001650 0x7f9450003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.422 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:41.403+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:41.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.421+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f9450001650 0x7f9450003b10 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.422+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9450006bb0 con 0x7f9450001650 2024-09-19T05:44:41.466 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.462+0000 7f9468ff9640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9450005bd0 con 0x7f9450001650 2024-09-19T05:44:41.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.467+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9450005bd0 con 0x7f9450001650 2024-09-19T05:44:41.470 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:41.470 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:41.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.470+0000 7f9468ff9640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f9450004190 0x7f9450004600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.470+0000 7f9468ff9640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9450004b40 con 0x7f9450004190 2024-09-19T05:44:41.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.470+0000 7f947aa60640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f9450004190 0x7f9450004600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.471+0000 7f947aa60640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f9450004190 0x7f9450004600 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.472+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9450004b40 con 0x7f9450004190 2024-09-19T05:44:41.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.501+0000 7f9468ff9640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9450004fd0 con 0x7f9450004190 2024-09-19T05:44:41.509 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:41.509 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:41.509 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:41.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.503+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9450004fd0 con 0x7f9450004190 2024-09-19T05:44:41.510 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:41.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.505+0000 7f947c4c3640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f945000a480 0x7f945000c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.505+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f945000f740 con 0x7f945000a480 2024-09-19T05:44:41.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.507+0000 7f947a25f640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f945000a480 0x7f945000c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.514 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:41.512+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:41.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.512+0000 7f947a25f640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f945000a480 0x7f945000c870 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.513+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f945000f740 con 0x7f945000a480 2024-09-19T05:44:41.535 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:41.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.544+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f945000f710 con 0x7f945000a480 2024-09-19T05:44:41.553 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:41.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.549+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f945000f710 con 0x7f945000a480 2024-09-19T05:44:41.553 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:41.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.552+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=75}) -- 0x7f945000f720 con 0x7f94741528e0 2024-09-19T05:44:41.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.552+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f945000fc00 con 0x7f94741528e0 2024-09-19T05:44:41.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.553+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f9464091070 con 0x7f94741528e0 2024-09-19T05:44:41.556 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:41.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.554+0000 7f947c4c3640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f945000d680 0x7f945000dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.554+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f945000f420 con 0x7f945000d680 2024-09-19T05:44:41.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.555+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f945000d680 0x7f945000dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.557+0000 7f9479a5e640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f945000d680 0x7f945000dac0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.558+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f945000f420 con 0x7f945000d680 2024-09-19T05:44:41.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.590+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f945000e140 con 0x7f945000d680 2024-09-19T05:44:41.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.593+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f945000e140 con 0x7f945000d680 2024-09-19T05:44:41.595 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:41.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:41.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.594+0000 7f947c4c3640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f945000e8b0 0x7f945000ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:41.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.594+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f94500131e0 con 0x7f945000e8b0 2024-09-19T05:44:41.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.594+0000 7f947aa60640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f945000e8b0 0x7f945000ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:41.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.599+0000 7f947aa60640 1 --2- 172.21.15.2:0/751288451 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f945000e8b0 0x7f945000ed30 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:41.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.600+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94500131e0 con 0x7f945000e8b0 2024-09-19T05:44:41.611 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:41.604+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:41.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.628+0000 7f947c4c3640 1 -- 172.21.15.2:0/751288451 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9450011fe0 con 0x7f945000e8b0 2024-09-19T05:44:41.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:41.629+0000 7f946affd640 1 -- 172.21.15.2:0/751288451 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9450011fe0 con 0x7f945000e8b0 2024-09-19T05:44:41.632 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:41.632 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:41.639 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:41.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:41.713 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:41.705+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:41.813 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:41.806+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:41.858 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:41.859 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:41.889 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:41.910 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:41.908+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:42.010 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:42.008+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:42.120 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:42.109+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:42.193 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2140 3.3c 3:3ebd280f:::100000001f8.00000459:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.596288962999999, 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0038021129999999, 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2140 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641694+0000", 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049504800000000002 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646652+0000", 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049580040000000002 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646670+0000", 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646681+0000", 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1555e-05 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429025+0000", 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.782343268 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429156+0000", 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013093299999999999 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.435554+0000", 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063981860000000002 2024-09-19T05:44:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549237+0000", 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113683458 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595633+0000", 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046396021000000003 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595950+0000", 2024-09-19T05:44:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000316514 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595991+0000", 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1430000000000001e-05 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2232 2.14 2:2e2fa760:::200.00000001:head [write 2396859~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.385574601, 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9689986130000001, 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802904+0000", 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-07 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802912+0000", 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9489999999999996e-06 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802920+0000", 2024-09-19T05:44:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5970000000000003e-06 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802925+0000", 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5040000000000002e-06 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730249+0000", 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9273239709999999 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730352+0000", 2024-09-19T05:44:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102648 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730514+0000", 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016198499999999999 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755135+0000", 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024621601 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771776+0000", 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016640773000000001 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771841+0000", 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5178000000000005e-05 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771902+0000", 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0825999999999997e-05 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2233 2.14 2:2e2fa760:::200.00000001:head [write 2398939~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.385544505, 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969058076, 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807300+0000", 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043666679999999998 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807312+0000", 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2199e-05 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807327+0000", 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4260999999999999e-05 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807339+0000", 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737540+0000", 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9302006619999998 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737660+0000", 2024-09-19T05:44:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119881 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737878+0000", 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218614 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755190+0000", 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017311756000000001 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771913+0000", 2024-09-19T05:44:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016722954000000002 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771958+0000", 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123000000000001e-05 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771992+0000", 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3657e-05 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2234 2.14 2:2e2fa760:::200.00000001:head [write 2401019~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.381090753000001, 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9661180190000001, 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807388+0000", 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-07 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807399+0000", 2024-09-19T05:44:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1338999999999999e-05 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807409+0000", 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807416+0000", 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1180000000000003e-06 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743599+0000", 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9361828380000001 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743702+0000", 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103303 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743868+0000", 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771998+0000", 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028130420999999999 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773391+0000", 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013930699999999999 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773448+0000", 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6672999999999998e-05 2024-09-19T05:44:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773505+0000", 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7445000000000003e-05 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2235 2.14 2:2e2fa760:::200.00000001:head [write 2403099~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.381051539, 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9662057229999998, 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4000000000000002e-07 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807435+0000", 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.072e-06 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807478+0000", 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2460999999999999e-05 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807486+0000", 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5350000000000004e-06 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.745914+0000", 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9384274700000002 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.747669+0000", 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017549239999999999 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.748294+0000", 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00062547500000000001 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773338+0000", 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025043950999999998 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773551+0000", 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021271199999999999 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773599+0000", 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7876e-05 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773632+0000", 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3707000000000001e-05 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2236 2.14 2:2e2fa760:::200.00000001:head [write 2405179~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.380981422, 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9899550850000001, 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807502+0000", 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4850000000000003e-06 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807509+0000", 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0859999999999996e-06 2024-09-19T05:44:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807515+0000", 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9179999999999999e-06 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755230+0000", 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9477156679999998 2024-09-19T05:44:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755342+0000", 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111703 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755522+0000", 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000179937 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.783617+0000", 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028095472 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797341+0000", 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013723506999999999 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797404+0000", 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797452+0000", 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7567e-05 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2237 2.14 2:2e2fa760:::200.00000001:head [write 2407259~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.380956748999999, 2024-09-19T05:44:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0092068749999998, 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807522+0000", 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807530+0000", 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8930000000000005e-06 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807577+0000", 2024-09-19T05:44:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7064999999999997e-05 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807606+0000", 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8963e-05 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761580+0000", 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.953974901 2024-09-19T05:44:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761685+0000", 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010435800000000001 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761827+0000", 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142313 2024-09-19T05:44:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792591+0000", 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030764073999999999 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816632+0000", 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024040371000000001 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816688+0000", 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5951e-05 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816728+0000", 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0636000000000002e-05 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2238 2.14 2:2e2fa760:::200.00000001:head [write 2409339~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.380858893999999, 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0122208439999998, 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834275+0000", 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026656197999999999 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834287+0000", 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1691e-05 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834301+0000", 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3974e-05 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834310+0000", 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8659999999999995e-06 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772040+0000", 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9377302950000002 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772148+0000", 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108136 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772314+0000", 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016526400000000001 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792597+0000", 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020283483000000001 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819691+0000", 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027094025000000001 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819771+0000", 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0022000000000001e-05 2024-09-19T05:44:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819840+0000", 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8889999999999999e-05 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2239 2.14 2:2e2fa760:::200.00000001:head [write 2411419~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.354148910999999, 2024-09-19T05:44:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.985646596, 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851429+0000", 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017099436999999999 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851444+0000", 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5449000000000001e-05 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851460+0000", 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.556e-05 2024-09-19T05:44:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851470+0000", 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0001e-05 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773666+0000", 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9221966369999999 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773757+0000", 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1160000000000001e-05 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773902+0000", 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014419500000000001 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795500+0000", 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021598346000000001 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819898+0000", 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024398474 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819942+0000", 2024-09-19T05:44:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3151000000000001e-05 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819976+0000", 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4186000000000001e-05 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2240 2.14 2:2e2fa760:::200.00000001:head [write 2413499~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.336990189, 2024-09-19T05:44:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9685921560000001, 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862188+0000", 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010699774 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862213+0000", 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5536000000000001e-05 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862225+0000", 2024-09-19T05:44:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1328000000000001e-05 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862231+0000", 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.55e-06 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.780995+0000", 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.918763854 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781115+0000", 2024-09-19T05:44:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120538 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781261+0000", 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145785 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795505+0000", 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014244158 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820012+0000", 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024506607 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820051+0000", 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8597000000000003e-05 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820080+0000", 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9428999999999999e-05 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2245 2.14 2:2e2fa760:::200.00000001:head [write 2423899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.326114274, 2024-09-19T05:44:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969609441, 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.08e-07 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862368+0000", 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7929999999999999e-06 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862375+0000", 2024-09-19T05:44:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6680000000000004e-06 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862380+0000", 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797491+0000", 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9351107490000001 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797581+0000", 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9559000000000001e-05 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797723+0000", 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014230199999999999 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820590+0000", 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022867386 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827763+0000", 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071727190000000001 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.828797+0000", 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010343570000000001 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831973+0000", 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003175925 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2246 2.14 2:2e2fa760:::200.00000001:head [write 2425979~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.32609036, 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9696936549999999, 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.96e-07 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862397+0000", 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862404+0000", 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862409+0000", 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1320000000000002e-06 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804733+0000", 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.94232443 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804840+0000", 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106724 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804998+0000", 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015801500000000001 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826045+0000", 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021046664999999999 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831983+0000", 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059387570000000002 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832038+0000", 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4475999999999998e-05 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832081+0000", 2024-09-19T05:44:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3591000000000003e-05 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2247 2.14 2:2e2fa760:::200.00000001:head [write 2428059~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.326063279, 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969738048, 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-19T05:44:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862426+0000", 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1090999999999999e-05 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862433+0000", 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8070000000000002e-06 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862438+0000", 2024-09-19T05:44:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1599999999999997e-06 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806006+0000", 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.943567885 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806105+0000", 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8782000000000004e-05 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806289+0000", 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183678 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827739+0000", 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021450258999999999 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832087+0000", 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043483280000000003 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832124+0000", 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6816000000000001e-05 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832153+0000", 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.887e-05 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2248 2.14 2:2e2fa760:::200.00000001:head [write 2430139~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.326033281000001, 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9698627229999999, 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877451+0000", 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015005956000000001 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877461+0000", 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0034000000000001e-05 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.896137+0000", 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018675967000000002 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909217+0000", 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013080629999999999 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807553+0000", 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8983351339999999 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807647+0000", 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4289000000000005e-05 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807794+0000", 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000147467 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832160+0000", 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024365753 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832224+0000", 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3689000000000006e-05 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832274+0000", 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0649000000000002e-05 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832308+0000", 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3154999999999997e-05 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.275 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2250 2.14 2:2e2fa760:::200.00000001:head [write 2434299~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.278592431, 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9595307040000001, 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939724+0000", 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029838587 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939743+0000", 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9077e-05 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941640+0000", 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018968310000000001 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941651+0000", 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0379e-05 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814230+0000", 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.872579934 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814430+0000", 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199481 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814571+0000", 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014074399999999999 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832173+0000", 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017602199999999998 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869325+0000", 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037152223999999998 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869374+0000", 2024-09-19T05:44:42.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9149e-05 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869416+0000", 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2098000000000002e-05 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2337 3.3c 3:3eebf982:::100000001f8.0000051e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:42.281 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.176446291, 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.014058033, 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2337 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.018343+0000", 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063114770000000002 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030374+0000", 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012031086999999999 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030396+0000", 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.156e-05 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030411+0000", 2024-09-19T05:44:42.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4875000000000001e-05 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897205+0000", 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8667940299999999 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897418+0000", 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021300400000000001 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897573+0000", 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015481300000000001 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.943251+0000", 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045678424000000002 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026021+0000", 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082769669000000004 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026059+0000", 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8296000000000001e-05 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026090+0000", 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0797999999999999e-05 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2267 2.14 2:2e2fa760:::200.00000001:head [write 2469659~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.119804023, 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9703313549999999, 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076283+0000", 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076092460000000001 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076295+0000", 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1857e-05 2024-09-19T05:44:42.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076309+0000", 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3661e-05 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076319+0000", 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0589e-05 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963409+0000", 2024-09-19T05:44:42.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.887089915 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963490+0000", 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1137000000000006e-05 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963615+0000", 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012412999999999999 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012633+0000", 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049018171999999999 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038886+0000", 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026253216999999999 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038950+0000", 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3876999999999998e-05 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039005+0000", 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5553999999999997e-05 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2268 2.14 2:2e2fa760:::200.00000001:head [write 2471739~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.112128524999999, 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962740785, 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076361+0000", 2024-09-19T05:44:42.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076368+0000", 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4240000000000004e-06 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076374+0000", 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0179999999999998e-06 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975236+0000", 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.898862345 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975332+0000", 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5217999999999995e-05 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975458+0000", 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125986 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012673+0000", 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037215224999999998 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039014+0000", 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026341256 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039056+0000", 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1944999999999999e-05 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039090+0000", 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4301999999999998e-05 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2269 2.14 2:2e2fa760:::200.00000001:head [write 2473819~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.112094967000001, 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9627868230000001, 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076384+0000", 2024-09-19T05:44:42.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076392+0000", 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.174e-06 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076398+0000", 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4309999999999999e-06 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076403+0000", 2024-09-19T05:44:42.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6399999999999996e-06 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976688+0000", 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9002852180000001 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976785+0000", 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6963e-05 2024-09-19T05:44:42.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976951+0000", 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016594799999999999 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012708+0000", 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035757376 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039097+0000", 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026388551 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039137+0000", 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0025999999999999e-05 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039170+0000", 2024-09-19T05:44:42.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3065000000000003e-05 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2270 2.14 2:2e2fa760:::200.00000001:head [write 2475899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.112069002, 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962839365, 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102351+0000", 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025941703999999999 2024-09-19T05:44:42.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102359+0000", 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102372+0000", 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3749e-05 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102382+0000", 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0229999999999998e-06 2024-09-19T05:44:42.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978315+0000", 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.875933294 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978422+0000", 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107007 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978575+0000", 2024-09-19T05:44:42.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015296099999999999 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012746+0000", 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034170849000000003 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039176+0000", 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026430282999999999 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:42.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039215+0000", 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9368e-05 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039248+0000", 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3157000000000001e-05 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.310 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:42.312 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:42.313 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:42.310+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:42.427 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:42.418+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:42.517 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:42.512+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:42.615 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:42.613+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:42.632 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:42.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:4982 3.30 3:0ca3d5de:::100000001f8.00000573:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.414539+0000", 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.21452349400000001, 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21459219299999999, 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 4982 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.414539+0000", 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.414539+0000", 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.414542+0000", 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0259999999999999e-06 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.416452+0000", 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001909509 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.416481+0000", 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9122999999999999e-05 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.416500+0000", 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9624999999999999e-05 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.416796+0000", 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029611500000000001 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.416934+0000", 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013759899999999999 2024-09-19T05:44:42.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.417169+0000", 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023513700000000001 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.498783+0000", 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.081614056000000004 2024-09-19T05:44:42.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:4984 3.14 3:2ba62aaa:::100000001f8.00000575:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.453189+0000", 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.17587317999999999, 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.176062886, 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 4984 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.453189+0000", 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.453189+0000", 2024-09-19T05:44:42.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.453193+0000", 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5769999999999998e-06 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.456909+0000", 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037156440000000002 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.456935+0000", 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6662e-05 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.456949+0000", 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3658999999999999e-05 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.457056+0000", 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107454 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.457158+0000", 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010128000000000001 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.457352+0000", 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000194494 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.482984+0000", 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025631688999999999 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:4989 3.9 3:937f2b07:::100000001f8.0000057a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.480717+0000", 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.148345214, 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.14864047999999999, 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 4989 2024-09-19T05:44:42.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.480717+0000", 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.480717+0000", 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.481398+0000", 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00068090900000000005 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.485575+0000", 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041772689999999999 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.485609+0000", 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3422e-05 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.485628+0000", 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8635999999999999e-05 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.486862+0000", 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001234436 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.486981+0000", 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119486 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.487176+0000", 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000194248 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.591076+0000", 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103900517 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5001 3.11 3:89e800ba:::100000001f8.00000586:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.491743+0000", 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.137319951, 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13769880400000001, 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5001 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.491743+0000", 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.491743+0000", 2024-09-19T05:44:42.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.491745+0000", 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5490000000000001e-06 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495236+0000", 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034908460000000001 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495256+0000", 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0509e-05 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495280+0000", 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3096e-05 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495633+0000", 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035318699999999998 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495771+0000", 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013792500000000001 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.522029+0000", 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026258014999999999 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.583871+0000", 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.061842674 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5012 3.d 3:b28c6918:::100000001f8.00000591:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.495312+0000", 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.13375005700000001, 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.134206187, 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5012 2024-09-19T05:44:42.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495312+0000", 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495312+0000", 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.495315+0000", 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7539999999999998e-06 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.497585+0000", 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002270162 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.497623+0000", 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7746000000000001e-05 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.497655+0000", 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1894999999999999e-05 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.497906+0000", 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025121699999999998 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.498076+0000", 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000170144 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.498319+0000", 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024228699999999999 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.553077+0000", 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054758165999999997 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5013 3.14 3:2af86730:::100000001f8.00000592:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.498187+0000", 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.13087594299999999, 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13140843399999999, 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.658 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5013 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.498187+0000", 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.498187+0000", 2024-09-19T05:44:42.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.498189+0000", 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.915e-06 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.501512+0000", 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0033220189999999998 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.501539+0000", 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7809000000000001e-05 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.501554+0000", 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4866e-05 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.501693+0000", 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013850700000000001 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.501815+0000", 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012274399999999999 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.502150+0000", 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000334996 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.567185+0000", 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.065034579999999995 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.662 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5025 3.1d 3:bb151557:::100000001f8.0000059e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.505643+0000", 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.123419554, 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12403033300000001, 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5025 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.663 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.505643+0000", 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.505643+0000", 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.505647+0000", 2024-09-19T05:44:42.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6969999999999999e-06 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.521253+0000", 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015606261 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.540841+0000", 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019587638000000001 2024-09-19T05:44:42.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.540871+0000", 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0309999999999999e-05 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.541045+0000", 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017453600000000001 2024-09-19T05:44:42.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.541173+0000", 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012789400000000001 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.541364+0000", 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019099699999999999 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.667 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5026 3.2d 3:b7ef5b08:::100000001f8.0000059f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.540897+0000", 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.088165073999999996, 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.088847870999999995, 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5026 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.540897+0000", 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.540897+0000", 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.540900+0000", 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8540000000000001e-06 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.544577+0000", 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003676374 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.544600+0000", 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3527e-05 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.544614+0000", 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3869999999999999e-05 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.553150+0000", 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0085364120000000002 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.553329+0000", 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017819299999999999 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.553607+0000", 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027873900000000002 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5030 3.1d 3:b940ac0e:::100000001f8.000005a3:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.544642+0000", 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.084420149, 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.085175133, 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5030 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.544642+0000", 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.544642+0000", 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.544645+0000", 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9009999999999998e-06 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.549289+0000", 2024-09-19T05:44:42.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046435540000000003 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.549339+0000", 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0658999999999997e-05 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.549354+0000", 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4168000000000001e-05 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.566169+0000", 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016815513000000001 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.566764+0000", 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00059481400000000004 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.567019+0000", 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025530099999999999 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5034 3.17 3:ebb2ffd0:::100000001f8.000005a7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.549380+0000", 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.079682510999999998, 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.080508142000000005, 2024-09-19T05:44:42.677 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5034 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.549380+0000", 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.549380+0000", 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.549383+0000", 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5610000000000001e-06 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571569+0000", 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022186146 2024-09-19T05:44:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571594+0000", 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4947999999999999e-05 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571610+0000", 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6004000000000001e-05 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571803+0000", 2024-09-19T05:44:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000193427 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571937+0000", 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013393199999999999 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5046 3.2d 3:b72b662f:::100000001f8.000005b3:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.571662+0000", 2024-09-19T05:44:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.057400991999999998, 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05829256, 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5046 2024-09-19T05:44:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571662+0000", 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571662+0000", 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.571665+0000", 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5140000000000002e-06 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.573677+0000", 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020122930000000001 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.573719+0000", 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1732e-05 2024-09-19T05:44:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.573733+0000", 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3582e-05 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.626334+0000", 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052601850999999998 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.626446+0000", 2024-09-19T05:44:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111635 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.626622+0000", 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000176348 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:4987 3.26 e74/20 3:6642ca4e:::100000001f8.00000578:head v 74'115, mlcod=74'112)", 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.574388+0000", 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.054674876999999997, 2024-09-19T05:44:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.055654242999999999, 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.574388+0000", 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.574388+0000", 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.574388+0000", 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8000000000000001e-07 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578714+0000", 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043262919999999998 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578734+0000", 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9264000000000001e-05 2024-09-19T05:44:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578745+0000", 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.145e-05 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578905+0000", 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015961999999999999 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578933+0000", 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8302000000000001e-05 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:4992 3.16 e74/20 3:6bde259d:::100000001f8.0000057d:head v 74'141, mlcod=74'136)", 2024-09-19T05:44:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.578806+0000", 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.050256579000000003, 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.051297285999999997, 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578806+0000", 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578806+0000", 2024-09-19T05:44:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.578806+0000", 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6100000000000001e-07 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.581110+0000", 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023032970000000002 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.583109+0000", 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019996929999999999 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.583149+0000", 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9969000000000003e-05 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.585097+0000", 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019470749999999999 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.585126+0000", 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9646999999999999e-05 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:5006 3.26 e74/20)", 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.591095+0000", 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.037967819999999999, 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039073774999999998, 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.591095+0000", 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.591095+0000", 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.591095+0000", 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1799999999999995e-07 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.593859+0000", 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027634920000000002 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.593886+0000", 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7512e-05 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.593903+0000", 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6900000000000001e-05 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5057 3.17 3:eaf2c8b6:::100000001f8.000005be:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.573757+0000", 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.055305769999999997, 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.056456654000000002, 2024-09-19T05:44:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5057 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.573757+0000", 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.573757+0000", 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.573760+0000", 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7870000000000002e-06 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.606847+0000", 2024-09-19T05:44:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033087552999999999 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.606884+0000", 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7149e-05 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.606899+0000", 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4579999999999999e-05 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.607813+0000", 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00091370100000000005 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.607938+0000", 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125666 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5061 3.2f 3:f4e1ea89:::100000001f8.000005c2:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.606934+0000", 2024-09-19T05:44:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022128480999999998, 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.023351664000000001, 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5061 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.606934+0000", 2024-09-19T05:44:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.606934+0000", 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.606937+0000", 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3280000000000002e-06 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.611772+0000", 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048344199999999999 2024-09-19T05:44:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.611828+0000", 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6039999999999999e-05 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.611843+0000", 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4914e-05 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.614243+0000", 2024-09-19T05:44:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023999049999999999 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.614440+0000", 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019709999999999999 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.614714+0000", 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027378800000000002 2024-09-19T05:44:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5062 3.13 3:c9096c28:::100000001f8.000005c3:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.611885+0000", 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017177735, 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01848812, 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5062 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.611885+0000", 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.611885+0000", 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.611888+0000", 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.145e-06 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.617084+0000", 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051956160000000001 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.617105+0000", 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1178e-05 2024-09-19T05:44:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.617117+0000", 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2017e-05 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.617919+0000", 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00080224500000000004 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.618088+0000", 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016938500000000001 2024-09-19T05:44:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.618327+0000", 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000238149 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5064 3.13 3.3407dc93 (undecoded) ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.617149+0000", 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011913755, 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013307716000000001, 2024-09-19T05:44:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5064 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.617149+0000", 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.617149+0000", 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.617152+0000", 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.128e-06 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.619488+0000", 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023357220000000001 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.619507+0000", 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9218000000000001e-05 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.619519+0000", 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2387e-05 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 18 2024-09-19T05:44:42.713 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:42.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:42.716 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:42.714+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:42.823 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:42.821+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:42.933 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:42.920+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:43.010 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:44:43.060 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:43.056+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:43.103 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:43.103 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:43.103 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:43.120 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:43.118+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:43.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:43.264 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:43.236+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:43.329 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:43.320+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.359+0000 7f7a1108f640 1 Processor -- start 2024-09-19T05:44:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.360+0000 7f7a1108f640 1 -- start start 2024-09-19T05:44:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.364+0000 7f7a1108f640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7a0c0b5b10 0x7f7a0c0d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.364+0000 7f7a1108f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.364+0000 7f7a1108f640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a0c0c9b00 0x7f7a0c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.364+0000 7f7a1108f640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f7a0c0c85b0 con 0x7f7a0c0b5b10 2024-09-19T05:44:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.364+0000 7f7a1108f640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7a0c159560 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.364+0000 7f7a1108f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0c0c8870 con 0x7f7a0c0d0820 2024-09-19T05:44:43.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.365+0000 7f7a0b7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:43.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.365+0000 7f7a1088e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a0c0c9b00 0x7f7a0c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:43.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.365+0000 7f7a1088e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a0c0c9b00 0x7f7a0c0cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:38660/0 (socket says 172.21.15.2:38660) 2024-09-19T05:44:43.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.365+0000 7f7a1088e640 1 -- 172.21.15.2:0/822916920 learned_addr learned my addr 172.21.15.2:0/822916920 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.366+0000 7f7a1088e640 1 -- 172.21.15.2:0/822916920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 msgr2=0x7f7a0c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.366+0000 7f7a1088e640 1 --2- 172.21.15.2:0/822916920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c0c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.366+0000 7f7a1088e640 1 -- 172.21.15.2:0/822916920 >> v2:172.21.15.2:3300/0 conn(0x7f7a0c0b5b10 msgr2=0x7f7a0c0d02e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:44:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.366+0000 7f7a1088e640 1 --2- 172.21.15.2:0/822916920 >> v2:172.21.15.2:3300/0 conn(0x7f7a0c0b5b10 0x7f7a0c0d02e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.366+0000 7f7a1088e640 1 -- 172.21.15.2:0/822916920 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c0c9f50 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.368+0000 7f7a1088e640 1 --2- 172.21.15.2:0/822916920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c0cc9d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c08e4c0 tx=0x7f79fc02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b850f026b24ebb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:43.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.370+0000 7f7a0affd640 1 -- 172.21.15.2:0/822916920 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc037070 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.370+0000 7f7a0affd640 1 -- 172.21.15.2:0/822916920 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79fc032440 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.372+0000 7f7a1108f640 1 -- 172.21.15.2:0/822916920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0c9b00 msgr2=0x7f7a0c0cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:43.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.372+0000 7f7a1108f640 1 --2- 172.21.15.2:0/822916920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c0cc9d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c08e4c0 tx=0x7f79fc02a210 comp rx=0 tx=0).stop 2024-09-19T05:44:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.373+0000 7f7a1108f640 1 -- 172.21.15.2:0/822916920 shutdown_connections 2024-09-19T05:44:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.373+0000 7f7a1108f640 1 --2- 172.21.15.2:0/822916920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c0cc9d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.373+0000 7f7a1108f640 1 --2- 172.21.15.2:0/822916920 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.373+0000 7f7a1108f640 1 --2- 172.21.15.2:0/822916920 >> v2:172.21.15.2:3300/0 conn(0x7f7a0c0b5b10 0x7f7a0c0d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.373+0000 7f7a1108f640 1 -- 172.21.15.2:0/822916920 >> 172.21.15.2:0/822916920 conn(0x7f7a0c06cb70 msgr2=0x7f7a0c06e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:43.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.374+0000 7f7a1108f640 1 -- 172.21.15.2:0/822916920 shutdown_connections 2024-09-19T05:44:43.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.374+0000 7f7a1108f640 1 -- 172.21.15.2:0/822916920 wait complete. 2024-09-19T05:44:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.383+0000 7f7a1108f640 1 Processor -- start 2024-09-19T05:44:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.383+0000 7f7a1108f640 1 -- start start 2024-09-19T05:44:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a1108f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0b5b10 0x7f7a0c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:43.387 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a1108f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a1108f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a1108f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c0c8ae0 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a1108f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c0cda40 con 0x7f7a0c0b5b10 2024-09-19T05:44:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a1108f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0c0c8dc0 con 0x7f7a0c0d0820 2024-09-19T05:44:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a0b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c15b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60210/0 (socket says 172.21.15.2:60210) 2024-09-19T05:44:43.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a0b7fe640 1 -- 172.21.15.2:0/1451476654 learned_addr learned my addr 172.21.15.2:0/1451476654 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.385+0000 7f7a1088e640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.386+0000 7f7a0b7fe640 1 -- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 msgr2=0x7f7a0c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.386+0000 7f7a0b7fe640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.386+0000 7f7a0b7fe640 1 -- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0b5b10 msgr2=0x7f7a0c15ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.386+0000 7f7a0b7fe640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0b5b10 0x7f7a0c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.386+0000 7f7a0b7fe640 1 -- 172.21.15.2:0/1451476654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c070700 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.387+0000 7f7a0b7fe640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c15b900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7a04007e90 tx=0x7f7a0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:43.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.389+0000 7f7a08ff9640 1 -- 172.21.15.2:0/1451476654 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04014420 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.390+0000 7f7a08ff9640 1 -- 172.21.15.2:0/1451476654 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a04019070 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.390+0000 7f7a08ff9640 1 -- 172.21.15.2:0/1451476654 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040147e0 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.391+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0c0c6a90 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.394 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.391+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0c06f1f0 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.393+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a0c0cb4e0 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.400+0000 7f7a08ff9640 1 -- 172.21.15.2:0/1451476654 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7a04014980 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.405 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.402+0000 7f7a08ff9640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f806c380 0x7f79f806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:43.407 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.404+0000 7f7a08ff9640 1 -- 172.21.15.2:0/1451476654 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5799+0+0 (secure 0 0 0) 0x7f7a0408cb00 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.407+0000 7f7a08ff9640 1 -- 172.21.15.2:0/1451476654 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7a0c070700 con 0x7f7a0c0c9b00 2024-09-19T05:44:43.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.409+0000 7f7a0bfff640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f806c380 0x7f79f806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:43.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:43.422+0000 7f7a0bfff640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f806c380 0x7f79f806e840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7a000096f0 tx=0x7f7a00006c40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:43.429 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:43.427+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:43.556 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:43.523+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:43.623 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:43.623 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:43.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:43.623 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:43.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.623 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2653 3.9 3:916fb10b:::100000001f8.0000065a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.799828+0000", 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.818763151000001, 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1159312100000001, 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2653 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.799828+0000", 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.799828+0000", 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.799830+0000", 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4439999999999998e-06 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.841194+0000", 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041363977000000003 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.841211+0000", 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7343e-05 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.841223+0000", 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1669e-05 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.851903+0000", 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010679953000000001 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.853016+0000", 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011128010000000001 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.853207+0000", 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019156500000000001 2024-09-19T05:44:43.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.879156+0000", 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025949014999999999 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.915650+0000", 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0364931740000001 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.915707+0000", 2024-09-19T05:44:43.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7129999999999997e-05 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.915759+0000", 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2138999999999997e-05 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2666 3.9 3:92ca8d9b:::100000001f8.00000667:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.723866243, 2024-09-19T05:44:43.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3293169570000001, 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2666 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894727+0000", 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8439999999999998e-06 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897799+0000", 2024-09-19T05:44:43.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030718199999999998 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897817+0000", 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7847999999999999e-05 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897831+0000", 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3845999999999999e-05 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897938+0000", 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010742399999999999 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898111+0000", 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017279099999999999 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898360+0000", 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024885800000000002 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956644+0000", 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058284217999999999 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223960+0000", 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.267316119 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224004+0000", 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4039000000000003e-05 2024-09-19T05:44:43.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224042+0000", 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7150000000000002e-05 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2667 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.720726395, 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3258909560000001, 2024-09-19T05:44:43.635 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2667 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897867+0000", 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9299999999999999e-06 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902567+0000", 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004699356 2024-09-19T05:44:43.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902588+0000", 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1631e-05 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902602+0000", 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3478e-05 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.906894+0000", 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042925760000000002 2024-09-19T05:44:43.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907078+0000", 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183067 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907289+0000", 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021123499999999999 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956671+0000", 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049382078000000003 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223652+0000", 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.266981122 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223704+0000", 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2244999999999999e-05 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223755+0000", 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1238e-05 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2673 3.2d 3:b45a3f48:::100000001f8.0000066e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.707728968, 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.373980822, 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2673 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:43.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910864+0000", 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915100+0000", 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042352259999999999 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915123+0000", 2024-09-19T05:44:43.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3334999999999999e-05 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915137+0000", 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4178999999999999e-05 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915220+0000", 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2693000000000003e-05 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915395+0000", 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017551099999999999 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915574+0000", 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178357 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951064+0000", 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035490116000000002 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284764+0000", 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.333700259 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284808+0000", 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3813999999999998e-05 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284843+0000", 2024-09-19T05:44:43.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4807000000000001e-05 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2700 3.30 3:0e790ae4:::100000001f8.00000689:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.675026866, 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.25937777, 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2700 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943567+0000", 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.083e-06 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946402+0000", 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028349439999999998 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946419+0000", 2024-09-19T05:44:43.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7105999999999999e-05 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946429+0000", 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8709999999999992e-06 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.947951+0000", 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015219960000000001 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948153+0000", 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020153400000000001 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948415+0000", 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026250200000000002 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.064312+0000", 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115897369 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202811+0000", 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.138498891 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202872+0000", 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0529000000000003e-05 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202942+0000", 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9944999999999995e-05 2024-09-19T05:44:43.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2711 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.667509257000001, 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.27282242, 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2711 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951084+0000", 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.553e-06 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956906+0000", 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058222400000000002 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956926+0000", 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9619e-05 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956937+0000", 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0594999999999999e-05 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958546+0000", 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001609369 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958792+0000", 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024587600000000002 2024-09-19T05:44:43.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.959831+0000", 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010394079999999999 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293509+0000", 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33367725100000001 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223817+0000", 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93030886999999995 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223864+0000", 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6465000000000003e-05 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223904+0000", 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0173999999999999e-05 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2732 3.9 3:92ca8380:::100000001f8.000006a9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.610639653, 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2619013319999999, 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2732 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.659 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007954+0000", 2024-09-19T05:44:43.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018132+0000", 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010177734000000001 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018148+0000", 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6555999999999999e-05 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018187+0000", 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8760000000000002e-05 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020347+0000", 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021597589999999998 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020514+0000", 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016697899999999999 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020695+0000", 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018172199999999999 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.375158+0000", 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35446218200000001 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269697+0000", 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89453943599999997 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269768+0000", 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0619999999999998e-05 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269853+0000", 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4876000000000005e-05 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2746 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.583102440999999, 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1890195509999999, 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2746 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:43.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035491+0000", 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043360+0000", 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0078693240000000005 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043383+0000", 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2853e-05 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.045231+0000", 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001847476 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.122884+0000", 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077653244999999996 2024-09-19T05:44:43.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123275+0000", 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039126200000000002 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123895+0000", 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061987200000000004 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293518+0000", 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.169623147 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224430+0000", 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93091173699999996 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224474+0000", 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3816000000000002e-05 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224508+0000", 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4369000000000002e-05 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2753 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.557079604, 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.188208317, 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2753 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061513+0000", 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.871e-06 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078405+0000", 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016891461 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078433+0000", 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8436e-05 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078447+0000", 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3607e-05 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137285+0000", 2024-09-19T05:44:43.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058837911 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137457+0000", 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017199399999999999 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137637+0000", 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180173 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.460405+0000", 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.322768426 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249610+0000", 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78920498100000003 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249664+0000", 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3668000000000002e-05 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249720+0000", 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5788999999999997e-05 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2769 3.11 3:8be9d197:::100000001f8.000006ce:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.51582711, 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1472854269999999, 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2769 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102764+0000", 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.102767+0000", 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.021e-06 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136259+0000", 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.033491981999999997 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136291+0000", 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1773999999999998e-05 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.136302+0000", 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1154e-05 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.239524+0000", 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103221854 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240715+0000", 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001191042 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.242825+0000", 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002110289 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534133+0000", 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29130808400000002 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249969+0000", 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71583560999999996 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.250018+0000", 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9459000000000002e-05 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.250049+0000", 2024-09-19T05:44:43.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1158000000000002e-05 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2773 3.9 3:9102af4b:::100000001f8.000006d2:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.472646642999999, 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124067851, 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2773 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145944+0000", 2024-09-19T05:44:43.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.145947+0000", 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.162e-06 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149477+0000", 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035300800000000001 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149524+0000", 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6264999999999998e-05 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.149539+0000", 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5169e-05 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234044+0000", 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.084504863999999999 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234273+0000", 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022875599999999999 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234509+0000", 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00023624799999999999 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534140+0000", 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29963108999999999 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269911+0000", 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73577157699999995 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269966+0000", 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4512000000000003e-05 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.270012+0000", 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6128000000000003e-05 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.688 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2796 3.9 3:9157a25d:::100000001f8.000006e9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.388890416000001, 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0747865430000001, 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2796 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229700+0000", 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.229704+0000", 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1860000000000001e-06 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234027+0000", 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043232710000000001 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234040+0000", 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3485999999999999e-05 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.234051+0000", 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1022999999999999e-05 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364098+0000", 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.130046151 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364248+0000", 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000150793 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.364444+0000", 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195735 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534144+0000", 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.16970036399999999 2024-09-19T05:44:43.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.304323+0000", 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77017807800000004 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.304400+0000", 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7881999999999997e-05 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.304487+0000", 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6574000000000002e-05 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2808 3.11 3:8bd5b656:::100000001f8.000006f5:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.378463341, 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.160580451, 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2808 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240131+0000", 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244285+0000", 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041546919999999998 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244303+0000", 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7646999999999999e-05 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244316+0000", 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2931999999999999e-05 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373296+0000", 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12897989000000001 2024-09-19T05:44:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373463+0000", 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166847 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373621+0000", 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015846400000000001 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.585995+0000", 2024-09-19T05:44:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.212374013 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400581+0000", 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81458587900000001 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400646+0000", 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5036999999999994e-05 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400708+0000", 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1937e-05 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2821 3.9 3:9040d340:::100000001f8.00000702:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.361461090000001, 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.06449839, 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2821 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257130+0000", 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.257133+0000", 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.146e-06 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266617+0000", 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0094844509999999996 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266644+0000", 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6942999999999999e-05 2024-09-19T05:44:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.266660+0000", 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5379e-05 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459066+0000", 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19240588 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459208+0000", 2024-09-19T05:44:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014251699999999999 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.459363+0000", 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000154535 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786586+0000", 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32722325099999999 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.321507+0000", 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53492102799999997 2024-09-19T05:44:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.321586+0000", 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8886000000000003e-05 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.321628+0000", 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2373999999999998e-05 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2858 3.9 3:92f6ecc1:::100000001f8.00000727:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.312263455, 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0285565109999999, 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2858 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306327+0000", 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.306329+0000", 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0719999999999998e-06 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311583+0000", 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052534000000000001 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311607+0000", 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3716000000000001e-05 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311625+0000", 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8681999999999998e-05 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.528241+0000", 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.216615688 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.528918+0000", 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067717000000000005 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.529096+0000", 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017772600000000001 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786593+0000", 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25749734800000001 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.334738+0000", 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54814497299999998 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.334814+0000", 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6044000000000006e-05 2024-09-19T05:44:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.334884+0000", 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9691999999999996e-05 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2862 3.9 3:907f8708:::100000001f8.0000072b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.306923179, 2024-09-19T05:44:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.023459962, 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2862 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311668+0000", 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311670+0000", 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316240+0000", 2024-09-19T05:44:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045697189999999999 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316271+0000", 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1341999999999999e-05 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.316324+0000", 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2984e-05 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534179+0000", 2024-09-19T05:44:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.21785516099999999 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534368+0000", 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018821699999999999 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.534552+0000", 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018435599999999999 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786599+0000", 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.25204730600000003 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335039+0000", 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54843970500000006 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335084+0000", 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5235999999999999e-05 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335128+0000", 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3479999999999997e-05 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2877 3.9 3:91dc4dd5:::100000001f8.0000073a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.286333743, 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0029870350000001, 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2877 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332257+0000", 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.332260+0000", 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9689999999999999e-06 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337127+0000", 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048665890000000002 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337155+0000", 2024-09-19T05:44:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8632e-05 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.337203+0000", 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7228000000000003e-05 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586056+0000", 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.24885332199999999 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586595+0000", 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053866000000000005 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.586768+0000", 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017356199999999999 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.786605+0000", 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19983721300000001 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335170+0000", 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54856438100000005 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335211+0000", 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1696000000000002e-05 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.335244+0000", 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2783000000000001e-05 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2895 3.9 3:939d0220:::100000001f8.0000074c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.264689898, 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0565064120000001, 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2895 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353901+0000", 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.353903+0000", 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356485+0000", 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025819390000000001 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356507+0000", 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2109000000000001e-05 2024-09-19T05:44:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.356521+0000", 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3560999999999999e-05 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.696659+0000", 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.34013843500000002 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.696845+0000", 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000185873 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.698377+0000", 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015312240000000001 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.805489+0000", 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10711206199999999 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.410285+0000", 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60479590599999999 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.410350+0000", 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5244000000000001e-05 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.410407+0000", 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7589e-05 2024-09-19T05:44:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2908 3.11 3:8a4f5bd7:::100000001f8.00000759:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.260147624, 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1380135730000001, 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2908 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358443+0000", 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.358445+0000", 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.08e-06 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362030+0000", 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0035843250000000002 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362050+0000", 2024-09-19T05:44:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9860999999999998e-05 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.362063+0000", 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3145999999999999e-05 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.715923+0000", 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35386083800000001 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.734420+0000", 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018496133000000001 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.734712+0000", 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029234500000000001 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852483+0000", 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11777127499999999 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.496362+0000", 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.643879058 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.496412+0000", 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9428999999999997e-05 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.496457+0000", 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5083000000000002e-05 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2921 3.11 3:88e2f4e7:::100000001f8.00000766:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.248249892, 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1412101940000001, 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2921 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370341+0000", 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.370344+0000", 2024-09-19T05:44:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373083+0000", 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027388270000000001 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373102+0000", 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9789e-05 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373116+0000", 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3456e-05 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778465+0000", 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.405348808 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778616+0000", 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015118400000000001 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.778791+0000", 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017549300000000001 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.852488+0000", 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073696686999999997 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.511418+0000", 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65892995200000004 2024-09-19T05:44:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.511495+0000", 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6765000000000001e-05 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.511551+0000", 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6366999999999998e-05 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:43.742 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:43.742 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:43.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:43.760 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:43.741+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:43.844 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:43.842+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:43.916 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5264 3.2b 3:d5fb1e74:::100000001f8.0000068d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.911920756, 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.91196740300000001, 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5264 2024-09-19T05:44:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000115+0000", 2024-09-19T05:44:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0650000000000001e-06 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013720+0000", 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013604842000000001 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013737+0000", 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7173999999999999e-05 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013745+0000", 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4850000000000003e-06 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013799+0000", 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4543000000000003e-05 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013869+0000", 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0032999999999998e-05 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013997+0000", 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012786399999999999 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.018990+0000", 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049925860000000002 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5286 3.2 3:41c49e27:::100000001f8.000006a3:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.89826902799999997, 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.89838745600000003, 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5286 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013767+0000", 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0320000000000002e-06 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020719+0000", 2024-09-19T05:44:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069518940000000001 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020732+0000", 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3560999999999999e-05 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020743+0000", 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0156000000000001e-05 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020811+0000", 2024-09-19T05:44:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8093e-05 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020888+0000", 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7484000000000006e-05 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.021010+0000", 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012149 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032253+0000", 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011243367000000001 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5291 3.2 3:433c69f2:::100000001f8.000006a8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.89126417099999999, 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.89142888399999998, 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5291 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020771+0000", 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5850000000000001e-06 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027794+0000", 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070227889999999998 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027809+0000", 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4959e-05 2024-09-19T05:44:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027817+0000", 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027886+0000", 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8356999999999996e-05 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027986+0000", 2024-09-19T05:44:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010082900000000001 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.028127+0000", 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000140829 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032805+0000", 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046781679999999999 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5529 3.1b 3:d99b0240:::100000001f8.00000796:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.602849+0000", 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.30918463000000002, 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.30941488099999997, 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5529 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.602849+0000", 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.602849+0000", 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.602852+0000", 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6900000000000001e-06 2024-09-19T05:44:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.610990+0000", 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0081379710000000008 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611000+0000", 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9760000000000007e-06 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611005+0000", 2024-09-19T05:44:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8319999999999996e-06 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611068+0000", 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2953999999999994e-05 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611185+0000", 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011679 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611356+0000", 2024-09-19T05:44:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017188500000000001 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.615791+0000", 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044343409999999996 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5723 3.2 3:4214d263:::100000001f8.00000858:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.14499305000000001, 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.14528270300000001, 2024-09-19T05:44:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5723 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767043+0000", 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.041e-06 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782398+0000", 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015355056000000001 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782422+0000", 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4190999999999999e-05 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782441+0000", 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782517+0000", 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6186000000000005e-05 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782591+0000", 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3604999999999997e-05 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782725+0000", 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013430699999999999 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.786761+0000", 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035673 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5530 3.2 3:43043671:::100000001f8.00000797:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.30101624300000002, 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.301395729, 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5530 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611019+0000", 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2619999999999999e-06 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618028+0000", 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070088199999999998 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618048+0000", 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0616e-05 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618061+0000", 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2678e-05 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618127+0000", 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6276e-05 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618227+0000", 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9356000000000002e-05 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618379+0000", 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000152151 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.623797+0000", 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054177050000000001 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5403 3.2b 3:d515b219:::100000001f8.00000718:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.57692686900000001, 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57739879500000002, 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5403 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335110+0000", 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.27e-06 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351404+0000", 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016293426 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351429+0000", 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5131999999999999e-05 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351442+0000", 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3281e-05 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351543+0000", 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100887 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351640+0000", 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6569000000000003e-05 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351796+0000", 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015682900000000001 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.358222+0000", 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064259030000000002 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5536 3.1b 3:dba4d56b:::100000001f8.0000079d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.618092+0000", 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.29394171600000002, 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.29448445099999998, 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5536 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618092+0000", 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618092+0000", 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618094+0000", 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1670000000000002e-06 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.625258+0000", 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071637070000000001 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.625277+0000", 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9105000000000001e-05 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.625288+0000", 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0284e-05 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.625359+0000", 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1894e-05 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.625489+0000", 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000129667 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.625594+0000", 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105235 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.629835+0000", 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042406600000000003 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5405 3.2 3:41109be1:::100000001f8.0000071a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.56055853200000005, 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56116536800000005, 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5405 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351477+0000", 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.846e-06 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367351+0000", 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015873627000000001 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367458+0000", 2024-09-19T05:44:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106702 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367496+0000", 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8389000000000001e-05 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367630+0000", 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013432400000000001 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367737+0000", 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010691700000000001 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367859+0000", 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012197999999999999 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.373387+0000", 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055282539999999998 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5752 3.2b 3:d5611f28:::100000001f8.00000875:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.790918+0000", 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.121115904, 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.121763897, 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5752 2024-09-19T05:44:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.790918+0000", 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.790918+0000", 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.790926+0000", 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0430000000000008e-06 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800280+0000", 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093541479999999996 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800296+0000", 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5981000000000002e-05 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800305+0000", 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.251e-06 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800371+0000", 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5335999999999999e-05 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800470+0000", 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9289000000000004e-05 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800619+0000", 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014881200000000001 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.812656+0000", 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012037176 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5753 3.2b 3:d4286d14:::100000001f8.00000876:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.800327+0000", 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.11170716, 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.112419959, 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5753 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800327+0000", 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800327+0000", 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.800336+0000", 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7299999999999994e-06 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.809227+0000", 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088910199999999995 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.809248+0000", 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1132e-05 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.809259+0000", 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1146e-05 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.809321+0000", 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2528999999999997e-05 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.809394+0000", 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2638000000000005e-05 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.809505+0000", 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111192 2024-09-19T05:44:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.812945+0000", 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0034395720000000001 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5759 3.2 3:42bacaf4:::100000001f8.0000087c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.090157878999999996, 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.090936510999999998, 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5759 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:44:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821879+0000", 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829742+0000", 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078630269999999999 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829755+0000", 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3308e-05 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829763+0000", 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1839999999999999e-06 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829849+0000", 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6017e-05 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829955+0000", 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106059 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.830113+0000", 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015798799999999999 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834572+0000", 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004458755 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5760 3.19 3:9ad30d00:::100000001f8.0000087d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.829791+0000", 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.082242896999999995, 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.083068046000000006, 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5760 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829791+0000", 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829791+0000", 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829793+0000", 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9240000000000001e-06 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.836750+0000", 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069570930000000001 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.836769+0000", 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8819000000000001e-05 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.836778+0000", 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5899999999999997e-06 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.836859+0000", 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0530000000000003e-05 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.836983+0000", 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012440899999999999 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.837157+0000", 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017382699999999999 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.843875+0000", 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0067176459999999999 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5170 3.2 3:40ee304f:::100000001f8.0000062f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.06342833, 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.064323269, 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5170 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848608+0000", 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6699999999999998e-06 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861615+0000", 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013006636 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861637+0000", 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2447e-05 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861651+0000", 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3431000000000001e-05 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861732+0000", 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1124000000000005e-05 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861822+0000", 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9981000000000005e-05 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861969+0000", 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014714700000000001 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.868505+0000", 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065363340000000004 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5617 3.1b 3:da4a88b9:::100000001f8.000007ee:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.984 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.678582+0000", 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.23345175100000001, 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.234446879, 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5617 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.678582+0000", 2024-09-19T05:44:43.985 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.678582+0000", 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.678585+0000", 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3350000000000001e-06 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.986 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685628+0000", 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070438849999999997 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685657+0000", 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8328999999999999e-05 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685668+0000", 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1583000000000001e-05 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.987 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685749+0000", 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0797999999999994e-05 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685825+0000", 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5995999999999995e-05 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685947+0000", 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121478 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.988 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.690797+0000", 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048505229999999998 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5776 3.2 3:42907a88:::100000001f8.0000088d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.053424462999999998, 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.054513697999999999, 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.989 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5776 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.990 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858612+0000", 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866396+0000", 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077837369999999998 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.991 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866413+0000", 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7481000000000001e-05 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866423+0000", 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9799999999999993e-06 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866567+0000", 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000143683 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.992 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866677+0000", 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110042 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866889+0000", 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021198500000000001 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873285+0000", 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063963800000000001 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.993 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5785 3.2b 3:d4838122:::100000001f8.00000896:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.866487+0000", 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.045547397000000003, 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.046729018999999997, 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.994 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5785 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866487+0000", 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866487+0000", 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.995 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866489+0000", 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4549999999999998e-06 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873494+0000", 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070045819999999996 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873507+0000", 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3222e-05 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.996 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873514+0000", 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4610000000000002e-06 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873588+0000", 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3925999999999997e-05 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873698+0000", 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010953199999999999 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.997 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873854+0000", 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000156494 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.879576+0000", 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057217630000000004 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5475 3.1b 3:d80cab9c:::100000001f8.00000760:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:43.998 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.462647+0000", 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.44938677700000002, 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.45065417899999999, 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5475 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:43.999 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.462647+0000", 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.462647+0000", 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.462650+0000", 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.723e-06 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.000 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482022+0000", 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019372397 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482040+0000", 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7759000000000001e-05 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482047+0000", 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1729999999999998e-06 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.001 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482101+0000", 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3304000000000003e-05 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482170+0000", 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9680999999999999e-05 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482289+0000", 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011883999999999999 2024-09-19T05:44:44.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.487694+0000", 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054046019999999997 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5786 3.19 3:9bb5c4e2:::100000001f8.00000897:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.873547+0000", 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.038486925999999998, 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.039822679999999999, 2024-09-19T05:44:44.003 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5786 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873547+0000", 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.004 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873547+0000", 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873549+0000", 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.238e-06 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880475+0000", 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069257440000000002 2024-09-19T05:44:44.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880490+0000", 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5345e-05 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880499+0000", 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7679999999999997e-06 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880567+0000", 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8175000000000004e-05 2024-09-19T05:44:44.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880676+0000", 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108361 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880838+0000", 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000162144 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.886437+0000", 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055989280000000004 2024-09-19T05:44:44.007 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5360 3.2b 3:d6b2cae8:::100000001f8.000006ed:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.75486777500000002, 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.75624545300000001, 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.008 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5360 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.009 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157169+0000", 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089999999999999e-06 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173455+0000", 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016286248 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.010 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173477+0000", 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1716999999999999e-05 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173488+0000", 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1620000000000001e-05 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173563+0000", 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4161999999999999e-05 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.011 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173653+0000", 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9974000000000003e-05 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173776+0000", 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012363000000000001 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.179905+0000", 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061287110000000002 2024-09-19T05:44:44.012 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5671 3.2b 3:d7657d1c:::100000001f8.00000824:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.20527636199999999, 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.20669454700000001, 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.013 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5671 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.014 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706760+0000", 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.243e-06 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715117+0000", 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083569019999999994 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715133+0000", 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5818e-05 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.015 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715140+0000", 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1849999999999998e-06 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715202+0000", 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2315000000000002e-05 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715304+0000", 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101786 2024-09-19T05:44:44.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715421+0000", 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117333 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.720417+0000", 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049962840000000001 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.017 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5791 3.19 3:99010fc7:::100000001f8.0000089c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.880522+0000", 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.031511560000000001, 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.032969547000000002, 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5791 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.018 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880522+0000", 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880522+0000", 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.880525+0000", 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.137e-06 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.019 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887698+0000", 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071738619999999996 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887720+0000", 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1152e-05 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887729+0000", 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7089999999999997e-06 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.020 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887797+0000", 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7447999999999995e-05 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887906+0000", 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000109529 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.888036+0000", 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012943300000000001 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.021 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.892437+0000", 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044009619999999996 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5363 3.2 3:41c08734:::100000001f8.000006f0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.73851721999999997, 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.74001583199999998, 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.022 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5363 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:44.023 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173519+0000", 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3089999999999998e-06 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186577+0000", 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013058449999999999 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.024 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186607+0000", 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9872000000000001e-05 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186618+0000", 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0772e-05 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186771+0000", 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000153024 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.025 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186856+0000", 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4907999999999994e-05 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186976+0000", 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011951 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.193253+0000", 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062774969999999999 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.026 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5478 3.2b 3:d663163e:::100000001f8.00000763:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.41814881100000001, 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.419687431, 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5478 2024-09-19T05:44:44.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.028 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493886+0000", 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2529999999999999e-06 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506674+0000", 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012787552000000001 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506696+0000", 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2259000000000002e-05 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.029 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506712+0000", 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5965e-05 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506793+0000", 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1075000000000006e-05 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506944+0000", 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151065 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.030 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.507151+0000", 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020636600000000001 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.515901+0000", 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087503040000000004 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5191 3.2 3:41cf9845:::100000001f8.00000644:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.031 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.99990937800000002, 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.001495239, 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5191 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:44.032 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912127+0000", 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9400000000000001e-06 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.033 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925516+0000", 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013389509000000001 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925536+0000", 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9579e-05 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925544+0000", 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3450000000000006e-06 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.034 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925643+0000", 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9018000000000006e-05 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925711+0000", 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8351999999999999e-05 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925816+0000", 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104832 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.035 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.931570+0000", 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057541279999999998 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5491 3.2 3:418769c2:::100000001f8.00000770:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.40529043199999998, 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.406919959, 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.036 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5491 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506746+0000", 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8059999999999999e-06 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.520997+0000", 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014250966 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521021+0000", 2024-09-19T05:44:44.038 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.368e-05 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521038+0000", 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6708e-05 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521168+0000", 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013036300000000001 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521266+0000", 2024-09-19T05:44:44.039 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7849000000000005e-05 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521457+0000", 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000191385 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.528522+0000", 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070645830000000001 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.040 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5795 3.1b 3:d849a4fe:::100000001f8.000008a0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.887753+0000", 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.024280962999999999, 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.025952119999999999, 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5795 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.041 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887753+0000", 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887753+0000", 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.887756+0000", 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0469999999999998e-06 2024-09-19T05:44:44.042 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.894778+0000", 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070219089999999998 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.894797+0000", 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8873999999999999e-05 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.894807+0000", 2024-09-19T05:44:44.043 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9259999999999995e-06 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.894894+0000", 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7687000000000003e-05 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.895018+0000", 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012388699999999999 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.044 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.895208+0000", 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018949099999999999 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.899610+0000", 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044023380000000004 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5497 3.1b 3:d95cf893:::100000001f8.00000776:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.521081+0000", 2024-09-19T05:44:44.045 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.39095251600000003, 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.39266375399999998, 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5497 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521081+0000", 2024-09-19T05:44:44.046 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521081+0000", 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521084+0000", 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3250000000000002e-06 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.047 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539011+0000", 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017927021000000001 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539027+0000", 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6405e-05 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539036+0000", 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4630000000000001e-06 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.048 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539112+0000", 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6197000000000002e-05 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539236+0000", 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000124067 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539431+0000", 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000194985 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.049 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.548708+0000", 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0092772830000000008 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5682 3.2 3:43bb8046:::100000001f8.0000082f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.19002454299999999, 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.19177760099999999, 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.050 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5682 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.051 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722012+0000", 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1849999999999998e-06 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729368+0000", 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073561759999999999 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.052 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729391+0000", 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3309e-05 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729404+0000", 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3016e-05 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729479+0000", 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4914999999999997e-05 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.053 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729591+0000", 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111607 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729717+0000", 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012588200000000001 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.735604+0000", 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058875100000000003 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.054 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5796 3.2b 3:d61b6bf4:::100000001f8.000008a1:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.894836+0000", 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.017197679, 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018992176, 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.055 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5796 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.894836+0000", 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.894836+0000", 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.056 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.894839+0000", 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8389999999999998e-06 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.902667+0000", 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078274020000000007 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.902690+0000", 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.304e-05 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.057 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.902704+0000", 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4083999999999999e-05 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.902778+0000", 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4505000000000006e-05 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.902878+0000", 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9495999999999997e-05 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.058 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.903036+0000", 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000158083 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.909490+0000", 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064541119999999997 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5383 3.2 3:42776a0c:::100000001f8.00000704:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.059 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.69884181899999998, 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.70068014199999995, 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5383 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.060 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213194+0000", 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000002e-06 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230888+0000", 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017693296000000001 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230908+0000", 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0624000000000001e-05 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230918+0000", 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5119999999999997e-06 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231017+0000", 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9586000000000005e-05 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231189+0000", 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172053 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231464+0000", 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274916 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.238533+0000", 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007068469 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5390 3.2 3:4215b5dd:::100000001f8.0000070b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.68106055300000001, 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.68294066600000003, 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.064 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5390 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:44:44.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230976+0000", 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.419e-06 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244642+0000", 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013666417 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244663+0000", 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0454000000000001e-05 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244690+0000", 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6962e-05 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244765+0000", 2024-09-19T05:44:44.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4884999999999999e-05 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244873+0000", 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108904 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.245011+0000", 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013788200000000001 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.251134+0000", 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006122677 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5506 3.2b 3:d506a813:::100000001f8.0000077f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.37296821099999999, 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.37489742300000001, 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.069 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5506 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:44:44.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539068+0000", 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0710000000000002e-06 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555200+0000", 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016132480000000001 2024-09-19T05:44:44.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555230+0000", 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9728999999999999e-05 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555243+0000", 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2697999999999999e-05 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555334+0000", 2024-09-19T05:44:44.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1113999999999995e-05 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555436+0000", 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010182 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.556369+0000", 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00093375000000000001 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.562703+0000", 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063334139999999999 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 33 2024-09-19T05:44:44.074 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.053+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f7a0c0cbdd0 con 0x7f7a0c0c9b00 2024-09-19T05:44:44.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:44.076 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:44.074+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.077+0000 7f6245798640 1 Processor -- start 2024-09-19T05:44:44.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.079+0000 7f6245798640 1 -- start start 2024-09-19T05:44:44.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.081+0000 7f6245798640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62400c73d0 0x7f62401514c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.081+0000 7f6245798640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62400c4cc0 con 0x7f62400c7df0 2024-09-19T05:44:44.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.081+0000 7f6245798640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62400c85e0 con 0x7f6240151a00 2024-09-19T05:44:44.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.081+0000 7f6245798640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f62400c81d0 con 0x7f62400c73d0 2024-09-19T05:44:44.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.083+0000 7f6237fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62400c73d0 0x7f62401514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.083+0000 7f6237fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62400c73d0 0x7f62401514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:57018/0 (socket says 172.21.15.2:57018) 2024-09-19T05:44:44.085 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.083+0000 7f6237fff640 1 -- 172.21.15.2:0/1066733465 learned_addr learned my addr 172.21.15.2:0/1066733465 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:44.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.086+0000 7f623e7fc640 1 -- 172.21.15.2:0/1066733465 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2351544232 0 0) 0x7f62400c4cc0 con 0x7f62400c7df0 2024-09-19T05:44:44.089 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.086+0000 7f623e7fc640 1 -- 172.21.15.2:0/1066733465 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f622c003520 con 0x7f62400c7df0 2024-09-19T05:44:44.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.088+0000 7f6237fff640 1 -- 172.21.15.2:0/1066733465 >> v1:172.21.15.123:6789/0 conn(0x7f6240151a00 legacy=0x7f62400706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:44.091 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.088+0000 7f6237fff640 1 -- 172.21.15.2:0/1066733465 >> v1:172.21.15.2:6789/0 conn(0x7f62400c7df0 legacy=0x7f62400c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:44.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.088+0000 7f6237fff640 1 -- 172.21.15.2:0/1066733465 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62400c88c0 con 0x7f62400c73d0 2024-09-19T05:44:44.096 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.093+0000 7f6237fff640 1 --2- 172.21.15.2:0/1066733465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62400c73d0 0x7f62401514c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62280098a0 tx=0x7f622802e7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d64409fec28b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.095+0000 7f623e7fc640 1 -- 172.21.15.2:0/1066733465 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622803a070 con 0x7f62400c73d0 2024-09-19T05:44:44.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.095+0000 7f623e7fc640 1 -- 172.21.15.2:0/1066733465 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6228033710 con 0x7f62400c73d0 2024-09-19T05:44:44.098 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.095+0000 7f623e7fc640 1 -- 172.21.15.2:0/1066733465 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622803a070 con 0x7f62400c73d0 2024-09-19T05:44:44.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.099+0000 7f6245798640 1 -- 172.21.15.2:0/1066733465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62400c73d0 msgr2=0x7f62401514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:44.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.099+0000 7f6245798640 1 --2- 172.21.15.2:0/1066733465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62400c73d0 0x7f62401514c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62280098a0 tx=0x7f622802e7a0 comp rx=0 tx=0).stop 2024-09-19T05:44:44.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.101+0000 7f6245798640 1 -- 172.21.15.2:0/1066733465 shutdown_connections 2024-09-19T05:44:44.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.101+0000 7f6245798640 1 --2- 172.21.15.2:0/1066733465 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f62400c73d0 0x7f62401514c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:44.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.101+0000 7f6245798640 1 -- 172.21.15.2:0/1066733465 >> 172.21.15.2:0/1066733465 conn(0x7f6240072ad0 msgr2=0x7f6240072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:44.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.103+0000 7f6245798640 1 -- 172.21.15.2:0/1066733465 shutdown_connections 2024-09-19T05:44:44.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.106+0000 7f6245798640 1 -- 172.21.15.2:0/1066733465 wait complete. 2024-09-19T05:44:44.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.108+0000 7f6245798640 1 Processor -- start 2024-09-19T05:44:44.113 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.111+0000 7f6245798640 1 -- start start 2024-09-19T05:44:44.115 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.113+0000 7f6245798640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62400c73d0 0x7f624006ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.116 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.114+0000 7f6245798640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62400c7df0 0x7f62402007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.114+0000 7f623effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62400c73d0 0x7f624006ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.114+0000 7f623effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62400c73d0 0x7f624006ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:38678/0 (socket says 172.21.15.2:38678) 2024-09-19T05:44:44.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.114+0000 7f6237fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62400c7df0 0x7f62402007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.115+0000 7f6237fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62400c7df0 0x7f62402007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60214/0 (socket says 172.21.15.2:60214) 2024-09-19T05:44:44.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.115+0000 7f623effd640 1 -- 172.21.15.2:0/1539963323 learned_addr learned my addr 172.21.15.2:0/1539963323 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:44.117 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.115+0000 7f6245798640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6240151a00 0x7f6240200d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.115+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f62400c8e80 con 0x7f62400c7df0 2024-09-19T05:44:44.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.115+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6240151e90 con 0x7f62400c73d0 2024-09-19T05:44:44.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.115+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f62400c4fa0 con 0x7f6240151a00 2024-09-19T05:44:44.118 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.115+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6240151a00 0x7f6240200d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.116+0000 7f6237fff640 1 -- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6240151a00 msgr2=0x7f6240200d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:44.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.116+0000 7f6237fff640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6240151a00 0x7f6240200d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:44.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.116+0000 7f6237fff640 1 -- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62400c73d0 msgr2=0x7f624006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:44.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.116+0000 7f6237fff640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62400c73d0 0x7f624006ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:44.119 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.116+0000 7f6237fff640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624014b3c0 con 0x7f62400c7df0 2024-09-19T05:44:44.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.124+0000 7f6237fff640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62400c7df0 0x7f62402007f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6228035000 tx=0x7f62280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.124+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6240151a00 0x7f6240200d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:44.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.124+0000 7f623effd640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62400c73d0 0x7f624006ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:44.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.124+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228045070 con 0x7f62400c7df0 2024-09-19T05:44:44.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.125+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6228010340 con 0x7f62400c7df0 2024-09-19T05:44:44.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.125+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6240148660 con 0x7f62400c7df0 2024-09-19T05:44:44.133 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.131+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624014c1f0 con 0x7f62400c7df0 2024-09-19T05:44:44.136 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.134+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622803a070 con 0x7f62400c7df0 2024-09-19T05:44:44.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.136+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f62402004b0 con 0x7f62400c7df0 2024-09-19T05:44:44.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.137+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6228048070 con 0x7f62400c7df0 2024-09-19T05:44:44.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.137+0000 7f62377fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f622006c3b0 0x7f622006e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.140 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.138+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5799+0+0 (secure 0 0 0) 0x7f62280b0170 con 0x7f62400c7df0 2024-09-19T05:44:44.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.141+0000 7f623effd640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f622006c3b0 0x7f622006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.141+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v74) ==== 54+0+16 (secure 0 0 0) 0x7f622807a330 con 0x7f62400c7df0 2024-09-19T05:44:44.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.145+0000 7f6245798640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6218001650 0x7f6218003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.145+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6218006bb0 con 0x7f6218001650 2024-09-19T05:44:44.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.145+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6218001650 0x7f6218003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.145+0000 7f623effd640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f622006c3b0 0x7f622006e870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f62300011e0 tx=0x7f6230009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.161 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.159+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f6218001650 0x7f6218003b10 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.176 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:44.175+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:44.229 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.227+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6218006bb0 con 0x7f6218001650 2024-09-19T05:44:44.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.277+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6218005c60 con 0x7f6218001650 2024-09-19T05:44:44.287 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:44.276+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:44.289 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:44.289 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:44.289 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:44.289 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:44.289 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:44.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:44.383 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:44.379+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2915 3.1b 3:d80cab9c:::100000001f8.00000760:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.030674+0000", 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.432224439000001, 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.53960979899999995, 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.466 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2915 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030674+0000", 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.467 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030674+0000", 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.030677+0000", 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7650000000000002e-06 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.040963+0000", 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010285608 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.468 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041031+0000", 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8497999999999994e-05 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041049+0000", 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7691000000000001e-05 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041216+0000", 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000167529 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.469 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041348+0000", 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013218600000000001 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.041483+0000", 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013436500000000001 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.046423+0000", 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049396969999999998 2024-09-19T05:44:44.470 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.570173+0000", 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52375065499999995 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.570234+0000", 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0613999999999999e-05 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.570284+0000", 2024-09-19T05:44:44.471 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0191000000000001e-05 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2937 3.1b 3:d95cf893:::100000001f8.00000776:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.072238+0000", 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.390660349000001, 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.62993552500000005, 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.472 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2937 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072238+0000", 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.473 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072238+0000", 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.072240+0000", 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1950000000000002e-06 2024-09-19T05:44:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081623+0000", 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093823429999999996 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081636+0000", 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3338e-05 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081649+0000", 2024-09-19T05:44:44.475 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2711000000000001e-05 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081727+0000", 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8064000000000003e-05 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.081875+0000", 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000148283 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.082099+0000", 2024-09-19T05:44:44.476 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00022402300000000001 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.087700+0000", 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056004999999999996 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702066+0000", 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.61436600399999997 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.477 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702125+0000", 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9320000000000001e-05 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702174+0000", 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8744e-05 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2969 3.1b 3:d99b0240:::100000001f8.00000796:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.123365+0000", 2024-09-19T05:44:44.478 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.339533616000001, 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.608564514, 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2969 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.123365+0000", 2024-09-19T05:44:44.479 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.123365+0000", 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.123368+0000", 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.57e-06 2024-09-19T05:44:44.480 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134313+0000", 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010945217 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134336+0000", 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3365000000000001e-05 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134345+0000", 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8599999999999999e-06 2024-09-19T05:44:44.481 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134425+0000", 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0414000000000006e-05 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134597+0000", 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017141699999999999 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.482 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.134749+0000", 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015180599999999999 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.139528+0000", 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047798290000000002 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731839+0000", 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.59231012500000002 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.483 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731890+0000", 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1752000000000002e-05 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731930+0000", 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9159000000000002e-05 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2976 3.1b 3:dba4d56b:::100000001f8.0000079d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.144911+0000", 2024-09-19T05:44:44.484 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.317987667000001, 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.58715358200000001, 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2976 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.144911+0000", 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.485 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.144911+0000", 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.144913+0000", 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.584e-06 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163241+0000", 2024-09-19T05:44:44.486 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018328420000000002 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163276+0000", 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5377999999999999e-05 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163394+0000", 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117496 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163494+0000", 2024-09-19T05:44:44.487 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010056300000000001 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163643+0000", 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014862400000000001 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.163749+0000", 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105504 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.488 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169203+0000", 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054548019999999999 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.731983+0000", 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56277920999999997 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732022+0000", 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9869000000000001e-05 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.489 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732065+0000", 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2132000000000003e-05 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3057 3.1b 3:da4a88b9:::100000001f8.000007ee:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.225809+0000", 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 12.237089903999999, 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.50634741400000005, 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3057 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.225809+0000", 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.225809+0000", 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.491 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.225811+0000", 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.492e-06 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239326+0000", 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013515070000000001 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239351+0000", 2024-09-19T05:44:44.492 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5068000000000001e-05 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239364+0000", 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2916e-05 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239489+0000", 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012427799999999999 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.493 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239708+0000", 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021967699999999999 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.239965+0000", 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000256576 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.245413+0000", 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054483509999999997 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.494 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732093+0000", 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48667946600000001 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732128+0000", 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5404000000000002e-05 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.732156+0000", 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8116e-05 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.495 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3235 3.1b 3:d849a4fe:::100000001f8.000008a0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.512061+0000", 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.950837836, 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.59435483, 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3235 2024-09-19T05:44:44.496 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.512061+0000", 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.512061+0000", 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.512066+0000", 2024-09-19T05:44:44.497 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0830000000000003e-06 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522551+0000", 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010484745 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522563+0000", 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2646999999999999e-05 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.498 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522570+0000", 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6379999999999998e-06 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522649+0000", 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9091000000000006e-05 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522752+0000", 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000102589 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.499 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.522858+0000", 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010684299999999999 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.529143+0000", 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062849569999999999 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.106342+0000", 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.57719904799999999 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.500 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.106383+0000", 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0482999999999998e-05 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.106416+0000", 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2706000000000002e-05 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3352 3.1b 3:d821d6cb:::100000001f8.00000915:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:44.501 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.689256+0000", 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.773642425, 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.47540637000000002, 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3352 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.502 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.689256+0000", 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.689256+0000", 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.689259+0000", 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7030000000000002e-06 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.503 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696787+0000", 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075276129999999998 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696819+0000", 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2326000000000002e-05 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696833+0000", 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3791999999999999e-05 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.504 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696904+0000", 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0955000000000001e-05 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.697059+0000", 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000155797 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.697237+0000", 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017717500000000001 2024-09-19T05:44:44.505 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702015+0000", 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047785739999999998 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164557+0000", 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46254178200000001 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164616+0000", 2024-09-19T05:44:44.506 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8789999999999998e-05 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164663+0000", 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6863000000000002e-05 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3359 3.1b 3:d9005dad:::100000001f8.0000091c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.696863+0000", 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.766035841000001, 2024-09-19T05:44:44.507 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46792231000000001, 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3359 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696863+0000", 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.508 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696863+0000", 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.696865+0000", 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6400000000000001e-06 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702619+0000", 2024-09-19T05:44:44.509 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057532829999999997 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702631+0000", 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2748000000000001e-05 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702640+0000", 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3699999999999995e-06 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.510 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702702+0000", 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1847000000000006e-05 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702822+0000", 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012001 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702935+0000", 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113186 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.511 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.707872+0000", 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049367109999999999 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164710+0000", 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.45683812400000001 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.512 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164745+0000", 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.489e-05 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164785+0000", 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0500999999999997e-05 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3383 3.1b 3:dacb2766:::100000001f8.00000934:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:44.513 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.702665+0000", 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 11.760233331, 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46223359400000003, 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3383 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702665+0000", 2024-09-19T05:44:44.514 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702665+0000", 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.702667+0000", 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6610000000000001e-06 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.515 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709793+0000", 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071257580000000003 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709812+0000", 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8824000000000001e-05 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709820+0000", 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6770000000000001e-06 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.516 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.709904+0000", 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3947000000000001e-05 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.710079+0000", 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000175274 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.710249+0000", 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000169877 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.517 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.714908+0000", 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046582999999999998 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164834+0000", 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.44992615899999999 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164867+0000", 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3368999999999998e-05 2024-09-19T05:44:44.518 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.164899+0000", 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1748000000000002e-05 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5170 3.2 3:40ee304f:::100000001f8.0000062f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.6142929860000002, 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.07064564, 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.519 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5170 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.520 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848608+0000", 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6699999999999998e-06 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861615+0000", 2024-09-19T05:44:44.521 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013006636 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861637+0000", 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2447e-05 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861651+0000", 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3431000000000001e-05 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861732+0000", 2024-09-19T05:44:44.522 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1124000000000005e-05 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861822+0000", 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9981000000000005e-05 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861969+0000", 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014714700000000001 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.523 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.868505+0000", 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065363340000000004 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919134+0000", 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.050628731 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919203+0000", 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9207000000000003e-05 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.524 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919251+0000", 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7932000000000001e-05 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5191 3.2 3:41cf9845:::100000001f8.00000644:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.550774034, 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1864950409999999, 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.525 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5191 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.526 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912127+0000", 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9400000000000001e-06 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925516+0000", 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013389509000000001 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925536+0000", 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9579e-05 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925544+0000", 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3450000000000006e-06 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925643+0000", 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9018000000000006e-05 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.528 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925711+0000", 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8351999999999999e-05 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925816+0000", 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104832 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.931570+0000", 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057541279999999998 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.529 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098516+0000", 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1669453750000001 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098577+0000", 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1736999999999995e-05 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098620+0000", 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2225999999999999e-05 2024-09-19T05:44:44.530 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5264 3.2b 3:d5fb1e74:::100000001f8.0000068d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.4627854120000001, 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.098400515, 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.531 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5264 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.532 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000115+0000", 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0650000000000001e-06 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013720+0000", 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013604842000000001 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013737+0000", 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7173999999999999e-05 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.533 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013745+0000", 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4850000000000003e-06 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013799+0000", 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4543000000000003e-05 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013869+0000", 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0032999999999998e-05 2024-09-19T05:44:44.534 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013997+0000", 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012786399999999999 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.018990+0000", 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049925860000000002 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098402+0000", 2024-09-19T05:44:44.535 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0794119390000001 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098460+0000", 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8700999999999999e-05 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098514+0000", 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3282999999999998e-05 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.536 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5286 3.2 3:41c49e27:::100000001f8.000006a3:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.449133684, 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.368367997, 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5286 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.537 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.538 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013767+0000", 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0320000000000002e-06 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020719+0000", 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069518940000000001 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020732+0000", 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3560999999999999e-05 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.539 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020743+0000", 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0156000000000001e-05 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020811+0000", 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8093e-05 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020888+0000", 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7484000000000006e-05 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.540 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.021010+0000", 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012149 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032253+0000", 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011243367000000001 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382067+0000", 2024-09-19T05:44:44.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3498136039999999 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382105+0000", 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8525000000000001e-05 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382133+0000", 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7790999999999999e-05 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.542 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5291 3.2 3:433c69f2:::100000001f8.000006a8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.4421288269999999, 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.361485925, 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.543 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5291 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020771+0000", 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5850000000000001e-06 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027794+0000", 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070227889999999998 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027809+0000", 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4959e-05 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027817+0000", 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027886+0000", 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8356999999999996e-05 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027986+0000", 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010082900000000001 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.028127+0000", 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000140829 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032805+0000", 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046781679999999999 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382178+0000", 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3493729270000001 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382219+0000", 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0924999999999999e-05 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382256+0000", 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6430999999999997e-05 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5360 3.2b 3:d6b2cae8:::100000001f8.000006ed:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.305732431, 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1641596679999999, 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5360 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.549 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157169+0000", 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089999999999999e-06 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173455+0000", 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016286248 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173477+0000", 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1716999999999999e-05 2024-09-19T05:44:44.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173488+0000", 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1620000000000001e-05 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173563+0000", 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4161999999999999e-05 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173653+0000", 2024-09-19T05:44:44.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9974000000000003e-05 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173776+0000", 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012363000000000001 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.179905+0000", 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061287110000000002 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321250+0000", 2024-09-19T05:44:44.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1413448260000001 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321287+0000", 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6937999999999998e-05 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321326+0000", 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9233000000000002e-05 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5363 3.2 3:41c08734:::100000001f8.000006f0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.289381876, 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2088135710000001, 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5363 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173519+0000", 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3089999999999998e-06 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186577+0000", 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013058449999999999 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186607+0000", 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9872000000000001e-05 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186618+0000", 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0772e-05 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186771+0000", 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000153024 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186856+0000", 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4907999999999994e-05 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186976+0000", 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011951 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.193253+0000", 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062774969999999999 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382286+0000", 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1890333310000001 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382310+0000", 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4083e-05 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382330+0000", 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9814999999999999e-05 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5403 3.2b 3:d515b219:::100000001f8.00000718:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.1277915249999999, 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.99261407000000001, 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5403 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335110+0000", 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.27e-06 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351404+0000", 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016293426 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351429+0000", 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5131999999999999e-05 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351442+0000", 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3281e-05 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351543+0000", 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100887 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351640+0000", 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6569000000000003e-05 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351796+0000", 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015682900000000001 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.358222+0000", 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064259030000000002 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327636+0000", 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.96941361299999995 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327678+0000", 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1897000000000002e-05 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327721+0000", 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3263000000000002e-05 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.566 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_repop(client.4534.0:5171 3.3c e74/20 3:3cb3fdcc:::100000001f8.00000630:head v 74'104, mlcod=74'101)", 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.405862+0000", 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.0570365159999999, 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.52611487199999996, 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.405862+0000", 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.405862+0000", 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.405863+0000", 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4999999999999998e-07 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.918966+0000", 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.51310295299999997 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.918979+0000", 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3757999999999999e-05 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.918988+0000", 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.789e-06 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919102+0000", 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113942 2024-09-19T05:44:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919126+0000", 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4388000000000001e-05 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.931878+0000", 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012751734000000001 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.931977+0000", 2024-09-19T05:44:44.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8857999999999994e-05 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5475 3.1b 3:d80cab9c:::100000001f8.00000760:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.462647+0000", 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.0002514330000001, 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.46208049099999998, 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5475 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.462647+0000", 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.462647+0000", 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.462650+0000", 2024-09-19T05:44:44.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.723e-06 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482022+0000", 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.019372397 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482040+0000", 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7759000000000001e-05 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482047+0000", 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1729999999999998e-06 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482101+0000", 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3304000000000003e-05 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482170+0000", 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9680999999999999e-05 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.482289+0000", 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011883999999999999 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.487694+0000", 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054046019999999997 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.924648+0000", 2024-09-19T05:44:44.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.43695451600000002 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.924693+0000", 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4437999999999997e-05 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.924728+0000", 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5058000000000002e-05 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5617 3.1b 3:da4a88b9:::100000001f8.000007ee:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.678582+0000", 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.78431640700000005, 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.48429988000000002, 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5617 2024-09-19T05:44:44.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.678582+0000", 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.678582+0000", 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.678585+0000", 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3350000000000001e-06 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685628+0000", 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070438849999999997 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685657+0000", 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8328999999999999e-05 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685668+0000", 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1583000000000001e-05 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685749+0000", 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0797999999999994e-05 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685825+0000", 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5995999999999995e-05 2024-09-19T05:44:44.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.685947+0000", 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121478 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.690797+0000", 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048505229999999998 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.162784+0000", 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.47198683899999999 2024-09-19T05:44:44.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.162823+0000", 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9490999999999998e-05 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.162882+0000", 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8622999999999998e-05 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.586 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:44.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.546+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6218005c60 con 0x7f6218001650 2024-09-19T05:44:44.588 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:44.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:44.590 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:44.586+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.547+0000 7f62357fa640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f621c001650 0x7f621c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.547+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f621c006bb0 con 0x7f621c001650 2024-09-19T05:44:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.547+0000 7f623effd640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f621c001650 0x7f621c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.548+0000 7f623effd640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f621c001650 0x7f621c003b10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.552+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f621c006bb0 con 0x7f621c001650 2024-09-19T05:44:44.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.585+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f621c005c30 con 0x7f621c001650 2024-09-19T05:44:44.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.589+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f621c005c30 con 0x7f621c001650 2024-09-19T05:44:44.596 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:44.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:44.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.594+0000 7f6245798640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6218004190 0x7f62180045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.594+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f6218004bc0 con 0x7f6218004190 2024-09-19T05:44:44.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.598+0000 7f6237fff640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6218004190 0x7f62180045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.599+0000 7f6237fff640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f6218004190 0x7f62180045d0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.604+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6218004bc0 con 0x7f6218004190 2024-09-19T05:44:44.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.644+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f62180050d0 con 0x7f6218004190 2024-09-19T05:44:44.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.656+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f62180050d0 con 0x7f6218004190 2024-09-19T05:44:44.658 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:44.659 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:44.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.656+0000 7f6245798640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6218009370 0x7f621800b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.656+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f621800e640 con 0x7f6218009370 2024-09-19T05:44:44.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.660+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6218009370 0x7f621800b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.667 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.665+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6218009370 0x7f621800b760 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.682+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f621800e640 con 0x7f6218009370 2024-09-19T05:44:44.689 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:44.687+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:44.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.713+0000 7f6245798640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f621800bfa0 con 0x7f6218009370 2024-09-19T05:44:44.719 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.717+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f621800bfa0 con 0x7f6218009370 2024-09-19T05:44:44.720 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:44.720 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:44.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.719+0000 7f62357fa640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f621c004190 0x7f621c0045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.719+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f621c004b70 con 0x7f621c004190 2024-09-19T05:44:44.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.719+0000 7f623effd640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f621c004190 0x7f621c0045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.721+0000 7f623effd640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f621c004190 0x7f621c0045b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.725 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.722+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f621c004b70 con 0x7f621c004190 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5661 3.12 3:4aca477d:::100000001f8.0000081a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.668464+0000", 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.0690195, 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.069045856, 2024-09-19T05:44:44.739 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5661 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.668464+0000", 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.740 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.668464+0000", 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.668465+0000", 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.426e-06 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.741 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.673801+0000", 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053361850000000002 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.673818+0000", 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7248000000000002e-05 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.673835+0000", 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6929e-05 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.742 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.673907+0000", 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1721e-05 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.673995+0000", 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7489000000000002e-05 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.674098+0000", 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010381400000000001 2024-09-19T05:44:44.743 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.678564+0000", 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044658759999999997 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:6049 3.12 3:4b5b43b4:::100000001f8.0000099e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:44.110289+0000", 2024-09-19T05:44:44.744 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.62719388099999995, 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.62727050600000001, 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 6049 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.110289+0000", 2024-09-19T05:44:44.745 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.110289+0000", 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.110291+0000", 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1390000000000002e-06 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.746 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.128730+0000", 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018438942999999999 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.128754+0000", 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4009999999999999e-05 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.747 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.128766+0000", 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.199e-05 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.128871+0000", 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010430299999999999 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.129012+0000", 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014110899999999999 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.129145+0000", 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000133304 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.134463+0000", 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0053183630000000004 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.749 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5827 3.12 3:49d1bd5a:::100000001f8.000008c0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.827580+0000", 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.90990353000000002, 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.91002186399999996, 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5827 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.750 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.827580+0000", 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.827580+0000", 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.827582+0000", 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5040000000000001e-06 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.751 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834595+0000", 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070129119999999996 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834610+0000", 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5038000000000001e-05 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834628+0000", 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8406e-05 2024-09-19T05:44:44.752 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834692+0000", 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3306999999999995e-05 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834793+0000", 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100888 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834948+0000", 2024-09-19T05:44:44.753 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000155551 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.840038+0000", 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050898480000000001 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5666 3.12 3:48a69cf9:::100000001f8.0000081f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.688125+0000", 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.049358056, 2024-09-19T05:44:44.754 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0495174940000001, 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5666 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.755 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.688125+0000", 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.688125+0000", 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.688128+0000", 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.757e-06 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.756 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.701595+0000", 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013467382999999999 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.701611+0000", 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6107e-05 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.757 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.701624+0000", 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2231e-05 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.701706+0000", 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2719000000000005e-05 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.701803+0000", 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6340000000000003e-05 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.758 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.701919+0000", 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000115998 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.708239+0000", 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063202780000000004 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.759 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5567 3.12 3:48356219:::100000001f8.000007bc:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.533613+0000", 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.203870497, 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.204069485, 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5567 2024-09-19T05:44:44.760 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.533613+0000", 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.533613+0000", 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.761 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.533614+0000", 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.753e-06 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.551453+0000", 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017838712 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.551473+0000", 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0259999999999999e-05 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.762 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.551483+0000", 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0020000000000001e-05 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.551601+0000", 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011805 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.551735+0000", 2024-09-19T05:44:44.763 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013398300000000001 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.551928+0000", 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00019218499999999999 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.557356+0000", 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054287609999999998 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.764 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:6003 3.12 3:4be71d45:::100000001f8.00000970:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:44.009374+0000", 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.72810927700000005, 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.72836367300000004, 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 6003 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.009374+0000", 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.009374+0000", 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.766 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.009376+0000", 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2309999999999999e-06 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.018440+0000", 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0090643500000000005 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.018459+0000", 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8698e-05 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.767 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.018475+0000", 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6099e-05 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.018584+0000", 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108979 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.018672+0000", 2024-09-19T05:44:44.768 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8072999999999995e-05 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.018804+0000", 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013207200000000001 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.023500+0000", 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046953239999999999 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.769 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5512 3.12 3:49429726:::100000001f8.00000785:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.365221+0000", 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.372262498, 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.372555744, 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5512 2024-09-19T05:44:44.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.365221+0000", 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.365221+0000", 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.771 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.365223+0000", 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5940000000000001e-06 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.382174+0000", 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016950296 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.772 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.382190+0000", 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6962000000000001e-05 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.382202+0000", 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1781e-05 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.773 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.382436+0000", 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00023381300000000001 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.382766+0000", 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000330062 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.382914+0000", 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014755399999999999 2024-09-19T05:44:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.391514+0000", 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0086001480000000002 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5875 3.12 3:4962d06a:::100000001f8.000008f0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.871064+0000", 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.86641936399999997, 2024-09-19T05:44:44.775 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.86675021399999996, 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5875 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.871064+0000", 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.776 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.777 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.777 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.777 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.871064+0000", 2024-09-19T05:44:44.777 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.780 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.871066+0000", 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.187e-06 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.878180+0000", 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071139439999999996 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.878199+0000", 2024-09-19T05:44:44.781 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8856999999999999e-05 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.878209+0000", 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8640000000000008e-06 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.878283+0000", 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4326999999999995e-05 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.782 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.878360+0000", 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6688999999999997e-05 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.878491+0000", 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000131488 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.883039+0000", 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045478109999999997 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.783 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:6105 3.12 3:4b497566:::100000001f8.000009d6:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:44.172692+0000", 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.56479132399999998, 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.56515972999999997, 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 6105 2024-09-19T05:44:44.784 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.172692+0000", 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.172692+0000", 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.785 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.172694+0000", 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.407e-06 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.179632+0000", 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069375039999999997 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.179649+0000", 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7244e-05 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.786 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.179660+0000", 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1304000000000001e-05 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.179752+0000", 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1990999999999994e-05 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.179858+0000", 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000105559 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.787 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.180035+0000", 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000176847 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.185967+0000", 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059321950000000003 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5739 3.12 3:4a7c9af2:::100000001f8.00000868:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.788 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.731040+0000", 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 1.0064428620000001, 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.006849221, 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5739 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.789 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.731040+0000", 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.731040+0000", 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.731042+0000", 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.733e-06 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.790 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.739138+0000", 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0080960040000000004 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.739152+0000", 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4175000000000001e-05 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.739162+0000", 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5310000000000007e-06 2024-09-19T05:44:44.791 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.739217+0000", 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5556000000000001e-05 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.739306+0000", 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8450999999999997e-05 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.792 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.739483+0000", 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017751099999999999 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.745567+0000", 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060838589999999996 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:6015 3.12 3:493add79:::100000001f8.0000097c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:44.048865+0000", 2024-09-19T05:44:44.793 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 0.68861833299999997, 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.68908947600000003, 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "waiting for sub ops", 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 6015 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.048865+0000", 2024-09-19T05:44:44.794 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.048865+0000", 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.048867+0000", 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1459999999999999e-06 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.055983+0000", 2024-09-19T05:44:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0071158530000000001 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.056001+0000", 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7742000000000001e-05 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.056010+0000", 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6849999999999996e-06 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:44.796 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.056097+0000", 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.7113000000000004e-05 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.056167+0000", 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9945999999999997e-05 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.056286+0000", 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011899 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:44.797 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.060169+0000", 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0038828909999999999 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: ], 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 11 2024-09-19T05:44:44.798 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:44.798 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-19T05:44:44.798 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:44:44.799 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-19T05:44:44.799 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-19T05:44:44.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-19T05:44:44.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.755+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f621c004b70 con 0x7f621c004190 2024-09-19T05:44:44.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.757+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f621c004b70 con 0x7f621c004190 2024-09-19T05:44:44.801 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:44.802 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.759+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=75}) -- 0x7f621c004e60 con 0x7f62400c7df0 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.759+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f621c004b70 con 0x7f62400c7df0 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.761+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6228080040 con 0x7f62400c7df0 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.762+0000 7f62357fa640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f621c009310 0x7f621c00b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.762+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f621c00e630 con 0x7f621c009310 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.763+0000 7f6237fff640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f621c009310 0x7f621c00b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.764+0000 7f6237fff640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f621c009310 0x7f621c00b700 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.766+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f621c00e630 con 0x7f621c009310 2024-09-19T05:44:44.804 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:44.798+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:44.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.804+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f621c00bc40 con 0x7f621c009310 2024-09-19T05:44:44.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.806+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f621c00bc40 con 0x7f621c009310 2024-09-19T05:44:44.809 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:44.809 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:44.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.808+0000 7f62357fa640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f621c00c2c0 0x7f621c00c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:44.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.809+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f621c00e100 con 0x7f621c00c2c0 2024-09-19T05:44:44.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.809+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f621c00c2c0 0x7f621c00c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:44.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.816+0000 7f623f7fe640 1 --2- 172.21.15.2:0/1539963323 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f621c00c2c0 0x7f621c00c770 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:44.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.817+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f621c00e100 con 0x7f621c00c2c0 2024-09-19T05:44:44.843 INFO:tasks.ceph.osd.5:Started 2024-09-19T05:44:44.844 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:44.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.847+0000 7f62357fa640 1 -- 172.21.15.2:0/1539963323 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f621c00e050 con 0x7f621c00c2c0 2024-09-19T05:44:44.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:44.855+0000 7f62377fe640 1 -- 172.21.15.2:0/1539963323 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f621c00e050 con 0x7f621c00c2c0 2024-09-19T05:44:44.865 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:44.865 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:44.873 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:44.873 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:44.916 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:44.913+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:44.947 INFO:teuthology.orchestra.run.smithi123.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-19T05:44:44.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:44:44.950 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-19T05:44:44.960 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:44.960 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:44.969 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:45.003 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:45.001+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:45.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.019+0000 7f7a08ff9640 1 -- 172.21.15.2:0/1451476654 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v75) ==== 91+0+0 (secure 0 0 0) 0x7f7a0405b1f0 con 0x7f7a0c0c9b00 2024-09-19T05:44:45.026 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:44:45.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.025+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f806c380 msgr2=0x7f79f806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:45.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.025+0000 7f7a1108f640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f806c380 0x7f79f806e840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7a000096f0 tx=0x7f7a00006c40 comp rx=0 tx=0).stop 2024-09-19T05:44:45.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.025+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c0c9b00 msgr2=0x7f7a0c15b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:45.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.025+0000 7f7a1108f640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c15b900 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7a04007e90 tx=0x7f7a0400e4b0 comp rx=0 tx=0).stop 2024-09-19T05:44:45.027 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.026+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 shutdown_connections 2024-09-19T05:44:45.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.026+0000 7f7a1108f640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f806c380 0x7f79f806e840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:45.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.026+0000 7f7a1108f640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7a0c0d0820 0x7f7a0c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:45.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.026+0000 7f7a1108f640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c0c9b00 0x7f7a0c15b900 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:45.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.026+0000 7f7a1108f640 1 --2- 172.21.15.2:0/1451476654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0b5b10 0x7f7a0c15ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:45.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.026+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 >> 172.21.15.2:0/1451476654 conn(0x7f7a0c06cb70 msgr2=0x7f7a0c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:45.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.031+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 shutdown_connections 2024-09-19T05:44:45.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:45.032+0000 7f7a1108f640 1 -- 172.21.15.2:0/1451476654 wait complete. 2024-09-19T05:44:45.103 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:45.100+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:45.122 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.122 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:45.205 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:45.203+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:45.249 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.249 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:45.300 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:45.300 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 rewriters = 1407116.12 kB/sec 2024-09-19T05:44:45.300 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 rewriters = 1407077.00 kB/sec 2024-09-19T05:44:45.300 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 1407116.12 kB/sec 2024-09-19T05:44:45.300 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 1407116.12 kB/sec 2024-09-19T05:44:45.301 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 1407116.12 kB/sec 2024-09-19T05:44:45.301 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:44:45.304 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:45.302+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:45.362 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.362 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:45.413 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:45.412+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:45.473 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.473 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.482 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:45.526 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:45.524+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:45.595 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.595 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.604 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:45.606 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:45.605+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:45.707 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:45.705+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:45.751 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.751 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.760 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:45.808 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:45.806+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:45.869 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.869 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.878 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:45.922 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:45.920+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:45.985 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:45.985 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:45.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:46.010 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:46.008+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:46.111 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:46.109+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:46.143 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:46.143 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:46.154 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:46.212 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:46.210+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:46.265 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:46.265 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:46.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:46.318 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:46.316+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:46.347 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:46.346+0000 7f9b31412640 -1 Falling back to public interface 2024-09-19T05:44:46.381 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:46.382 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:46.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:46.414 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:46.412+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:46.515 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:46.513+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:46.538 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:46.538 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:46.547 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:46.616 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:46.614+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:46.655 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:46.655 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:46.664 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:46.717 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:46.715+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:46.773 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:46.773 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:46.781 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:46.823 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:46.821+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:46.875 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:46.918 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:46.917+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:46.930 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:46.931 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:46.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:46.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.974+0000 7f6585759640 1 Processor -- start 2024-09-19T05:44:46.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.974+0000 7f6585759640 1 -- start start 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.975+0000 7f6585759640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f65800cd4e0 0x7f65800b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.975+0000 7f6585759640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f658015ebe0 con 0x7f658015aa90 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.975+0000 7f6585759640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65800b2e00 con 0x7f65800cd8c0 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.975+0000 7f6585759640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f658015eec0 con 0x7f65800cd4e0 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.975+0000 7f657e7fc640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f658015aa90 0x7f658015df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:56810/0 (socket says 172.21.15.2:56810) 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.975+0000 7f657effd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f65800cd4e0 0x7f65800b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.975+0000 7f657e7fc640 1 -- 172.21.15.2:0/2676089530 learned_addr learned my addr 172.21.15.2:0/2676089530 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.976+0000 7f657effd640 1 -- 172.21.15.2:0/2676089530 >> v1:172.21.15.123:6789/0 conn(0x7f65800cd8c0 legacy=0x7f65800c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.976+0000 7f657effd640 1 -- 172.21.15.2:0/2676089530 >> v1:172.21.15.2:6789/0 conn(0x7f658015aa90 legacy=0x7f658015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:46.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.976+0000 7f657effd640 1 -- 172.21.15.2:0/2676089530 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65800b30e0 con 0x7f65800cd4e0 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.976+0000 7f657effd640 1 --2- 172.21.15.2:0/2676089530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd4e0 0x7f65800b3bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f65700097a0 tx=0x7f657002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0f939cb30ac65e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f657dffb640 1 -- 172.21.15.2:0/2676089530 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570037070 con 0x7f65800cd4e0 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f657dffb640 1 -- 172.21.15.2:0/2676089530 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6570032440 con 0x7f65800cd4e0 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f657dffb640 1 -- 172.21.15.2:0/2676089530 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570032810 con 0x7f65800cd4e0 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f6585759640 1 -- 172.21.15.2:0/2676089530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd4e0 msgr2=0x7f65800b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f6585759640 1 --2- 172.21.15.2:0/2676089530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd4e0 0x7f65800b3bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f65700097a0 tx=0x7f657002a1e0 comp rx=0 tx=0).stop 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f6585759640 1 -- 172.21.15.2:0/2676089530 shutdown_connections 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f6585759640 1 --2- 172.21.15.2:0/2676089530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd4e0 0x7f65800b3bc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f6585759640 1 -- 172.21.15.2:0/2676089530 >> 172.21.15.2:0/2676089530 conn(0x7f6580067b30 msgr2=0x7f6580067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:46.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f6585759640 1 -- 172.21.15.2:0/2676089530 shutdown_connections 2024-09-19T05:44:46.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.977+0000 7f6585759640 1 -- 172.21.15.2:0/2676089530 wait complete. 2024-09-19T05:44:46.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 Processor -- start 2024-09-19T05:44:46.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 -- start start 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65800cd4e0 0x7f65800cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd8c0 0x7f65800c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658015aa90 0x7f65801f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f658015e640 con 0x7f65800cd8c0 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65800b36a0 con 0x7f65800cd4e0 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f6585759640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f658015e7c0 con 0x7f658015aa90 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f657e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd8c0 0x7f65800c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f657e7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd8c0 0x7f65800c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60222/0 (socket says 172.21.15.2:60222) 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.979+0000 7f657e7fc640 1 -- 172.21.15.2:0/2634418043 learned_addr learned my addr 172.21.15.2:0/2634418043 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657effd640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65800cd4e0 0x7f65800cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657f7fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658015aa90 0x7f65801f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:46.983 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657e7fc640 1 -- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658015aa90 msgr2=0x7f65801f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657e7fc640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f658015aa90 0x7f65801f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657e7fc640 1 -- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65800cd4e0 msgr2=0x7f65800cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657e7fc640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65800cd4e0 0x7f65800cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657e7fc640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65800ce420 con 0x7f65800cd8c0 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f657e7fc640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65800cd8c0 0x7f65800c8c10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f656800caa0 tx=0x7f656800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568017070 con 0x7f65800cd8c0 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656800ac00 con 0x7f65800cd8c0 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6568012400 con 0x7f65800cd8c0 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65800ce700 con 0x7f65800cd8c0 2024-09-19T05:44:46.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.980+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65800cecc0 con 0x7f65800cd8c0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.981+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f65800b59a0 con 0x7f65800cd8c0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.982+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f656801a070 con 0x7f65800cd8c0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.982+0000 7f65777fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f656006c2b0 0x7f656006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.982+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5799+0+0 (secure 0 0 0) 0x7f656808b290 con 0x7f65800cd8c0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.983+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v75) ==== 54+0+16 (secure 0 0 0) 0x7f65800b59a0 con 0x7f65800cd8c0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.983+0000 7f657effd640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f656006c2b0 0x7f656006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.983+0000 7f6585759640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f65801e5e80 0x7f65800cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.983+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f658015e3f0 con 0x7f65801e5e80 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.983+0000 7f657f7fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f65801e5e80 0x7f65800cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.983+0000 7f657f7fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f65801e5e80 0x7f65800cb7e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:46.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.983+0000 7f657effd640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f656006c2b0 0x7f656006e770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6570034000 tx=0x7f657002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:46.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.984+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f658015e3f0 con 0x7f65801e5e80 2024-09-19T05:44:47.001 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:46.999+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f65800b6220 con 0x7f65801e5e80 2024-09-19T05:44:47.003 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.001+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f65800b6220 con 0x7f65801e5e80 2024-09-19T05:44:47.004 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:47.004 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:47.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.001+0000 7f6585759640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f658014bb00 0x7f65801f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.001+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6580154d40 con 0x7f658014bb00 2024-09-19T05:44:47.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.002+0000 7f657effd640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f658014bb00 0x7f65801f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.002+0000 7f657effd640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f658014bb00 0x7f65801f42b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.002+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6580154d40 con 0x7f658014bb00 2024-09-19T05:44:47.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.017+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f6580068320 con 0x7f658014bb00 2024-09-19T05:44:47.020 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:47.018+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:47.021 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.019+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6580068320 con 0x7f658014bb00 2024-09-19T05:44:47.025 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.019+0000 7f6585759640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f65801f4d00 0x7f6580062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.019+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f6580068040 con 0x7f65801f4d00 2024-09-19T05:44:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.019+0000 7f657e7fc640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f65801f4d00 0x7f6580062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.020+0000 7f657e7fc640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f65801f4d00 0x7f6580062630 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.020+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6580068040 con 0x7f65801f4d00 2024-09-19T05:44:47.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.035+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f65801e47f0 con 0x7f65801f4d00 2024-09-19T05:44:47.038 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.036+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f65801e47f0 con 0x7f65801f4d00 2024-09-19T05:44:47.038 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:47.039 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:47.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.036+0000 7f6585759640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6580063080 0x7f658006c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.036+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f6580063480 con 0x7f6580063080 2024-09-19T05:44:47.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.037+0000 7f657f7fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6580063080 0x7f658006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.039 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.037+0000 7f657f7fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f6580063080 0x7f658006c970 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.040 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.038+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6580063480 con 0x7f6580063080 2024-09-19T05:44:47.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.052+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f658006d180 con 0x7f6580063080 2024-09-19T05:44:47.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.053+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f658006d180 con 0x7f6580063080 2024-09-19T05:44:47.056 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:47.056 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:47.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.054+0000 7f6585759640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f658002c640 0x7f658002cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.054+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f658006d100 con 0x7f658002c640 2024-09-19T05:44:47.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.054+0000 7f657effd640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f658002c640 0x7f658002cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.054+0000 7f657effd640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f658002c640 0x7f658002cac0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.055+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f658006d100 con 0x7f658002c640 2024-09-19T05:44:47.058 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2666 3.9 3:92ca8d9b:::100000001f8.00000667:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.161256761000001, 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3293169570000001, 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2666 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894727+0000", 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8439999999999998e-06 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897799+0000", 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030718199999999998 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897817+0000", 2024-09-19T05:44:47.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7847999999999999e-05 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897831+0000", 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3845999999999999e-05 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897938+0000", 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010742399999999999 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898111+0000", 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017279099999999999 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898360+0000", 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024885800000000002 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956644+0000", 2024-09-19T05:44:47.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058284217999999999 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223960+0000", 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.267316119 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224004+0000", 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4039000000000003e-05 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224042+0000", 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7150000000000002e-05 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2667 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.158116913000001, 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3258909560000001, 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2667 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:47.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897867+0000", 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9299999999999999e-06 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902567+0000", 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004699356 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902588+0000", 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1631e-05 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902602+0000", 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3478e-05 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.906894+0000", 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042925760000000002 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907078+0000", 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183067 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907289+0000", 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021123499999999999 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956671+0000", 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049382078000000003 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223652+0000", 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.266981122 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223704+0000", 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2244999999999999e-05 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223755+0000", 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1238e-05 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.070 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2673 3.2d 3:b45a3f48:::100000001f8.0000066e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.145119486, 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.373980822, 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2673 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910864+0000", 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915100+0000", 2024-09-19T05:44:47.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042352259999999999 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915123+0000", 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3334999999999999e-05 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915137+0000", 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4178999999999999e-05 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915220+0000", 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2693000000000003e-05 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915395+0000", 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017551099999999999 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915574+0000", 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178357 2024-09-19T05:44:47.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951064+0000", 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035490116000000002 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284764+0000", 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.333700259 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284808+0000", 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3813999999999998e-05 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284843+0000", 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4807000000000001e-05 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2700 3.30 3:0e790ae4:::100000001f8.00000689:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.112417384, 2024-09-19T05:44:47.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.25937777, 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2700 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943567+0000", 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.083e-06 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946402+0000", 2024-09-19T05:44:47.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028349439999999998 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946419+0000", 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7105999999999999e-05 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946429+0000", 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8709999999999992e-06 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.947951+0000", 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015219960000000001 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948153+0000", 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020153400000000001 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948415+0000", 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026250200000000002 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.064312+0000", 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115897369 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202811+0000", 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.138498891 2024-09-19T05:44:47.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202872+0000", 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0529000000000003e-05 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202942+0000", 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9944999999999995e-05 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2711 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:47.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.104899775, 2024-09-19T05:44:47.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.27282242, 2024-09-19T05:44:47.084 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.084 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2711 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951084+0000", 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.553e-06 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956906+0000", 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058222400000000002 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956926+0000", 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9619e-05 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956937+0000", 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0594999999999999e-05 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958546+0000", 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001609369 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958792+0000", 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024587600000000002 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.959831+0000", 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010394079999999999 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293509+0000", 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33367725100000001 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223817+0000", 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93030886999999995 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223864+0000", 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6465000000000003e-05 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223904+0000", 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0173999999999999e-05 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2732 3.9 3:92ca8380:::100000001f8.000006a9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.048030171000001, 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2619013319999999, 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2732 2024-09-19T05:44:47.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007954+0000", 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018132+0000", 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010177734000000001 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018148+0000", 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6555999999999999e-05 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018187+0000", 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8760000000000002e-05 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020347+0000", 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021597589999999998 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020514+0000", 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016697899999999999 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020695+0000", 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018172199999999999 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.375158+0000", 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35446218200000001 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269697+0000", 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89453943599999997 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269768+0000", 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0619999999999998e-05 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269853+0000", 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4876000000000005e-05 2024-09-19T05:44:47.096 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2746 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.020492959, 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1890195509999999, 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2746 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035491+0000", 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043360+0000", 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0078693240000000005 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043383+0000", 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2853e-05 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.045231+0000", 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001847476 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.122884+0000", 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077653244999999996 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123275+0000", 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039126200000000002 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123895+0000", 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061987200000000004 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293518+0000", 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.169623147 2024-09-19T05:44:47.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224430+0000", 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93091173699999996 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224474+0000", 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3816000000000002e-05 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224508+0000", 2024-09-19T05:44:47.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4369000000000002e-05 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2753 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.994470121999999, 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.188208317, 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2753 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061513+0000", 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.871e-06 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078405+0000", 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016891461 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078433+0000", 2024-09-19T05:44:47.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8436e-05 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078447+0000", 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3607e-05 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137285+0000", 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058837911 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137457+0000", 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017199399999999999 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137637+0000", 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180173 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.460405+0000", 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.322768426 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249610+0000", 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78920498100000003 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249664+0000", 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3668000000000002e-05 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249720+0000", 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5788999999999997e-05 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2808 3.11 3:8bd5b656:::100000001f8.000006f5:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.815853859000001, 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.160580451, 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2808 2024-09-19T05:44:47.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240131+0000", 2024-09-19T05:44:47.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244285+0000", 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041546919999999998 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244303+0000", 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7646999999999999e-05 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244316+0000", 2024-09-19T05:44:47.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2931999999999999e-05 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373296+0000", 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12897989000000001 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373463+0000", 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166847 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373621+0000", 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015846400000000001 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.585995+0000", 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.212374013 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400581+0000", 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81458587900000001 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400646+0000", 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5036999999999994e-05 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400708+0000", 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1937e-05 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5105 3.11 3:89c7a7f0:::100000001f8.000005ee:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:44:47.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.3355629069999999, 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2072587750000001, 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5105 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:44:47.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720422+0000", 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.241e-06 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723834+0000", 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003412389 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723847+0000", 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2404e-05 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723856+0000", 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6609999999999995e-06 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723919+0000", 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2928999999999993e-05 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.724040+0000", 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120505 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.724239+0000", 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019924600000000001 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.782736+0000", 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058496758000000003 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927592+0000", 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1448568539999999 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927639+0000", 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6495000000000001e-05 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927677+0000", 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8293000000000001e-05 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5127 3.11 3:8b184820:::100000001f8.00000604:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.291560467, 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.163370802, 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5127 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764428+0000", 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.778132+0000", 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013703958 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.790747+0000", 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012615167 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.790767+0000", 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9216999999999999e-05 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.793911+0000", 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031443019999999999 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.794025+0000", 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011406200000000001 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.794225+0000", 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001997 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.836693+0000", 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042467799000000001 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927716+0000", 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0910237540000001 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927758+0000", 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1959999999999998e-05 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927792+0000", 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3470000000000003e-05 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5160 3.14 3:2aefce75:::100000001f8.00000625:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1804657250000004, 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3210729219999999, 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5160 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875518+0000", 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.880298+0000", 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047798010000000002 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.888744+0000", 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084454649999999992 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.888764+0000", 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0517e-05 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889116+0000", 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000351391 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889240+0000", 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012396099999999999 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889440+0000", 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020088800000000001 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.939338+0000", 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049897237999999997 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196475+0000", 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.257137682 2024-09-19T05:44:47.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196526+0000", 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0680999999999998e-05 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196589+0000", 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2575000000000003e-05 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5205 3.2d 3:b48c1fbb:::100000001f8.00000652:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0055774739999999, 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1819966100000001, 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5205 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050407+0000", 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5149999999999998e-06 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053361+0000", 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029540410000000001 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053381+0000", 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9471999999999999e-05 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053393+0000", 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372360+0000", 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.318966784 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372504+0000", 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014391600000000001 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372718+0000", 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021371699999999999 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.499375+0000", 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12665742599999999 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232287+0000", 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73291224399999999 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232347+0000", 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9920000000000002e-05 2024-09-19T05:44:47.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232401+0000", 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3273999999999998e-05 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5227 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.9710671570000002, 2024-09-19T05:44:47.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.532865806, 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5227 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:44:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084919+0000", 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7650000000000001e-06 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091910+0000", 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069906250000000003 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091939+0000", 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9274000000000001e-05 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091955+0000", 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6280999999999999e-05 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165174+0000", 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073219124999999996 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165300+0000", 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012565699999999999 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165490+0000", 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000190519 2024-09-19T05:44:47.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.233620+0000", 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068129713999999994 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617666+0000", 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.384046068 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617724+0000", 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7410000000000001e-05 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617780+0000", 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6368e-05 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5271 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.8110547260000001, 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.372975533, 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5271 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244929+0000", 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6460000000000002e-06 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.251094+0000", 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061647539999999997 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258025+0000", 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00693065 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258041+0000", 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6708999999999999e-05 2024-09-19T05:44:47.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258150+0000", 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010840000000000001 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258311+0000", 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000160688 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.260274+0000", 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001963027 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359521+0000", 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099247940000000007 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617826+0000", 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.258304525 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617868+0000", 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2454999999999999e-05 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617902+0000", 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3738999999999997e-05 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5306 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:44:47.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6624391100000002, 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.224487935, 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5306 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:44:47.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393546+0000", 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6389999999999998e-06 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.398380+0000", 2024-09-19T05:44:47.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048341859999999999 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.433026+0000", 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034645428999999998 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.433052+0000", 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6945e-05 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.435900+0000", 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028475969999999999 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.436007+0000", 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010702 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.436183+0000", 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017633300000000001 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531619+0000", 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.095435107000000005 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617951+0000", 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0863320190000001 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617995+0000", 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4474999999999997e-05 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.618030+0000", 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5185000000000003e-05 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5313 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6179554120000001, 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2520088149999999, 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.155 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5313 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438028+0000", 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1610000000000001e-06 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440687+0000", 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026591789999999998 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440706+0000", 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8236999999999998e-05 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440717+0000", 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.171e-05 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444350+0000", 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036329819999999999 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444438+0000", 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7292000000000004e-05 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444578+0000", 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013099999999999 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.684423+0000", 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.239845633 2024-09-19T05:44:47.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.689914+0000", 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.005491175 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.689975+0000", 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0943999999999998e-05 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690035+0000", 2024-09-19T05:44:47.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9370999999999998e-05 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5329 3.11 3:8be9d197:::100000001f8.000006ce:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6092662899999999, 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.24346679, 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5329 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446718+0000", 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2169999999999999e-06 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453551+0000", 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068323860000000002 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453567+0000", 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6538e-05 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453579+0000", 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.218e-05 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.475998+0000", 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022418223000000001 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.477939+0000", 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001941639 2024-09-19T05:44:47.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478106+0000", 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016716699999999999 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.780733+0000", 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.302626495 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690099+0000", 2024-09-19T05:44:47.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90936588600000001 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690144+0000", 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5019999999999999e-05 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690182+0000", 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8039e-05 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5333 3.9 3:9102af4b:::100000001f8.000006d2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5979923519999999, 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1967455760000001, 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5333 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457991+0000", 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.08e-06 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462463+0000", 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044715370000000003 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462495+0000", 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2617999999999998e-05 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462510+0000", 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4452e-05 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531565+0000", 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069055635000000004 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531664+0000", 2024-09-19T05:44:47.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8634999999999993e-05 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531822+0000", 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015805600000000001 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.867710+0000", 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33588781299999998 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654623+0000", 2024-09-19T05:44:47.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78691265499999996 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654683+0000", 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0380000000000001e-05 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654735+0000", 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1715000000000002e-05 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5356 3.9 3:9157a25d:::100000001f8.000006e9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5741729490000003, 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.173045817, 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5356 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.173 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481811+0000", 2024-09-19T05:44:47.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7159999999999999e-06 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493724+0000", 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011912668 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493759+0000", 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4968999999999997e-05 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493772+0000", 2024-09-19T05:44:47.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2845e-05 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645428+0000", 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15165590900000001 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645559+0000", 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013117000000000001 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:47.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645810+0000", 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000251432 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.867748+0000", 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22193777100000001 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654779+0000", 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78703058599999998 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654821+0000", 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2540999999999998e-05 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654854+0000", 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210000000000002e-05 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:47.178 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:47.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:44:47.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:47.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.069+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f658006ceb0 con 0x7f658002c640 2024-09-19T05:44:47.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.071+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f658006ceb0 con 0x7f658002c640 2024-09-19T05:44:47.181 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:47.182 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.071+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=76}) -- 0x7f658006ceb0 con 0x7f65800cd8c0 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.071+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f658002d000 con 0x7f65800cd8c0 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.072+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6568092070 con 0x7f65800cd8c0 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.072+0000 7f6585759640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6580205630 0x7f6580207a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.072+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f658002dba0 con 0x7f6580205630 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.072+0000 7f657e7fc640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6580205630 0x7f6580207a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.073+0000 7f657e7fc640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6580205630 0x7f6580207a20 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.073+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f658002dba0 con 0x7f6580205630 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.087+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f658002dba0 con 0x7f6580205630 2024-09-19T05:44:47.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.089+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f658002dba0 con 0x7f6580205630 2024-09-19T05:44:47.184 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:47.184 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.090+0000 7f6585759640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6580208470 0x7f658020a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.090+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f658020d6b0 con 0x7f6580208470 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.090+0000 7f657f7fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6580208470 0x7f658020a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.090+0000 7f657f7fe640 1 --2- 172.21.15.2:0/2634418043 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f6580208470 0x7f658020a860 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.091+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f658020d6b0 con 0x7f6580208470 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.105+0000 7f6585759640 1 -- 172.21.15.2:0/2634418043 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f658020ae20 con 0x7f6580208470 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.107+0000 7f65777fe640 1 -- 172.21.15.2:0/2634418043 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f658020ae20 con 0x7f6580208470 2024-09-19T05:44:47.185 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:47.186 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:47.186 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:47.187 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:47.187 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:47.182+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:47.281 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:47.279+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:47.308 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:47.309 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:47.309 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:47.309 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:47.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:47.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.330+0000 7f5b5a04c640 1 Processor -- start 2024-09-19T05:44:47.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.330+0000 7f5b5a04c640 1 -- start start 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b5a04c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b540c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b5a04c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b540c4070 con 0x7f5b54073c70 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b5a04c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b541579d0 con 0x7f5b54073890 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b5a04c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5b54157cb0 con 0x7f5b54072bd0 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b52ffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5b54073c70 0x7f5b540724c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:56812/0 (socket says 172.21.15.2:56812) 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b52ffd640 1 -- 172.21.15.2:0/3140950634 learned_addr learned my addr 172.21.15.2:0/3140950634 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b53fff640 1 --2- 172.21.15.2:0/3140950634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b540c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b527fc640 1 -- 172.21.15.2:0/3140950634 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2043767350 0 0) 0x7f5b540c4070 con 0x7f5b54073c70 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b527fc640 1 -- 172.21.15.2:0/3140950634 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b3c003520 con 0x7f5b54073c70 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b527fc640 1 -- 172.21.15.2:0/3140950634 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767831675 0 0) 0x7f5b541579d0 con 0x7f5b54073890 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b527fc640 1 -- 172.21.15.2:0/3140950634 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b540c4070 con 0x7f5b54073890 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b53fff640 1 -- 172.21.15.2:0/3140950634 >> v1:172.21.15.123:6789/0 conn(0x7f5b54073890 legacy=0x7f5b5415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.334 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b53fff640 1 -- 172.21.15.2:0/3140950634 >> v1:172.21.15.2:6789/0 conn(0x7f5b54073c70 legacy=0x7f5b540724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.335 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.331+0000 7f5b53fff640 1 -- 172.21.15.2:0/3140950634 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b54157f90 con 0x7f5b54072bd0 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.332+0000 7f5b53fff640 1 --2- 172.21.15.2:0/3140950634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b540c3950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009dc0 tx=0x7f5b4c02a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec47ab1c794ad03d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.332+0000 7f5b527fc640 1 -- 172.21.15.2:0/3140950634 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c007740 con 0x7f5b54072bd0 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.332+0000 7f5b527fc640 1 -- 172.21.15.2:0/3140950634 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c0078e0 con 0x7f5b54072bd0 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.332+0000 7f5b527fc640 1 -- 172.21.15.2:0/3140950634 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c007c90 con 0x7f5b54072bd0 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.333+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3140950634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 msgr2=0x7f5b540c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.333+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3140950634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b540c3950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009dc0 tx=0x7f5b4c02a760 comp rx=0 tx=0).stop 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.333+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3140950634 shutdown_connections 2024-09-19T05:44:47.336 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.333+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3140950634 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b540c3950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.333+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3140950634 >> 172.21.15.2:0/3140950634 conn(0x7f5b5406bc10 msgr2=0x7f5b5406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:47.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.333+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3140950634 shutdown_connections 2024-09-19T05:44:47.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.333+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3140950634 wait complete. 2024-09-19T05:44:47.337 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.334+0000 7f5b5a04c640 1 Processor -- start 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.334+0000 7f5b5a04c640 1 -- start start 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b5a04c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b541ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b5a04c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b54073890 0x7f5b5414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b5a04c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b54073c70 0x7f5b540cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b5a04c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54071bc0 con 0x7f5b54073890 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b5a04c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54158550 con 0x7f5b54073c70 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b5a04c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5b54151670 con 0x7f5b54072bd0 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b54073890 0x7f5b5414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b537fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b541ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b54073890 0x7f5b5414d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60224/0 (socket says 172.21.15.2:60224) 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 -- 172.21.15.2:0/3201050604 learned_addr learned my addr 172.21.15.2:0/3201050604 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b53fff640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b54073c70 0x7f5b540cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.339 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 -- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 msgr2=0x7f5b541ef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b541ef070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 -- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b54073c70 msgr2=0x7f5b540cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b54073c70 0x7f5b540cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.335+0000 7f5b52ffd640 1 -- 172.21.15.2:0/3201050604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b540caa00 con 0x7f5b54073890 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.336+0000 7f5b52ffd640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b54073890 0x7f5b5414d3e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c3e0 tx=0x7f5b4800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.336+0000 7f5b3bfff640 1 -- 172.21.15.2:0/3201050604 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48017070 con 0x7f5b54073890 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.336+0000 7f5b3bfff640 1 -- 172.21.15.2:0/3201050604 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b48005d00 con 0x7f5b54073890 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.336+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b540cace0 con 0x7f5b54073890 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.336+0000 7f5b3bfff640 1 -- 172.21.15.2:0/3201050604 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48007310 con 0x7f5b54073890 2024-09-19T05:44:47.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.336+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b540cb2a0 con 0x7f5b54073890 2024-09-19T05:44:47.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.337+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b540cba30 con 0x7f5b54073890 2024-09-19T05:44:47.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.338+0000 7f5b3bfff640 1 -- 172.21.15.2:0/3201050604 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5b4801a070 con 0x7f5b54073890 2024-09-19T05:44:47.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.338+0000 7f5b3bfff640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5b4006c2e0 0x7f5b4006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:47.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.338+0000 7f5b3bfff640 1 -- 172.21.15.2:0/3201050604 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5799+0+0 (secure 0 0 0) 0x7f5b4808a7b0 con 0x7f5b54073890 2024-09-19T05:44:47.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.338+0000 7f5b537fe640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5b4006c2e0 0x7f5b4006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:47.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.339+0000 7f5b537fe640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5b4006c2e0 0x7f5b4006e7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5b440011e0 tx=0x7f5b44009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:47.345 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.343+0000 7f5b3bfff640 1 -- 172.21.15.2:0/3201050604 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5b480589c0 con 0x7f5b54073890 2024-09-19T05:44:47.382 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:47.380+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:47.427 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:47.427 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:47.427 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:47.428 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:47.428 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:47.436 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:47.483 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:47.481+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:47.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:47.526+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f5b540c3950 con 0x7f5b54073890 2024-09-19T05:44:47.541 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:47.541 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:47.541 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:47.541 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5170 3.2 3:40ee304f:::100000001f8.0000062f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.6903739819999997, 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.07064564, 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5170 2024-09-19T05:44:47.542 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.543 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848608+0000", 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6699999999999998e-06 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861615+0000", 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013006636 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861637+0000", 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2447e-05 2024-09-19T05:44:47.544 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861651+0000", 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3431000000000001e-05 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861732+0000", 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1124000000000005e-05 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861822+0000", 2024-09-19T05:44:47.545 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9981000000000005e-05 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861969+0000", 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014714700000000001 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.868505+0000", 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065363340000000004 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.546 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919134+0000", 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.050628731 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919203+0000", 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9207000000000003e-05 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919251+0000", 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7932000000000001e-05 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.547 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5191 3.2 3:41cf9845:::100000001f8.00000644:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.6268550299999998, 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1864950409999999, 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.548 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5191 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.549 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912127+0000", 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9400000000000001e-06 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925516+0000", 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013389509000000001 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925536+0000", 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9579e-05 2024-09-19T05:44:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925544+0000", 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3450000000000006e-06 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925643+0000", 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9018000000000006e-05 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925711+0000", 2024-09-19T05:44:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8351999999999999e-05 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925816+0000", 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104832 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.931570+0000", 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057541279999999998 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098516+0000", 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1669453750000001 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098577+0000", 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1736999999999995e-05 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098620+0000", 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2225999999999999e-05 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5264 3.2b 3:d5fb1e74:::100000001f8.0000068d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.5388664079999996, 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.098400515, 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5264 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000115+0000", 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0650000000000001e-06 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013720+0000", 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013604842000000001 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013737+0000", 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7173999999999999e-05 2024-09-19T05:44:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013745+0000", 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4850000000000003e-06 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013799+0000", 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4543000000000003e-05 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013869+0000", 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0032999999999998e-05 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013997+0000", 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012786399999999999 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.018990+0000", 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049925860000000002 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098402+0000", 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0794119390000001 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098460+0000", 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8700999999999999e-05 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098514+0000", 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3282999999999998e-05 2024-09-19T05:44:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5286 3.2 3:41c49e27:::100000001f8.000006a3:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.5252146800000004, 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.368367997, 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5286 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013767+0000", 2024-09-19T05:44:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0320000000000002e-06 2024-09-19T05:44:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020719+0000", 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069518940000000001 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020732+0000", 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3560999999999999e-05 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020743+0000", 2024-09-19T05:44:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0156000000000001e-05 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020811+0000", 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8093e-05 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020888+0000", 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7484000000000006e-05 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.021010+0000", 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012149 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032253+0000", 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011243367000000001 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382067+0000", 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3498136039999999 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382105+0000", 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8525000000000001e-05 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382133+0000", 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7790999999999999e-05 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5291 3.2 3:433c69f2:::100000001f8.000006a8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.5182098230000003, 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.361485925, 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5291 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020771+0000", 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5850000000000001e-06 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027794+0000", 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070227889999999998 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027809+0000", 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4959e-05 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027817+0000", 2024-09-19T05:44:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027886+0000", 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8356999999999996e-05 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027986+0000", 2024-09-19T05:44:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010082900000000001 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.028127+0000", 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000140829 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032805+0000", 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046781679999999999 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382178+0000", 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3493729270000001 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382219+0000", 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0924999999999999e-05 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382256+0000", 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6430999999999997e-05 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5360 3.2b 3:d6b2cae8:::100000001f8.000006ed:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.381813427, 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1641596679999999, 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5360 2024-09-19T05:44:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:44:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157169+0000", 2024-09-19T05:44:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089999999999999e-06 2024-09-19T05:44:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173455+0000", 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016286248 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173477+0000", 2024-09-19T05:44:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1716999999999999e-05 2024-09-19T05:44:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173488+0000", 2024-09-19T05:44:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1620000000000001e-05 2024-09-19T05:44:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173563+0000", 2024-09-19T05:44:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4161999999999999e-05 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173653+0000", 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9974000000000003e-05 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173776+0000", 2024-09-19T05:44:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012363000000000001 2024-09-19T05:44:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.179905+0000", 2024-09-19T05:44:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061287110000000002 2024-09-19T05:44:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321250+0000", 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1413448260000001 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321287+0000", 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6937999999999998e-05 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321326+0000", 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9233000000000002e-05 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5363 3.2 3:41c08734:::100000001f8.000006f0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.3654628720000002, 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2088135710000001, 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5363 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:44:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173519+0000", 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3089999999999998e-06 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186577+0000", 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013058449999999999 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186607+0000", 2024-09-19T05:44:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9872000000000001e-05 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186618+0000", 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0772e-05 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186771+0000", 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000153024 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186856+0000", 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4907999999999994e-05 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186976+0000", 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011951 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.193253+0000", 2024-09-19T05:44:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062774969999999999 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382286+0000", 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1890333310000001 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382310+0000", 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4083e-05 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382330+0000", 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9814999999999999e-05 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5383 3.2 3:42776a0c:::100000001f8.00000704:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.3257874709999999, 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.270768092, 2024-09-19T05:44:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5383 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:44:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213194+0000", 2024-09-19T05:44:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000002e-06 2024-09-19T05:44:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230888+0000", 2024-09-19T05:44:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017693296000000001 2024-09-19T05:44:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230908+0000", 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0624000000000001e-05 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230918+0000", 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5119999999999997e-06 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231017+0000", 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9586000000000005e-05 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231189+0000", 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172053 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231464+0000", 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274916 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.238533+0000", 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007068469 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.483856+0000", 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2453235620000001 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.483912+0000", 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5911000000000001e-05 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.483960+0000", 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8047000000000003e-05 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5390 3.2 3:4215b5dd:::100000001f8.0000070b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.3080062049999999, 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.253141504, 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5390 2024-09-19T05:44:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230976+0000", 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.419e-06 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244642+0000", 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013666417 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244663+0000", 2024-09-19T05:44:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0454000000000001e-05 2024-09-19T05:44:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244690+0000", 2024-09-19T05:44:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6962e-05 2024-09-19T05:44:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244765+0000", 2024-09-19T05:44:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4884999999999999e-05 2024-09-19T05:44:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244873+0000", 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108904 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.245011+0000", 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013788200000000001 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.251134+0000", 2024-09-19T05:44:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006122677 2024-09-19T05:44:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484025+0000", 2024-09-19T05:44:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2328906040000001 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484074+0000", 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9221000000000002e-05 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484115+0000", 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1078999999999998e-05 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5403 3.2b 3:d515b219:::100000001f8.00000718:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.2038725210000001, 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.99261407000000001, 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5403 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335110+0000", 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.27e-06 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351404+0000", 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016293426 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351429+0000", 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5131999999999999e-05 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351442+0000", 2024-09-19T05:44:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3281e-05 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351543+0000", 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100887 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351640+0000", 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6569000000000003e-05 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351796+0000", 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015682900000000001 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.358222+0000", 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064259030000000002 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327636+0000", 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.96941361299999995 2024-09-19T05:44:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327678+0000", 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1897000000000002e-05 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327721+0000", 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3263000000000002e-05 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5405 3.2 3:41109be1:::100000001f8.0000071a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.1875041839999998, 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1327694880000001, 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5405 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351477+0000", 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.846e-06 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367351+0000", 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015873627000000001 2024-09-19T05:44:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367458+0000", 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106702 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367496+0000", 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8389000000000001e-05 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367630+0000", 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013432400000000001 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367737+0000", 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010691700000000001 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367859+0000", 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012197999999999999 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.373387+0000", 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055282539999999998 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484166+0000", 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.110778155 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484208+0000", 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2738000000000003e-05 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484245+0000", 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6556e-05 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5478 3.2b 3:d663163e:::100000001f8.00000763:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.0450944629999999, 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.059029182, 2024-09-19T05:44:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5478 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493886+0000", 2024-09-19T05:44:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2529999999999999e-06 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506674+0000", 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012787552000000001 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506696+0000", 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2259000000000002e-05 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506712+0000", 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5965e-05 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506793+0000", 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1075000000000006e-05 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506944+0000", 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151065 2024-09-19T05:44:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.507151+0000", 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020636600000000001 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.515901+0000", 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087503040000000004 2024-09-19T05:44:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552800+0000", 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0368988699999999 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552858+0000", 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7877000000000002e-05 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552914+0000", 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6595999999999998e-05 2024-09-19T05:44:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5491 3.2 3:418769c2:::100000001f8.00000770:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.032236084, 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.97763760899999996, 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5491 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:44:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506746+0000", 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8059999999999999e-06 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.520997+0000", 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014250966 2024-09-19T05:44:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521021+0000", 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.368e-05 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521038+0000", 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6708e-05 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521168+0000", 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013036300000000001 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521266+0000", 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7849000000000005e-05 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521457+0000", 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000191385 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.528522+0000", 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070645830000000001 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484294+0000", 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.95577166700000005 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484344+0000", 2024-09-19T05:44:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0512e-05 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484381+0000", 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7089999999999999e-05 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5506 3.2b 3:d506a813:::100000001f8.0000077f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:44:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.9999138629999997, 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.013993707, 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5506 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539068+0000", 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0710000000000002e-06 2024-09-19T05:44:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555200+0000", 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016132480000000001 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555230+0000", 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9728999999999999e-05 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555243+0000", 2024-09-19T05:44:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2697999999999999e-05 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555334+0000", 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1113999999999995e-05 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555436+0000", 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010182 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.556369+0000", 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00093375000000000001 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.562703+0000", 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063334139999999999 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552979+0000", 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.99027636699999999 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.638 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553029+0000", 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0214999999999999e-05 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553059+0000", 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0049e-05 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.639 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5530 3.2 3:43043671:::100000001f8.00000797:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.9279618950000001, 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.87348732500000004, 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5530 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.641 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611019+0000", 2024-09-19T05:44:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2619999999999999e-06 2024-09-19T05:44:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618028+0000", 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070088199999999998 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618048+0000", 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0616e-05 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618061+0000", 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2678e-05 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618127+0000", 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6276e-05 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618227+0000", 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9356000000000002e-05 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618379+0000", 2024-09-19T05:44:47.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000152151 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.623797+0000", 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054177050000000001 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484423+0000", 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.860626522 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484469+0000", 2024-09-19T05:44:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6242999999999998e-05 2024-09-19T05:44:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484505+0000", 2024-09-19T05:44:47.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5695999999999998e-05 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5671 3.2b 3:d7657d1c:::100000001f8.00000824:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.8322220140000001, 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.84642695999999995, 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.649 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5671 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:44:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706760+0000", 2024-09-19T05:44:47.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.243e-06 2024-09-19T05:44:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715117+0000", 2024-09-19T05:44:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083569019999999994 2024-09-19T05:44:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715133+0000", 2024-09-19T05:44:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5818e-05 2024-09-19T05:44:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715140+0000", 2024-09-19T05:44:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1849999999999998e-06 2024-09-19T05:44:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715202+0000", 2024-09-19T05:44:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2315000000000002e-05 2024-09-19T05:44:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715304+0000", 2024-09-19T05:44:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101786 2024-09-19T05:44:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715421+0000", 2024-09-19T05:44:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117333 2024-09-19T05:44:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.720417+0000", 2024-09-19T05:44:47.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049962840000000001 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553108+0000", 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.83269075100000001 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553150+0000", 2024-09-19T05:44:47.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2104000000000002e-05 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553185+0000", 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4239000000000002e-05 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5682 3.2 3:43bb8046:::100000001f8.0000082f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.659 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.8169701950000001, 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.90941017300000004, 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5682 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722012+0000", 2024-09-19T05:44:47.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1849999999999998e-06 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729368+0000", 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073561759999999999 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729391+0000", 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3309e-05 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729404+0000", 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3016e-05 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729479+0000", 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4914999999999997e-05 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729591+0000", 2024-09-19T05:44:47.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111607 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729717+0000", 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012588200000000001 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.735604+0000", 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058875100000000003 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.631310+0000", 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.89570617100000005 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.631368+0000", 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8213e-05 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.631420+0000", 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1189000000000001e-05 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5723 3.2 3:4214d263:::100000001f8.00000858:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.7719387019999999, 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.91036275700000002, 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.666 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5723 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767043+0000", 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.041e-06 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782398+0000", 2024-09-19T05:44:47.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015355056000000001 2024-09-19T05:44:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782422+0000", 2024-09-19T05:44:47.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4190999999999999e-05 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782441+0000", 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782517+0000", 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6186000000000005e-05 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782591+0000", 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3604999999999997e-05 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782725+0000", 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013430699999999999 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.786761+0000", 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035673 2024-09-19T05:44:47.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677290+0000", 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.89052959600000003 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677353+0000", 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2731000000000006e-05 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677404+0000", 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0680000000000003e-05 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5759 3.2 3:42bacaf4:::100000001f8.0000087c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.7171035310000002, 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.85564746599999997, 2024-09-19T05:44:47.673 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5759 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821879+0000", 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829742+0000", 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078630269999999999 2024-09-19T05:44:47.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829755+0000", 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3308e-05 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829763+0000", 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1839999999999999e-06 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829849+0000", 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6017e-05 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829955+0000", 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106059 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.830113+0000", 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015798799999999999 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834572+0000", 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004458755 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.678 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677448+0000", 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.84287563099999996 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677477+0000", 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9964999999999999e-05 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677524+0000", 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6075999999999997e-05 2024-09-19T05:44:47.679 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5776 3.2 3:42907a88:::100000001f8.0000088d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 3.6803701150000001, 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.81907758500000005, 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:47.680 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5776 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:47.681 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858612+0000", 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866396+0000", 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077837369999999998 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.682 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866413+0000", 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7481000000000001e-05 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866423+0000", 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9799999999999993e-06 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866567+0000", 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000143683 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.683 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866677+0000", 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110042 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866889+0000", 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021198500000000001 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873285+0000", 2024-09-19T05:44:47.684 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063963800000000001 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677576+0000", 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.80429050499999999 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677632+0000", 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6610999999999997e-05 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:47.685 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677687+0000", 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4725000000000002e-05 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:47.686 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:47.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:47.688 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:47.686+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:47.789 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:47.787+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:47.791 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:47.791 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:47.791 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:47.791 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:47.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:47.890 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:47.888+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:47.942 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:47.943 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:47.943 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:47.943 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:47.943 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:47.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:47.990 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:47.988+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:48.031 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.029+0000 7f5b3bfff640 1 -- 172.21.15.2:0/3201050604 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v76) ==== 101+0+0 (secure 0 0 0) 0x7f5b4805ccc0 con 0x7f5b54073890 2024-09-19T05:44:48.031 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:44:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5b4006c2e0 msgr2=0x7f5b4006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5b4006c2e0 0x7f5b4006e7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5b440011e0 tx=0x7f5b44009040 comp rx=0 tx=0).stop 2024-09-19T05:44:48.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b54073890 msgr2=0x7f5b5414d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b54073890 0x7f5b5414d3e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c3e0 tx=0x7f5b4800c8a0 comp rx=0 tx=0).stop 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 shutdown_connections 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5b4006c2e0 0x7f5b4006e7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b54073c70 0x7f5b540cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b54073890 0x7f5b5414d3e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 --2- 172.21.15.2:0/3201050604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b54072bd0 0x7f5b541ef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 >> 172.21.15.2:0/3201050604 conn(0x7f5b5406bc10 msgr2=0x7f5b5406f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 shutdown_connections 2024-09-19T05:44:48.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:48.032+0000 7f5b5a04c640 1 -- 172.21.15.2:0/3201050604 wait complete. 2024-09-19T05:44:48.059 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2720 3.12 3:4983891e:::100000001f8.0000069d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.796089+0000", 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.260672484000001, 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.139035335, 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.060 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2720 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.796089+0000", 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.061 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.796089+0000", 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.796092+0000", 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0419999999999998e-06 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807355+0000", 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011262872 2024-09-19T05:44:48.062 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807383+0000", 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7639000000000001e-05 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807397+0000", 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4405e-05 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.063 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807476+0000", 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8950999999999997e-05 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807739+0000", 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026290900000000003 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.807991+0000", 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000251666 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.064 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.817503+0000", 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095125000000000001 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.934855+0000", 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1173514680000001 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.934915+0000", 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9877999999999998e-05 2024-09-19T05:44:48.065 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935125+0000", 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021000500000000001 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2727 3.12 3:4bacb89b:::100000001f8.000006a4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.818067+0000", 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.238694759000001, 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.117480729, 2024-09-19T05:44:48.066 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2727 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.818067+0000", 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.067 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.818067+0000", 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.818069+0000", 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.345e-06 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829667+0000", 2024-09-19T05:44:48.068 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011597896 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829711+0000", 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4060999999999997e-05 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829725+0000", 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4073999999999999e-05 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.069 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.829832+0000", 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106382 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.830005+0000", 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000173265 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.830175+0000", 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016997299999999999 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.070 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.839760+0000", 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095851990000000008 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935410+0000", 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.095649686 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935473+0000", 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3139999999999995e-05 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.071 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.935548+0000", 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4708000000000004e-05 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2809 3.12 3:482c9566:::100000001f8.000006f6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:31.870050+0000", 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.186711675000002, 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.335518507, 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.072 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2809 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.870050+0000", 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.073 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.870050+0000", 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.870053+0000", 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.909e-06 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882108+0000", 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012055003 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.074 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882128+0000", 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9837999999999999e-05 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882138+0000", 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6169999999999996e-06 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882209+0000", 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1849999999999998e-05 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.075 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882310+0000", 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100436 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.882406+0000", 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6428999999999994e-05 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:31.888085+0000", 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0056786060000000001 2024-09-19T05:44:48.076 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.205494+0000", 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.317409576 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.205536+0000", 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1519e-05 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.205569+0000", 2024-09-19T05:44:48.077 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2724e-05 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:2952 3.12 3:49429726:::100000001f8.00000785:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.032287+0000", 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.024475141, 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.381939864, 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.078 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 2952 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.032287+0000", 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.032287+0000", 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.079 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.032291+0000", 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.561e-06 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042630+0000", 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010338905000000001 2024-09-19T05:44:48.080 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042651+0000", 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0492999999999998e-05 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042660+0000", 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6700000000000006e-06 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.081 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042728+0000", 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7884000000000003e-05 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042871+0000", 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000142972 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.082 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.043038+0000", 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016656099999999999 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.048005+0000", 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049671289999999998 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.083 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414133+0000", 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.366128263 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414176+0000", 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2509000000000002e-05 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414227+0000", 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0917000000000001e-05 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3007 3.12 3:48356219:::100000001f8.000007bc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.158415+0000", 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.898346987, 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2559814359999999, 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3007 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.158415+0000", 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.158415+0000", 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.158417+0000", 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8589999999999999e-06 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173428+0000", 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015011648000000001 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173459+0000", 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1161999999999997e-05 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173475+0000", 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5998000000000002e-05 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173571+0000", 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5579000000000001e-05 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.173841+0000", 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00026997300000000001 2024-09-19T05:44:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.174125+0000", 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00028421099999999998 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.180126+0000", 2024-09-19T05:44:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060009809999999999 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414311+0000", 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2341851130000001 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414370+0000", 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8899e-05 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.414396+0000", 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6013e-05 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3095 3.33 3:cd66a262:::100000001f8.00000814:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.294970+0000", 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.761791881000001, 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.82468124399999998, 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3095 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.294970+0000", 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.294970+0000", 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.294973+0000", 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9069999999999999e-06 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306331+0000", 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011357848 2024-09-19T05:44:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306348+0000", 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6841e-05 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306358+0000", 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0081000000000001e-05 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306424+0000", 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6806999999999999e-05 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306543+0000", 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000118312 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 0", 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.306642+0000", 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8808000000000007e-05 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.312752+0000", 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061104210000000004 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.119572+0000", 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.80681970000000003 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.119615+0000", 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3238999999999997e-05 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.096 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.119651+0000", 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6279999999999998e-05 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3101 3.12 3:4aca477d:::100000001f8.0000081a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.319810+0000", 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.736951623, 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1290244970000001, 2024-09-19T05:44:48.097 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3101 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.319810+0000", 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.098 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.319810+0000", 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.319812+0000", 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8679999999999999e-06 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331254+0000", 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01144163 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.099 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331271+0000", 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7013000000000001e-05 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331281+0000", 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0412e-05 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331356+0000", 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5127000000000001e-05 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.100 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331506+0000", 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014979899999999999 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.331664+0000", 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015781399999999999 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.336192+0000", 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045283349999999997 2024-09-19T05:44:48.101 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.448735+0000", 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.112542908 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.448779+0000", 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3634000000000003e-05 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.448835+0000", 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5957e-05 2024-09-19T05:44:48.102 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3106 3.12 3:48a69cf9:::100000001f8.0000081f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.360705+0000", 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.696057201, 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.119816538, 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.103 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3106 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.360705+0000", 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.360705+0000", 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.104 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.360707+0000", 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1950000000000002e-06 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374370+0000", 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013662818 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374383+0000", 2024-09-19T05:44:48.105 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3203999999999999e-05 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374388+0000", 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5949999999999998e-06 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374454+0000", 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.5779999999999997e-05 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374602+0000", 2024-09-19T05:44:48.106 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014766599999999999 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.374725+0000", 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012355600000000001 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.381270+0000", 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065449549999999999 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480379+0000", 2024-09-19T05:44:48.107 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0991086860000001 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480447+0000", 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7473999999999998e-05 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480521+0000", 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4609000000000003e-05 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.108 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3179 3.12 3:4a7c9af2:::100000001f8.00000868:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.417716+0000", 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.639046147, 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.062994529, 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3179 2024-09-19T05:44:48.109 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.417716+0000", 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.417716+0000", 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.417718+0000", 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6759999999999999e-06 2024-09-19T05:44:48.110 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435215+0000", 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017496912999999999 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435233+0000", 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7737e-05 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435243+0000", 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0045e-05 2024-09-19T05:44:48.111 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435319+0000", 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6029e-05 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435454+0000", 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013470299999999999 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.435589+0000", 2024-09-19T05:44:48.112 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013483399999999999 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.440522+0000", 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049332739999999996 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480598+0000", 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.040076196 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.113 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480651+0000", 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3066000000000003e-05 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480710+0000", 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9055999999999998e-05 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3267 3.12 3:49d1bd5a:::100000001f8.000008c0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.539767+0000", 2024-09-19T05:44:48.114 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.516995261, 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.94104224199999997, 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3267 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.539767+0000", 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.115 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.539767+0000", 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.539768+0000", 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9410000000000001e-06 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.550897+0000", 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011128403 2024-09-19T05:44:48.116 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.550920+0000", 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.338e-05 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.550937+0000", 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7108999999999998e-05 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.117 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.551008+0000", 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0506999999999994e-05 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.551173+0000", 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016462399999999999 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.551322+0000", 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000149498 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.118 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.555948+0000", 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046262040000000001 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480757+0000", 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.92480870999999998 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480785+0000", 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8220000000000001e-05 2024-09-19T05:44:48.119 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.480809+0000", 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3646e-05 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3315 3.12 3:4962d06a:::100000001f8.000008f0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.611024+0000", 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.445738005999999, 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.90393573900000002, 2024-09-19T05:44:48.120 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3315 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.611024+0000", 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.121 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.611024+0000", 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.611027+0000", 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5030000000000002e-06 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618757+0000", 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077300169999999996 2024-09-19T05:44:48.122 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618773+0000", 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5778999999999999e-05 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618782+0000", 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.098e-06 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.618889+0000", 2024-09-19T05:44:48.123 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106937 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.619059+0000", 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017009800000000001 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.619224+0000", 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016449600000000001 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.124 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.625571+0000", 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006347559 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.514837+0000", 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.88926545599999995 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.514899+0000", 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2614999999999996e-05 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.125 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.514960+0000", 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0180999999999999e-05 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3443 3.12 3:4be71d45:::100000001f8.00000970:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.749886+0000", 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.306875432, 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.84167001100000005, 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.126 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3443 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.749886+0000", 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.127 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.749886+0000", 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.749890+0000", 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0060000000000003e-06 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756769+0000", 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068785080000000002 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.128 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756788+0000", 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9143999999999998e-05 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756799+0000", 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0858e-05 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756862+0000", 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3314999999999999e-05 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.129 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.756999+0000", 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013695999999999999 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.757098+0000", 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.8987000000000006e-05 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.761307+0000", 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042089789999999998 2024-09-19T05:44:48.130 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591482+0000", 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.83017514999999997 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591522+0000", 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9715999999999997e-05 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591556+0000", 2024-09-19T05:44:48.131 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4387999999999997e-05 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3455 3.12 3:493add79:::100000001f8.0000097c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.786731+0000", 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 15.270031143000001, 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.804915995, 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.132 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3455 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.786731+0000", 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.786731+0000", 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.133 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.786733+0000", 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3999999999999999e-06 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.134 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828604+0000", 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.041870624000000002 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828630+0000", 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6795999999999999e-05 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828642+0000", 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1823e-05 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.135 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828736+0000", 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.3364999999999998e-05 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828856+0000", 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012047100000000001 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.828979+0000", 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012258599999999999 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.136 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.834525+0000", 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055461620000000003 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591590+0000", 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.75706481599999997 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591622+0000", 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2410999999999999e-05 2024-09-19T05:44:48.137 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:33.591647+0000", 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4541000000000001e-05 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5041 3.12 3:48dc6f9d:::100000001f8.000005ae:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.480594+0000", 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5761676720000004, 2024-09-19T05:44:48.138 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2714557950000001, 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5041 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.480594+0000", 2024-09-19T05:44:48.139 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.480594+0000", 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.480597+0000", 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8650000000000001e-06 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.140 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.489132+0000", 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0085352770000000008 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.489151+0000", 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8414e-05 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.489161+0000", 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0379e-05 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.141 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.489241+0000", 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.9702000000000004e-05 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.489330+0000", 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9526999999999999e-05 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.489437+0000", 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106816 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.142 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.494207+0000", 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004769667 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.751939+0000", 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.257732171 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.752001+0000", 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2232999999999999e-05 2024-09-19T05:44:48.143 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.752050+0000", 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8744e-05 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5107 3.12 3:4b33a876:::100000001f8.000005f0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.548381+0000", 2024-09-19T05:44:48.144 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5083805840000002, 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2038084659999999, 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5107 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.145 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548381+0000", 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548381+0000", 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548385+0000", 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.281e-06 2024-09-19T05:44:48.146 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.555892+0000", 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075070390000000001 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.555912+0000", 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0492999999999998e-05 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.147 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.555929+0000", 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6894000000000001e-05 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.556057+0000", 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000127916 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.556145+0000", 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.8406999999999996e-05 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.148 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.556289+0000", 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014413699999999999 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.561072+0000", 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0047821349999999999 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.752108+0000", 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1910367609999999 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.149 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.752154+0000", 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6047000000000001e-05 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.752190+0000", 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5355999999999998e-05 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.150 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5143 3.12 3:494b1864:::100000001f8.00000614:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.828837+0000", 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2279246170000002, 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1164237619999999, 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5143 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.151 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.828837+0000", 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.828837+0000", 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.828839+0000", 2024-09-19T05:44:48.152 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.455e-06 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.843802+0000", 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014963175 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.843842+0000", 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0096999999999999e-05 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.153 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.843855+0000", 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2611e-05 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.843968+0000", 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000113942 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.844046+0000", 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7065999999999997e-05 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.154 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.844162+0000", 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000116078 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848560+0000", 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043985109999999999 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.945146+0000", 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0965855499999999 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.155 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.945207+0000", 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1074000000000005e-05 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.945261+0000", 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4203000000000003e-05 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5203 3.12 3:4bb8df49:::100000001f8.00000650:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.156 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.973268+0000", 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.0834939459999999, 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.97211812900000005, 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5203 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.157 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.973268+0000", 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.973268+0000", 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.973271+0000", 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7999999999999999e-06 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.158 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.989370+0000", 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016098971 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.989396+0000", 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6395000000000001e-05 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.989410+0000", 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3766e-05 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.159 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.989493+0000", 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3018999999999999e-05 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.989605+0000", 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000112421 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.989783+0000", 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017736499999999999 2024-09-19T05:44:48.160 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.995798+0000", 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0060155920000000002 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.945308+0000", 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.94950994499999997 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.945352+0000", 2024-09-19T05:44:48.161 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3464999999999998e-05 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.945386+0000", 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4390000000000001e-05 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5280 3.12 3:4983891e:::100000001f8.0000069d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.062977+0000", 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.9937843280000003, 2024-09-19T05:44:48.162 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.087348306, 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5280 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.062977+0000", 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.163 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.062977+0000", 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.062980+0000", 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.368e-06 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.164 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.075836+0000", 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012856340000000001 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.075857+0000", 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0373999999999998e-05 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.075872+0000", 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5024e-05 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.165 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.075950+0000", 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8597999999999995e-05 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.076022+0000", 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1724000000000006e-05 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.076130+0000", 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107736 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.166 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.080992+0000", 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004862636 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.150212+0000", 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0692201560000001 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.150272+0000", 2024-09-19T05:44:48.167 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.9091000000000001e-05 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.150326+0000", 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4258999999999997e-05 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5287 3.12 3:4bacb89b:::100000001f8.000006a4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.088993+0000", 2024-09-19T05:44:48.168 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.9677690329999997, 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.061515596, 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5287 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.088993+0000", 2024-09-19T05:44:48.169 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.088993+0000", 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.088995+0000", 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.148e-06 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.170 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102601+0000", 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013605745000000001 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102620+0000", 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9023999999999999e-05 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102636+0000", 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.63e-05 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.171 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102723+0000", 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6812999999999997e-05 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102847+0000", 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012434100000000001 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.103063+0000", 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021600300000000001 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.172 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.108532+0000", 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054691460000000003 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.150438+0000", 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0419058560000001 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.150480+0000", 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1485e-05 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.173 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.150508+0000", 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8734999999999998e-05 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5369 3.12 3:482c9566:::100000001f8.000006f6:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.141223+0000", 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 4.9155387670000001, 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4676537170000001, 2024-09-19T05:44:48.174 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5369 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.141223+0000", 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.175 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.141223+0000", 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.141226+0000", 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6390000000000001e-06 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.159559+0000", 2024-09-19T05:44:48.176 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.018333391000000001 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.159583+0000", 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4388000000000001e-05 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.159592+0000", 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2959999999999999e-06 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.159666+0000", 2024-09-19T05:44:48.177 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4034000000000003e-05 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.159780+0000", 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000114583 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 3", 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.159958+0000", 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000177481 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.164312+0000", 2024-09-19T05:44:48.178 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043544839999999996 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.608802+0000", 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.444490112 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.608839+0000", 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6745000000000001e-05 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.608877+0000", 2024-09-19T05:44:48.179 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7564000000000002e-05 2024-09-19T05:44:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.180 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.180 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:48.182 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:48.182 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:48.179+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:48.283 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:48.281+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:48.285 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:48.286 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:48.286 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:48.286 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:48.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:48.383 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:48.382+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:48.438 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:48.438 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:48.438 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:44:48.438 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:44:48.438 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:48.446 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:48.491 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:48.490+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:48.552 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3040 3.2e 3:75930891:::100000001f8.000007dd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.031041+0000", 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.518535798999999, 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.67679123200000002, 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.553 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3040 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.031041+0000", 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.031041+0000", 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.554 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.031043+0000", 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.685e-06 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042438+0000", 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011395285 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042462+0000", 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4326000000000001e-05 2024-09-19T05:44:48.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042476+0000", 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3495999999999999e-05 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042598+0000", 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012265799999999999 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.556 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042770+0000", 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017122099999999999 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.042949+0000", 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017983699999999999 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.047408+0000", 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0044583139999999997 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.707707+0000", 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.66029938899999996 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.707769+0000", 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2181000000000007e-05 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.707832+0000", 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2840000000000001e-05 2024-09-19T05:44:48.558 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3131 3.2e 3:772593bb:::100000001f8.00000838:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.097068+0000", 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.452508713, 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.69790961600000001, 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.559 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3131 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.097068+0000", 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.097068+0000", 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.560 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.097070+0000", 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8300000000000001e-06 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.107819+0000", 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010748824000000001 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.107840+0000", 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1644000000000001e-05 2024-09-19T05:44:48.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.107850+0000", 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9629999999999993e-06 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.107926+0000", 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5506999999999994e-05 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.108166+0000", 2024-09-19T05:44:48.562 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00024079599999999999 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.108341+0000", 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000174372 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.111914+0000", 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0035732170000000001 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.563 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.794854+0000", 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.68294024099999995 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.794917+0000", 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2829000000000004e-05 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.794978+0000", 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.0393000000000003e-05 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.564 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3135 3.2e 3:77628c4e:::100000001f8.0000083c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.117479+0000", 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.432097376000002, 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.70689776400000004, 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3135 2024-09-19T05:44:48.565 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.117479+0000", 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.117479+0000", 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.566 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.117482+0000", 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3860000000000001e-06 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.128404+0000", 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.01092248 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.128426+0000", 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2246999999999999e-05 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.567 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.128440+0000", 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3451999999999999e-05 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.128524+0000", 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4241999999999997e-05 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.128697+0000", 2024-09-19T05:44:48.568 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017251500000000001 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.128819+0000", 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000122404 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.133137+0000", 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043176669999999999 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824268+0000", 2024-09-19T05:44:48.569 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.69113145799999998 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824325+0000", 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6461e-05 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824377+0000", 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.2451999999999999e-05 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.570 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3168 3.2e 3:77f75ed6:::100000001f8.0000085d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.137711+0000", 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.411865454000001, 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.68686413000000002, 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3168 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.571 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.137711+0000", 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.137711+0000", 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.137714+0000", 2024-09-19T05:44:48.572 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3649999999999998e-06 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.150341+0000", 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012627002 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.150359+0000", 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8539999999999999e-05 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.150367+0000", 2024-09-19T05:44:48.573 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1419999999999993e-06 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.150436+0000", 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9097999999999995e-05 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.150579+0000", 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014286600000000001 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.574 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.150711+0000", 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000131913 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.154338+0000", 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0036269420000000002 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824459+0000", 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.67012138499999996 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.575 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824530+0000", 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0083e-05 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824575+0000", 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.5794000000000002e-05 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:3178 3.2e 3:757c9716:::100000001f8.00000867:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e71)", 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:32.160060+0000", 2024-09-19T05:44:48.576 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 16.389516883999999, 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.66468527099999997, 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 3178 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.577 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.160060+0000", 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.160060+0000", 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.160062+0000", 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5610000000000001e-06 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.578 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169564+0000", 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0095012059999999999 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169584+0000", 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0899000000000001e-05 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169596+0000", 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1732e-05 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.579 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169667+0000", 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1032999999999996e-05 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169772+0000", 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010457400000000001 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.169873+0000", 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101431 2024-09-19T05:44:48.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.174044+0000", 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041710530000000001 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824641+0000", 2024-09-19T05:44:48.581 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.65059662399999996 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824715+0000", 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3836000000000003e-05 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:32.824745+0000", 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0321999999999999e-05 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5048 3.2e 3:740912a7:::100000001f8.000005b5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.472949+0000", 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.0766280610000001, 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.88792483799999999, 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5048 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.583 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.472949+0000", 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.472949+0000", 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.584 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.472952+0000", 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.467e-06 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.485091+0000", 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012139308999999999 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.485108+0000", 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6521e-05 2024-09-19T05:44:48.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.485119+0000", 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1408e-05 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.485600+0000", 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00048029300000000002 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.586 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.485751+0000", 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015176799999999999 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.485935+0000", 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018343299999999999 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.490485+0000", 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0045497319999999999 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.587 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.360796+0000", 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.87031147600000003 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.360840+0000", 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3577e-05 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.360873+0000", 2024-09-19T05:44:48.588 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.3853999999999998e-05 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5071 3.2e 3:75b854b3:::100000001f8.000005cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.509209+0000", 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.0403675139999997, 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.85193618199999999, 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.589 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5071 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.509209+0000", 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.509209+0000", 2024-09-19T05:44:48.590 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.509211+0000", 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5340000000000001e-06 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.521816+0000", 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012605313 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.591 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.521880+0000", 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.4509000000000002e-05 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.521904+0000", 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3303e-05 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.522006+0000", 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101921 2024-09-19T05:44:48.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.522143+0000", 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013756599999999999 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.522302+0000", 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000158548 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.528528+0000", 2024-09-19T05:44:48.593 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062261210000000003 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.360972+0000", 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.832443669 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.361079+0000", 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107093 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.594 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.361145+0000", 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6605000000000004e-05 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5080 3.2e 3:75f1801c:::100000001f8.000005d5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.535518+0000", 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 6.014058908, 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.82572768900000004, 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.595 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5080 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.535518+0000", 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.596 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.535518+0000", 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.535519+0000", 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.547e-06 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548540+0000", 2024-09-19T05:44:48.597 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013021209000000001 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548559+0000", 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8198000000000001e-05 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548567+0000", 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1559999999999995e-06 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.598 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548657+0000", 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9758000000000004e-05 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548736+0000", 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8985000000000004e-05 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.548895+0000", 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015929900000000001 2024-09-19T05:44:48.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.554629+0000", 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057342699999999996 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.361196+0000", 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.806566483 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.600 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.361223+0000", 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7702999999999999e-05 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.361245+0000", 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2081000000000001e-05 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5142 3.2e 3:775a61bf:::100000001f8.00000613:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.601 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.787469+0000", 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7621078509999997, 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1250313249999999, 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5142 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.602 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.787469+0000", 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.787469+0000", 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.787470+0000", 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7129999999999999e-06 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.603 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.802681+0000", 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015210855000000001 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.802695+0000", 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3582e-05 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.802702+0000", 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9380000000000003e-06 2024-09-19T05:44:48.604 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.802751+0000", 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8650999999999999e-05 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.802832+0000", 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1868999999999998e-05 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.605 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.802965+0000", 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013226700000000001 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.807864+0000", 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0048989389999999997 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.912383+0000", 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1045190899999999 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.606 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.912444+0000", 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1531000000000004e-05 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.912500+0000", 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5890000000000002e-05 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5193 3.2e 3:76954c23:::100000001f8.00000646:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.828560+0000", 2024-09-19T05:44:48.607 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.7210167690000002, 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2650579820000001, 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5193 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.828560+0000", 2024-09-19T05:44:48.608 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.828560+0000", 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.828562+0000", 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2800000000000002e-06 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.609 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.837386+0000", 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0088238369999999993 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.837400+0000", 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3915e-05 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.837409+0000", 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9840000000000007e-06 2024-09-19T05:44:48.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.837547+0000", 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000138336 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.837643+0000", 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6220999999999999e-05 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.837755+0000", 2024-09-19T05:44:48.611 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111765 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.841860+0000", 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041047979999999998 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.093498+0000", 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2516377300000001 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.612 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.093560+0000", 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2463000000000001e-05 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.093618+0000", 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7652999999999998e-05 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5288 3.2e 3:77178000:::100000001f8.000006a5:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.613 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.891416+0000", 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.6581607849999997, 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.488215147, 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5288 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.614 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.891416+0000", 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.891416+0000", 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.891419+0000", 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.762e-06 2024-09-19T05:44:48.615 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.901829+0000", 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010410018 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.901846+0000", 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6915e-05 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.901854+0000", 2024-09-19T05:44:48.616 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6370000000000005e-06 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.901945+0000", 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.0506000000000004e-05 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.902050+0000", 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104999 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.617 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.902210+0000", 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000160772 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.908163+0000", 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00595299 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.379559+0000", 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4713958979999999 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.618 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.379598+0000", 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8946000000000003e-05 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.379631+0000", 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.2703999999999997e-05 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5394 3.2e 3:740b0c25:::100000001f8.0000070f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.619 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.975283+0000", 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5742933460000001, 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4044368819999999, 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5394 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.620 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.975283+0000", 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.975283+0000", 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.975286+0000", 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6359999999999998e-06 2024-09-19T05:44:48.621 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984652+0000", 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0093664030000000006 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984672+0000", 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9958000000000001e-05 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.622 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984681+0000", 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.585e-06 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984763+0000", 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.2191e-05 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984854+0000", 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1144999999999996e-05 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.623 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.985038+0000", 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00018393199999999999 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.991366+0000", 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063278680000000004 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.379669+0000", 2024-09-19T05:44:48.624 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3883032470000001 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.379699+0000", 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0164000000000001e-05 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.379720+0000", 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0752999999999999e-05 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.625 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5402 3.2e 3:74df89ff:::100000001f8.00000717:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.984710+0000", 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.5648670149999999, 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6491615639999999, 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.626 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5402 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984710+0000", 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984710+0000", 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.627 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.984712+0000", 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.379e-06 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.994696+0000", 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0099838949999999996 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.994713+0000", 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7496999999999999e-05 2024-09-19T05:44:48.628 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.994726+0000", 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2315e-05 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.994798+0000", 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2164999999999998e-05 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.994907+0000", 2024-09-19T05:44:48.629 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010885799999999999 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.995071+0000", 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000164425 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.999338+0000", 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0042664479999999999 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.630 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.633798+0000", 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.634460421 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.633840+0000", 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1718000000000003e-05 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.633871+0000", 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1442999999999997e-05 2024-09-19T05:44:48.631 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5498 3.2e 3:74b9618b:::100000001f8.00000777:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.102874+0000", 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.4467031109999997, 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.659357513, 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.632 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5498 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102874+0000", 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102874+0000", 2024-09-19T05:44:48.633 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.102875+0000", 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.73e-06 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.109833+0000", 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069575219999999998 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.109845+0000", 2024-09-19T05:44:48.634 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2602000000000001e-05 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.109854+0000", 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3480000000000004e-06 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.109931+0000", 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7707000000000007e-05 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.635 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.110036+0000", 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104551 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.110211+0000", 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00017488599999999999 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.117733+0000", 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0075221790000000004 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.636 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.762163+0000", 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.644430289 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.762200+0000", 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7039999999999998e-05 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.762231+0000", 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0658999999999999e-05 2024-09-19T05:44:48.637 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.640 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.640 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.640 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.640 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5499 3.2e 3:74e8fe5d:::100000001f8.00000778:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.109881+0000", 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.4396960029999999, 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.652429285, 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.641 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5499 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.109881+0000", 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.109881+0000", 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.642 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.109883+0000", 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1610000000000001e-06 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.118622+0000", 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087394199999999995 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.643 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.118635+0000", 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2656999999999999e-05 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.118642+0000", 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.2169999999999997e-06 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.118706+0000", 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3728999999999999e-05 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.644 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.118813+0000", 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000107459 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.118973+0000", 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00016021500000000001 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.124941+0000", 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0059670160000000003 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.645 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.762264+0000", 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.637323855 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.762290+0000", 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5485000000000001e-05 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.762310+0000", 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0071000000000001e-05 2024-09-19T05:44:48.646 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5570 3.2e 3:7418dfc9:::100000001f8.000007bf:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.170749+0000", 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3788271500000002, 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.621079599, 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.647 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5570 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.170749+0000", 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.170749+0000", 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.648 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.170751+0000", 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.435e-06 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.177980+0000", 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0072286310000000001 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.177996+0000", 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6911000000000001e-05 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.178005+0000", 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.602e-06 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.178077+0000", 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1518000000000002e-05 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.178200+0000", 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012369700000000001 2024-09-19T05:44:48.650 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.178353+0000", 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000152471 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.184930+0000", 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065774600000000003 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.791748+0000", 2024-09-19T05:44:48.651 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.606817441 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.791792+0000", 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.4567999999999997e-05 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.791829+0000", 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6865e-05 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.652 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5600 3.2e 3:75930891:::100000001f8.000007dd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.200443+0000", 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.3491335720000004, 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.607065781, 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5600 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.653 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.200443+0000", 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.200443+0000", 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.200446+0000", 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4789999999999999e-06 2024-09-19T05:44:48.654 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.208063+0000", 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0076172109999999996 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.208077+0000", 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4106e-05 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.208087+0000", 2024-09-19T05:44:48.655 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0455000000000001e-05 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.208216+0000", 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012898299999999999 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.208285+0000", 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8223000000000001e-05 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.656 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.208406+0000", 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000121673 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.212782+0000", 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0043756969999999996 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.807446+0000", 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.594664104 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.657 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.807481+0000", 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5219999999999998e-05 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.807509+0000", 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7630000000000001e-05 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5691 3.2e 3:772593bb:::100000001f8.00000838:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.250111+0000", 2024-09-19T05:44:48.658 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2994654360000002, 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5715109539999998, 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5691 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.659 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.250111+0000", 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.250111+0000", 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.660 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.250114+0000", 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.429e-06 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.257002+0000", 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0068882170000000003 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.661 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.257014+0000", 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1843e-05 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.257021+0000", 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8800000000000002e-06 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.257101+0000", 2024-09-19T05:44:48.662 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.0055999999999994e-05 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.257168+0000", 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.7323999999999994e-05 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.257277+0000", 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0001091 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.663 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.261391+0000", 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0041136810000000001 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821454+0000", 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.560063025 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821544+0000", 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9838000000000003e-05 2024-09-19T05:44:48.664 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821622+0000", 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.8560999999999995e-05 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5695 3.2e 3:77628c4e:::100000001f8.0000083c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.265173+0000", 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2844032309999998, 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5565358730000001, 2024-09-19T05:44:48.665 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5695 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.265173+0000", 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.666 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.265173+0000", 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.265175+0000", 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9819999999999998e-06 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.275707+0000", 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.010531933 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.667 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.275730+0000", 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3088999999999999e-05 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.275741+0000", 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0920999999999999e-05 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.275843+0000", 2024-09-19T05:44:48.668 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101704 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.275912+0000", 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9282000000000005e-05 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.276016+0000", 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000103915 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.669 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.281089+0000", 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0050731989999999996 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821656+0000", 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5405667809999999 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821687+0000", 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.1256e-05 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.670 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821709+0000", 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1810999999999998e-05 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5728 3.2e 3:77f75ed6:::100000001f8.0000085d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.284626+0000", 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 5.2649506150000001, 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5372444920000001, 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:44:48.671 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5728 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.284626+0000", 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.672 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.284626+0000", 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.284628+0000", 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3329999999999999e-06 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:44:48.673 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.292128+0000", 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0074993409999999996 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.292143+0000", 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5590000000000002e-05 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.292166+0000", 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3090000000000001e-05 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.292230+0000", 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.3279000000000002e-05 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.292305+0000", 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.5119999999999999e-05 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.292438+0000", 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013316599999999999 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.675 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.301222+0000", 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087840850000000005 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821826+0000", 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5206038930000001 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821851+0000", 2024-09-19T05:44:48.676 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5408000000000001e-05 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.821871+0000", 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9187000000000001e-05 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:44:48.677 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:48.679 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:48.679 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:48.677+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:48.779 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:48.777+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:48.800 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:48.800 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:48.800 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:48.800 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:48.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:48.895 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:48.892+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:48.965 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:48.965 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:48.966 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:48.966 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:48.966 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:48.976 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:49.018 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:49.016+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:49.082 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:49.080+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2637 3.3 3:c38cdf66:::100000001f8.0000064a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.306987711000001, 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.86377266600000002, 2024-09-19T05:44:49.140 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2637 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829833+0000", 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4700000000000004e-06 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834195+0000", 2024-09-19T05:44:49.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043615379999999999 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834217+0000", 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1713e-05 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834229+0000", 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2418e-05 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.139987+0000", 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.305758321 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140183+0000", 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195577 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:49.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140385+0000", 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020212799999999999 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.465853+0000", 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32546820700000001 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693468+0000", 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22761451199999999 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693549+0000", 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1267999999999995e-05 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693602+0000", 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2513999999999999e-05 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2681 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:49.146 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.154496976000001, 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.077969779, 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2681 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982323+0000", 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2370000000000001e-06 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986715+0000", 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004392618 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986751+0000", 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5398999999999998e-05 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986762+0000", 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1545999999999999e-05 2024-09-19T05:44:49.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706073+0000", 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71931063299999998 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706251+0000", 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017771099999999999 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060111+0000", 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35386035900000001 2024-09-19T05:44:49.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060200+0000", 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9259999999999996e-05 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060289+0000", 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9016000000000003e-05 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:3095 3.33 e71/20 3:cd66a262:::100000001f8.00000814:head v 71'69, mlcod=70'60)", 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.825680025, 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80810778000000005, 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311141+0000", 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.284e-06 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318008+0000", 2024-09-19T05:44:49.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068671770000000004 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318032+0000", 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4467000000000001e-05 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318051+0000", 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8284e-05 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693668+0000", 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37561766600000002 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693691+0000", 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2875e-05 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116967+0000", 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42327609900000002 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.119244+0000", 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022769280000000001 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5152 3.2c 3:350c5391:::100000001f8.0000061d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.2837893600000001, 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80414443700000005, 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5152 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853033+0000", 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6400000000000002e-06 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856724+0000", 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036910649999999999 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856747+0000", 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2719999999999999e-05 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856759+0000", 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2679e-05 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948007+0000", 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091247606999999994 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948133+0000", 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012556500000000001 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948378+0000", 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024540299999999999 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.240715+0000", 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29233733899999997 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657035+0000", 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41632016100000002 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657101+0000", 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691999999999994e-05 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657172+0000", 2024-09-19T05:44:49.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0566000000000002e-05 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5153 3.2c 3:35ced6af:::100000001f8.0000061e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.2800274619999996, 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84790699300000005, 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.162 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5153 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856791+0000", 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2239999999999998e-06 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859552+0000", 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027601930000000002 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859564+0000", 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2880999999999999e-05 2024-09-19T05:44:49.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859579+0000", 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4925e-05 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145653+0000", 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.286073313 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145820+0000", 2024-09-19T05:44:49.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166937 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.167125+0000", 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021305001 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.270722+0000", 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103597041 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704576+0000", 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43385462400000002 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704637+0000", 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1118000000000007e-05 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704696+0000", 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8736000000000001e-05 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5229 3.34 3:2ebd7ea6:::100000001f8.0000066a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.2041778379999997, 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90816846299999998, 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5229 2024-09-19T05:44:49.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932642+0000", 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5820000000000002e-06 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937356+0000", 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047141400000000003 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937389+0000", 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2929999999999998e-05 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937402+0000", 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2855e-05 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268202+0000", 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.330799856 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268403+0000", 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200464 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268639+0000", 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000236598 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.656059+0000", 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.387419719 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840676+0000", 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18461746900000001 2024-09-19T05:44:49.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840747+0000", 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0462000000000004e-05 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840807+0000", 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0387999999999998e-05 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.173 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5231 3.3b 3:ddf1e7a4:::100000001f8.0000066c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0825547249999996, 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1076528050000001, 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5231 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054273+0000", 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1253999999999999e-05 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059647+0000", 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053742479999999999 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059676+0000", 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8699000000000001e-05 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059689+0000", 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2476999999999999e-05 2024-09-19T05:44:49.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.863904+0000", 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.804215652 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.864024+0000", 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119848 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161715+0000", 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.297691133 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161795+0000", 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9671000000000003e-05 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161915+0000", 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011982299999999999 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.178 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5241 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0377297690000002, 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.063036227, 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5241 2024-09-19T05:44:49.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099090+0000", 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105109+0000", 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006018809 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105132+0000", 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2908000000000002e-05 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105143+0000", 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1826e-05 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.924904+0000", 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81976026999999996 2024-09-19T05:44:49.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925014+0000", 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011063 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162023+0000", 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23700855300000001 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162060+0000", 2024-09-19T05:44:49.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6715999999999999e-05 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162123+0000", 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3480999999999997e-05 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5250 3.3 3:c0cd83f8:::100000001f8.0000067f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.0017934420000003, 2024-09-19T05:44:49.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026991652, 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5250 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135025+0000", 2024-09-19T05:44:49.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3379999999999999e-06 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137223+0000", 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021978639999999999 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137247+0000", 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3978999999999999e-05 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.144008+0000", 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067608770000000002 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915821+0000", 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77181293399999995 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915914+0000", 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2493999999999999e-05 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.916430+0000", 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00051667599999999998 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.122065+0000", 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20563482999999999 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161930+0000", 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039865356999999997 2024-09-19T05:44:49.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161977+0000", 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6873999999999999e-05 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162015+0000", 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7428999999999997e-05 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5293 3.3 3:c2b9c98f:::100000001f8.000006aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.925159861, 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95068157399999997, 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5293 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211659+0000", 2024-09-19T05:44:49.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215320+0000", 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003660862 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215339+0000", 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8868999999999998e-05 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215351+0000", 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1802e-05 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939166+0000", 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72381503599999997 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939258+0000", 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1911999999999997e-05 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939406+0000", 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014874699999999999 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162132+0000", 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.222725545 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162260+0000", 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012829700000000001 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162304+0000", 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4215000000000003e-05 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162338+0000", 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3819000000000003e-05 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.196 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5301 3.c 3:3307abed:::100000001f8.000006b2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9161740829999996, 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75017270800000002, 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5301 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220645+0000", 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0509999999999999e-06 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238885+0000", 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018240014999999998 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238906+0000", 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1498999999999999e-05 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238919+0000", 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2565000000000001e-05 2024-09-19T05:44:49.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841065+0000", 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60214654400000001 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841146+0000", 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0688999999999999e-05 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841342+0000", 2024-09-19T05:44:49.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019650000000000001 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878211+0000", 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036868724999999998 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970700+0000", 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092489022000000004 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970754+0000", 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3480999999999998e-05 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970815+0000", 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1617000000000003e-05 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5312 3.2c 3:36326779:::100000001f8.000006bd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:49.202 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8781668040000001, 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71229407300000003, 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5312 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:49.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258653+0000", 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277446+0000", 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018793994000000001 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277464+0000", 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7181e-05 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277476+0000", 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2167999999999999e-05 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878243+0000", 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60076709500000003 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878346+0000", 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103516 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878493+0000", 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000146205 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970686+0000", 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092192838999999999 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970867+0000", 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018116700000000001 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970904+0000", 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7474000000000001e-05 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970944+0000", 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9711e-05 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5315 3.3 3:c11688a5:::100000001f8.000006c0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8593137219999996, 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1629463150000001, 2024-09-19T05:44:49.208 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5315 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277505+0000", 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-06 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280350+0000", 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028443909999999999 2024-09-19T05:44:49.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280373+0000", 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3711999999999999e-05 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280386+0000", 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2252999999999999e-05 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188178+0000", 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90779182800000002 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188297+0000", 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119034 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188468+0000", 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000171139 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.319232+0000", 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13076428000000001 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434608+0000", 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115376337 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440394+0000", 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057860999999999998 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440449+0000", 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4761e-05 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5319 3.3b 3:de9a4780:::100000001f8.000006c4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8564056879999997, 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1541835229999999, 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.214 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5319 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280414+0000", 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9969999999999999e-06 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285680+0000", 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052661560000000001 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285697+0000", 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6883000000000001e-05 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285712+0000", 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4892000000000001e-05 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253316+0000", 2024-09-19T05:44:49.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96760461799999997 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253448+0000", 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013193499999999999 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434435+0000", 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18098683800000001 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434490+0000", 2024-09-19T05:44:49.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4833000000000002e-05 2024-09-19T05:44:49.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434594+0000", 2024-09-19T05:44:49.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010437099999999999 2024-09-19T05:44:49.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5325 3.c 3:328b8b05:::100000001f8.000006ca:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8443120479999999, 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75472665100000003, 2024-09-19T05:44:49.221 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5325 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292508+0000", 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3699999999999999e-06 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301580+0000", 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0090716010000000003 2024-09-19T05:44:49.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301598+0000", 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8278999999999999e-05 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301610+0000", 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.184e-05 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893481+0000", 2024-09-19T05:44:49.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59187088300000001 2024-09-19T05:44:49.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893784+0000", 2024-09-19T05:44:49.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030305399999999998 2024-09-19T05:44:49.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893972+0000", 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018840600000000001 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.972882+0000", 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.078910441999999997 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047120+0000", 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074237769999999995 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047176+0000", 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6224000000000002e-05 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047231+0000", 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4781999999999998e-05 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5327 3.2c 3:347123bb:::100000001f8.000006cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.8351783170000004, 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93214956400000004, 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5327 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301641+0000", 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.903e-06 2024-09-19T05:44:49.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.304411+0000", 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027695879999999999 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336856+0000", 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032444796999999997 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336884+0000", 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8246999999999999e-05 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970976+0000", 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.634091768 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971067+0000", 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1092000000000001e-05 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971228+0000", 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016114800000000001 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047103+0000", 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075875451999999996 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.221630+0000", 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17452633000000001 2024-09-19T05:44:49.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233731+0000", 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012101569 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233788+0000", 2024-09-19T05:44:49.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6669999999999998e-05 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5355 3.1c 3:38412458:::100000001f8.000006e8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.6583276829999996, 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89709229800000001, 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5355 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478493+0000", 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9929999999999997e-06 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488576+0000", 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010082702000000001 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488591+0000", 2024-09-19T05:44:49.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5607000000000002e-05 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488601+0000", 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0035e-05 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047402+0000", 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55880116199999996 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047498+0000", 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5828000000000005e-05 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047655+0000", 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015719799999999999 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180902+0000", 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13324624500000001 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375474+0000", 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19457263499999999 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375539+0000", 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4295999999999996e-05 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375581+0000", 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2596999999999999e-05 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5365 3.34 3:2ec8418e:::100000001f8.000006f2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.6481911629999999, 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88704814600000004, 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5365 2024-09-19T05:44:49.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488628+0000", 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4760000000000001e-06 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505248+0000", 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016620078999999999 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505273+0000", 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4527000000000001e-05 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505283+0000", 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0351e-05 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073161+0000", 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56787809499999997 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073310+0000", 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014909799999999999 2024-09-19T05:44:49.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073513+0000", 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203088 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180947+0000", 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10743341100000001 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375593+0000", 2024-09-19T05:44:49.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19464604999999999 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375634+0000", 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1115999999999997e-05 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375674+0000", 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9854999999999997e-05 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5367 3.1c 3:38083d86:::100000001f8.000006f4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.6315153929999999, 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87052496000000001, 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5367 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505303+0000", 2024-09-19T05:44:49.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.198e-06 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527093+0000", 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021790053 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527114+0000", 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0774000000000001e-05 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527127+0000", 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2292e-05 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110650+0000", 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58352302099999998 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110816+0000", 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166174 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.113130+0000", 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023145470000000001 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180979+0000", 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067848435999999998 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375735+0000", 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19475654100000001 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375783+0000", 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7478000000000002e-05 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375826+0000", 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3445999999999997e-05 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5404 3.2c 3:357d9333:::100000001f8.00000719:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.4856736540000002, 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75485259100000002, 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5404 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:49.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651146+0000", 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7750000000000001e-06 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.666480+0000", 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015334166999999999 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709279+0000", 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042798738000000003 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709297+0000", 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8119999999999999e-05 2024-09-19T05:44:49.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242624+0000", 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53332776400000004 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242737+0000", 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112262 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:49.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242935+0000", 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019831499999999999 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405710+0000", 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.162774789 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405918+0000", 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020817599999999999 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:49.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405962+0000", 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3606999999999998e-05 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405996+0000", 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3878000000000003e-05 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:49.259 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:49.260 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:49.261 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:49.262 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:49.259+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:49.362 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:49.360+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:49.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.410+0000 7f8c48942640 1 Processor -- start 2024-09-19T05:44:49.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.410+0000 7f8c48942640 1 -- start start 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c48942640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 0x7f8c440b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c48942640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8c440cd8c0 0x7f8c440b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c48942640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c4415ebe0 con 0x7f8c4415aa90 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c48942640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8c4415eec0 con 0x7f8c440cd8c0 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c48942640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8c440b2e00 con 0x7f8c440cd4e0 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c42d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8c440cd8c0 0x7f8c440b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c41d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8c4415aa90 0x7f8c4415df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:56826/0 (socket says 172.21.15.2:56826) 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c42d9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8c440cd8c0 0x7f8c440b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60230/0 (socket says 172.21.15.2:60230) 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c41d9b640 1 -- 172.21.15.2:0/2893129843 learned_addr learned my addr 172.21.15.2:0/2893129843 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c4259c640 1 --2- 172.21.15.2:0/2893129843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 0x7f8c440b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.414 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c42d9d640 1 -- 172.21.15.2:0/2893129843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 msgr2=0x7f8c440b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:49.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/2893129843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 0x7f8c440b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c42d9d640 1 -- 172.21.15.2:0/2893129843 >> v1:172.21.15.2:6789/0 conn(0x7f8c4415aa90 legacy=0x7f8c4415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c4159a640 1 -- 172.21.15.2:0/2893129843 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268494704 0 0) 0x7f8c4415ebe0 con 0x7f8c4415aa90 2024-09-19T05:44:49.415 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.411+0000 7f8c42d9d640 1 -- 172.21.15.2:0/2893129843 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c440b30e0 con 0x7f8c440cd8c0 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.412+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/2893129843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 0x7f8c440b4100 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009d90 tx=0x7f8c3c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d481562f9c98ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.412+0000 7f8c4159a640 1 -- 172.21.15.2:0/2893129843 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c007740 con 0x7f8c440cd8c0 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.412+0000 7f8c4159a640 1 -- 172.21.15.2:0/2893129843 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0078e0 con 0x7f8c440cd8c0 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.412+0000 7f8c4159a640 1 -- 172.21.15.2:0/2893129843 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c007cb0 con 0x7f8c440cd8c0 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 -- 172.21.15.2:0/2893129843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 msgr2=0x7f8c440b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 --2- 172.21.15.2:0/2893129843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 0x7f8c440b4100 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009d90 tx=0x7f8c3c02a660 comp rx=0 tx=0).stop 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 -- 172.21.15.2:0/2893129843 shutdown_connections 2024-09-19T05:44:49.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 --2- 172.21.15.2:0/2893129843 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 0x7f8c440b4100 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 --2- 172.21.15.2:0/2893129843 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 0x7f8c440b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 -- 172.21.15.2:0/2893129843 >> 172.21.15.2:0/2893129843 conn(0x7f8c44067b30 msgr2=0x7f8c44067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:49.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 -- 172.21.15.2:0/2893129843 shutdown_connections 2024-09-19T05:44:49.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.413+0000 7f8c48942640 1 -- 172.21.15.2:0/2893129843 wait complete. 2024-09-19T05:44:49.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.415+0000 7f8c48942640 1 Processor -- start 2024-09-19T05:44:49.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.415+0000 7f8c48942640 1 -- start start 2024-09-19T05:44:49.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c48942640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 0x7f8c440cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c48942640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 0x7f8c4414ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c48942640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4415aa90 0x7f8c44150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c48942640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4415e640 con 0x7f8c440cd8c0 2024-09-19T05:44:49.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c48942640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c440b36a0 con 0x7f8c4415aa90 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c48942640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4415e7c0 con 0x7f8c440cd4e0 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 0x7f8c4414ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 0x7f8c4414ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60240/0 (socket says 172.21.15.2:60240) 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 -- 172.21.15.2:0/148977935 learned_addr learned my addr 172.21.15.2:0/148977935 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4415aa90 0x7f8c44150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c4259c640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 0x7f8c440cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 -- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 msgr2=0x7f8c440cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c440cd4e0 0x7f8c440cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 -- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4415aa90 msgr2=0x7f8c44150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c4415aa90 0x7f8c44150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.416+0000 7f8c41d9b640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c441f4670 con 0x7f8c440cd8c0 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.417+0000 7f8c41d9b640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8c440cd8c0 0x7f8c4414ffc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800c500 tx=0x7f8c3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.417+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38018070 con 0x7f8c440cd8c0 2024-09-19T05:44:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.417+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c38007580 con 0x7f8c440cd8c0 2024-09-19T05:44:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.417+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38004a20 con 0x7f8c440cd8c0 2024-09-19T05:44:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.417+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c440c3e30 con 0x7f8c440cd8c0 2024-09-19T05:44:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.417+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c440c40d0 con 0x7f8c440cd8c0 2024-09-19T05:44:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.418+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8c4415df30 con 0x7f8c440cd8c0 2024-09-19T05:44:49.421 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.419+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8c3801b070 con 0x7f8c440cd8c0 2024-09-19T05:44:49.422 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:49.422 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:49.422 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:49.422 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.419+0000 7f8c32ffd640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8c2406c2e0 0x7f8c2406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.420+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5799+0+0 (secure 0 0 0) 0x7f8c3808a640 con 0x7f8c440cd8c0 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.420+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v76) ==== 54+0+16 (secure 0 0 0) 0x7f8c38090170 con 0x7f8c440cd8c0 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.420+0000 7f8c4259c640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8c2406c2e0 0x7f8c2406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.420+0000 7f8c48942640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8c4414e470 0x7f8c4415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.420+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c441f4130 con 0x7f8c4414e470 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.420+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8c4414e470 0x7f8c4415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.420+0000 7f8c4259c640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8c2406c2e0 0x7f8c2406e7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009aa0 tx=0x7f8c34009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.421+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f8c4414e470 0x7f8c4415cb50 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.424 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.422+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c3c007640 con 0x7f8c4414e470 2024-09-19T05:44:49.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:49.439 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.437+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8c4406d420 con 0x7f8c4414e470 2024-09-19T05:44:49.441 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.439+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8c4406d420 con 0x7f8c4414e470 2024-09-19T05:44:49.442 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:49.442 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:49.442 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.440+0000 7f8c48942640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8c4415d5a0 0x7f8c4414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.440+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8c4415d980 con 0x7f8c4415d5a0 2024-09-19T05:44:49.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.440+0000 7f8c4259c640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8c4415d5a0 0x7f8c4414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.441+0000 7f8c4259c640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f8c4415d5a0 0x7f8c4414d210 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.443 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.441+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c4415d980 con 0x7f8c4415d5a0 2024-09-19T05:44:49.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.456+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8c4406d140 con 0x7f8c4415d5a0 2024-09-19T05:44:49.460 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.458+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8c4406d140 con 0x7f8c4415d5a0 2024-09-19T05:44:49.461 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:49.461 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:49.461 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.459+0000 7f8c48942640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8c4414dc60 0x7f8c44062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.459+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8c4414e040 con 0x7f8c4414dc60 2024-09-19T05:44:49.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.459+0000 7f8c41d9b640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8c4414dc60 0x7f8c44062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.459+0000 7f8c41d9b640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f8c4414dc60 0x7f8c44062630 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.462 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.460+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c4414e040 con 0x7f8c4414dc60 2024-09-19T05:44:49.463 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:49.461+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:49.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.474+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8c440cfed0 con 0x7f8c4414dc60 2024-09-19T05:44:49.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.477+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8c440cfed0 con 0x7f8c4414dc60 2024-09-19T05:44:49.480 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:49.481 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:49.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.477+0000 7f8c48942640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8c440ce280 0x7f8c440ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.477+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8c44063440 con 0x7f8c440ce280 2024-09-19T05:44:49.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.478+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8c440ce280 0x7f8c440ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.478+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8c440ce280 0x7f8c440ce660 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.479+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c44063440 con 0x7f8c440ce280 2024-09-19T05:44:49.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.493+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8c440cee70 con 0x7f8c440ce280 2024-09-19T05:44:49.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.495+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8c440cee70 con 0x7f8c440ce280 2024-09-19T05:44:49.497 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.495+0000 7f8c48942640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8c4402cb50 0x7f8c4402cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.495+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8c440cedf0 con 0x7f8c4402cb50 2024-09-19T05:44:49.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.496+0000 7f8c4259c640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8c4402cb50 0x7f8c4402cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.496+0000 7f8c4259c640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f8c4402cb50 0x7f8c4402cf50 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.497+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c440cedf0 con 0x7f8c4402cb50 2024-09-19T05:44:49.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.511+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8c440635f0 con 0x7f8c4402cb50 2024-09-19T05:44:49.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.513+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8c34004340 con 0x7f8c4402cb50 2024-09-19T05:44:49.515 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:49.517 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.513+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=77}) -- 0x7f8c440cefd0 con 0x7f8c440cd8c0 2024-09-19T05:44:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.513+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c4402d490 con 0x7f8c440cd8c0 2024-09-19T05:44:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.513+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8c38091070 con 0x7f8c440cd8c0 2024-09-19T05:44:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:49.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.513+0000 7f8c48942640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8c44205210 0x7f8c44207600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.513+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f8c4402daf0 con 0x7f8c44205210 2024-09-19T05:44:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.514+0000 7f8c41d9b640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8c44205210 0x7f8c44207600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.514+0000 7f8c41d9b640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f8c44205210 0x7f8c44207600 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.515+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c4402daf0 con 0x7f8c44205210 2024-09-19T05:44:49.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.529+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8c4402dac0 con 0x7f8c44205210 2024-09-19T05:44:49.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.531+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8c4402dac0 con 0x7f8c44205210 2024-09-19T05:44:49.534 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.531+0000 7f8c48942640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8c44208050 0x7f8c4420a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.531+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f8c4420d420 con 0x7f8c44208050 2024-09-19T05:44:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.532+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8c44208050 0x7f8c4420a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.532+0000 7f8c42d9d640 1 --2- 172.21.15.2:0/148977935 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8c44208050 0x7f8c4420a440 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:49.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.533+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c4420d420 con 0x7f8c44208050 2024-09-19T05:44:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.547+0000 7f8c48942640 1 -- 172.21.15.2:0/148977935 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8c4402d6c0 con 0x7f8c44208050 2024-09-19T05:44:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:49.549+0000 7f8c32ffd640 1 -- 172.21.15.2:0/148977935 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8c4402d6c0 con 0x7f8c44208050 2024-09-19T05:44:49.552 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:49.556 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:49.556 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:49.557 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:49.557 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:49.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:49.569 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:49.565+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:49.665 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:49.663+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:49.728 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:49.728 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:49.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:49.765 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:49.764+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:49.858 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:49.858 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:49.867 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:49.865+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:49.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:49.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:49.967 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:49.966+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:50.029 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:50.030 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:50.042 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:50.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:44:50.051 INFO:teuthology.orchestra.run.smithi123.stderr:no valid command found; 10 closest matches: 2024-09-19T05:44:50.051 INFO:teuthology.orchestra.run.smithi123.stderr:0 2024-09-19T05:44:50.051 INFO:teuthology.orchestra.run.smithi123.stderr:1 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:2 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:abort 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:assert 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:config diff 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:config diff get 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:config get 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:config help [] 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:config set ... 2024-09-19T05:44:50.052 INFO:teuthology.orchestra.run.smithi123.stderr:admin_socket: invalid command 2024-09-19T05:44:50.054 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T05:44:50.054 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-19T05:44:50.068 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:50.067+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:50.090 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:50.090 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 readers = 8932793.00 kB/sec 2024-09-19T05:44:50.090 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 readers = 8931173.80 kB/sec 2024-09-19T05:44:50.090 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 8932793.00 kB/sec 2024-09-19T05:44:50.090 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 8932793.00 kB/sec 2024-09-19T05:44:50.090 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 8932793.00 kB/sec 2024-09-19T05:44:50.091 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:44:50.169 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:50.167+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:50.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.185+0000 7ff71f903640 1 Processor -- start 2024-09-19T05:44:50.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.185+0000 7ff71f903640 1 -- start start 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71f903640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff718155000 0x7ff7180cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71f903640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7180caf40 0x7ff71815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71f903640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7180ccbc0 con 0x7ff718154c20 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71f903640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ff7180cc600 con 0x7ff718155000 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71f903640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff7180cc8e0 con 0x7ff7180caf40 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71ce9e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff718155000 0x7ff7180cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71ce9e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff718155000 0x7ff7180cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60258/0 (socket says 172.21.15.2:60258) 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71ce9e640 1 -- 172.21.15.2:0/3013551691 learned_addr learned my addr 172.21.15.2:0/3013551691 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71ce9e640 1 -- 172.21.15.2:0/3013551691 >> v2:172.21.15.123:3300/0 conn(0x7ff7180caf40 msgr2=0x7ff71815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71dea0640 1 --2- 172.21.15.2:0/3013551691 >> v2:172.21.15.123:3300/0 conn(0x7ff7180caf40 0x7ff71815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:50.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71ce9e640 1 --2- 172.21.15.2:0/3013551691 >> v2:172.21.15.123:3300/0 conn(0x7ff7180caf40 0x7ff71815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:50.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71ce9e640 1 -- 172.21.15.2:0/3013551691 >> v1:172.21.15.123:6789/0 conn(0x7ff718154c20 legacy=0x7ff718073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:50.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71ce9e640 1 -- 172.21.15.2:0/3013551691 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7180cbc60 con 0x7ff718155000 2024-09-19T05:44:50.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.186+0000 7ff71dea0640 1 --2- 172.21.15.2:0/3013551691 >> v2:172.21.15.123:3300/0 conn(0x7ff7180caf40 0x7ff71815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:44:50.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.187+0000 7ff71ce9e640 1 --2- 172.21.15.2:0/3013551691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718155000 0x7ff7180cd9e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff7180c5de0 tx=0x7ff70802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d96eda32959ede server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:50.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.187+0000 7ff70ffff640 1 -- 172.21.15.2:0/3013551691 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708004030 con 0x7ff718155000 2024-09-19T05:44:50.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.187+0000 7ff70ffff640 1 -- 172.21.15.2:0/3013551691 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7080041d0 con 0x7ff718155000 2024-09-19T05:44:50.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.187+0000 7ff70ffff640 1 -- 172.21.15.2:0/3013551691 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708003610 con 0x7ff718155000 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 -- 172.21.15.2:0/3013551691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718155000 msgr2=0x7ff7180cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 --2- 172.21.15.2:0/3013551691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718155000 0x7ff7180cd9e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff7180c5de0 tx=0x7ff70802a210 comp rx=0 tx=0).stop 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 -- 172.21.15.2:0/3013551691 shutdown_connections 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 --2- 172.21.15.2:0/3013551691 >> v2:172.21.15.123:3300/0 conn(0x7ff7180caf40 0x7ff71815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 --2- 172.21.15.2:0/3013551691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff718155000 0x7ff7180cd9e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 -- 172.21.15.2:0/3013551691 >> 172.21.15.2:0/3013551691 conn(0x7ff71806d0e0 msgr2=0x7ff71806d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 -- 172.21.15.2:0/3013551691 shutdown_connections 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.188+0000 7ff71f903640 1 -- 172.21.15.2:0/3013551691 wait complete. 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.189+0000 7ff71f903640 1 Processor -- start 2024-09-19T05:44:50.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.189+0000 7ff71f903640 1 -- start start 2024-09-19T05:44:50.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71f903640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7180caf40 0x7ff71814e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:50.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71f903640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718154c20 0x7ff718159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:50.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71f903640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718155000 0x7ff718156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:50.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71f903640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff718073a50 con 0x7ff7180caf40 2024-09-19T05:44:50.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71f903640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7180cb430 con 0x7ff718154c20 2024-09-19T05:44:50.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71f903640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff718073bd0 con 0x7ff718155000 2024-09-19T05:44:50.194 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:50.194 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:50.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7180caf40 0x7ff71814e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:50.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7180caf40 0x7ff71814e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60266/0 (socket says 172.21.15.2:60266) 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 -- 172.21.15.2:0/2323087835 learned_addr learned my addr 172.21.15.2:0/2323087835 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71ce9e640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718154c20 0x7ff718159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71dea0640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718155000 0x7ff718156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 -- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718155000 msgr2=0x7ff718156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718155000 0x7ff718156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 -- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718154c20 msgr2=0x7ff718159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718154c20 0x7ff718159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 -- 172.21.15.2:0/2323087835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff718158160 con 0x7ff7180caf40 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.190+0000 7ff71d69f640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7180caf40 0x7ff71814e1e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c410 tx=0x7ff71400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.191+0000 7ff70dffb640 1 -- 172.21.15.2:0/2323087835 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714017070 con 0x7ff7180caf40 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.191+0000 7ff70dffb640 1 -- 172.21.15.2:0/2323087835 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7140035f0 con 0x7ff7180caf40 2024-09-19T05:44:50.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.191+0000 7ff70dffb640 1 -- 172.21.15.2:0/2323087835 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714003a20 con 0x7ff7180caf40 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.191+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181e1150 con 0x7ff7180caf40 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.191+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff71814e720 con 0x7ff7180caf40 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.192+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7180c5770 con 0x7ff7180caf40 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.193+0000 7ff70dffb640 1 -- 172.21.15.2:0/2323087835 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff71401a070 con 0x7ff7180caf40 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.193+0000 7ff70dffb640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6fc06c2b0 0x7ff6fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.193+0000 7ff70dffb640 1 -- 172.21.15.2:0/2323087835 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5799+0+0 (secure 0 0 0) 0x7ff71408a810 con 0x7ff7180caf40 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.193+0000 7ff71ce9e640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6fc06c2b0 0x7ff6fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:50.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.194+0000 7ff71ce9e640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6fc06c2b0 0x7ff6fc06e770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff708033000 tx=0x7ff70802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:50.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.198+0000 7ff70dffb640 1 -- 172.21.15.2:0/2323087835 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff714058960 con 0x7ff7180caf40 2024-09-19T05:44:50.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:50.270 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:50.269+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:50.310 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:50.311 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:50.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:50.371 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:50.369+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:50.386 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:50.384+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7ff7180b32c0 con 0x7ff7180caf40 2024-09-19T05:44:50.403 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:50.402+0000 7f9b31412640 -1 osd.5 67 log_to_monitors true 2024-09-19T05:44:50.426 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:50.426 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:50.435 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:50.477 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:50.475+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:50.573 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:50.571+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:50.584 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:50.584 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:50.593 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:50.674 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:50.672+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:50.697 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:50.697 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:50.706 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:50.789 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:50.788+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:50.849 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:50.850 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:50.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:50.876 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:50.874+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:50.977 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:50.975+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:51.001 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:51.002 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:51.011 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:51.041 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.038+0000 7ff70dffb640 1 -- 172.21.15.2:0/2323087835 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v77) ==== 87+0+0 (secure 0 0 0) 0x7ff71405cc60 con 0x7ff7180caf40 2024-09-19T05:44:51.041 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6fc06c2b0 msgr2=0x7ff6fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6fc06c2b0 0x7ff6fc06e770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff708033000 tx=0x7ff70802e000 comp rx=0 tx=0).stop 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7180caf40 msgr2=0x7ff71814e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7180caf40 0x7ff71814e1e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c410 tx=0x7ff71400c8d0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 shutdown_connections 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff6fc06c2b0 0x7ff6fc06e770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff718154c20 0x7ff718159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff718155000 0x7ff718156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 --2- 172.21.15.2:0/2323087835 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7180caf40 0x7ff71814e1e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.041+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 >> 172.21.15.2:0/2323087835 conn(0x7ff71806d0e0 msgr2=0x7ff71814f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:51.044 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.042+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 shutdown_connections 2024-09-19T05:44:51.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.042+0000 7ff71f903640 1 -- 172.21.15.2:0/2323087835 wait complete. 2024-09-19T05:44:51.069 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:51.066+0000 7f9b24abe640 -1 osd.5 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T05:44:51.077 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:51.075+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:51.125 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:51.125 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:51.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:51.178 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:51.176+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:51.273 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:51.274 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:51.279 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:51.278+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:51.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:51.380 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:51.379+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:51.383 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:51.383 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:51.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:51.481 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:51.480+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:51.537 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:51.537 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:51.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:51.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:51.602 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:51.600+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:51.647 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:51.647 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:51.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.669+0000 7fe17519d640 1 Processor -- start 2024-09-19T05:44:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.669+0000 7fe17519d640 1 -- start start 2024-09-19T05:44:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe17519d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe170154bb0 0x7fe17015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe17519d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 0x7fe1700d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe17519d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe17015b8e0 con 0x7fe1700cac90 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe17519d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe1700cb220 con 0x7fe170154bb0 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe17519d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe1700cb500 con 0x7fe17006cd80 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe170154bb0 0x7fe17015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe170154bb0 0x7fe17015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39924/0 (socket says 172.21.15.2:39924) 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16ffff640 1 -- 172.21.15.2:0/3353571694 learned_addr learned my addr 172.21.15.2:0/3353571694 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/3353571694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 0x7fe1700d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16ffff640 1 -- 172.21.15.2:0/3353571694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 msgr2=0x7fe1700d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16ffff640 1 --2- 172.21.15.2:0/3353571694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 0x7fe1700d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16ffff640 1 -- 172.21.15.2:0/3353571694 >> v1:172.21.15.123:6789/0 conn(0x7fe1700cac90 legacy=0x7fe1700c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.670+0000 7fe16ffff640 1 -- 172.21.15.2:0/3353571694 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1700cb7e0 con 0x7fe170154bb0 2024-09-19T05:44:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.671+0000 7fe16ffff640 1 --2- 172.21.15.2:0/3353571694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe170154bb0 0x7fe17015b3a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe160009870 tx=0x7fe16002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe539ab24c8f4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.671+0000 7fe16effd640 1 -- 172.21.15.2:0/3353571694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160037070 con 0x7fe170154bb0 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.671+0000 7fe16effd640 1 -- 172.21.15.2:0/3353571694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16002ae50 con 0x7fe170154bb0 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.671+0000 7fe16effd640 1 -- 172.21.15.2:0/3353571694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160032860 con 0x7fe170154bb0 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 -- 172.21.15.2:0/3353571694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe170154bb0 msgr2=0x7fe17015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 --2- 172.21.15.2:0/3353571694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe170154bb0 0x7fe17015b3a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe160009870 tx=0x7fe16002a300 comp rx=0 tx=0).stop 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 -- 172.21.15.2:0/3353571694 shutdown_connections 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 --2- 172.21.15.2:0/3353571694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 0x7fe1700d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 --2- 172.21.15.2:0/3353571694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe170154bb0 0x7fe17015b3a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 -- 172.21.15.2:0/3353571694 >> 172.21.15.2:0/3353571694 conn(0x7fe17006e350 msgr2=0x7fe17006e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 -- 172.21.15.2:0/3353571694 shutdown_connections 2024-09-19T05:44:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.672+0000 7fe17519d640 1 -- 172.21.15.2:0/3353571694 wait complete. 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.673+0000 7fe17519d640 1 Processor -- start 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.673+0000 7fe17519d640 1 -- start start 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe17519d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 0x7fe17014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe17519d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1700cac90 0x7fe1700c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe17519d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170154bb0 0x7fe1700c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe17519d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe170154f90 con 0x7fe1700cac90 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe17519d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1700c5670 con 0x7fe170154bb0 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe17519d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe1700c62a0 con 0x7fe17006cd80 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe16f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1700cac90 0x7fe1700c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe16f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1700cac90 0x7fe1700c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39934/0 (socket says 172.21.15.2:39934) 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe16f7fe640 1 -- 172.21.15.2:0/2704386775 learned_addr learned my addr 172.21.15.2:0/2704386775 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe16ffff640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 0x7fe17014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.674+0000 7fe17499c640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170154bb0 0x7fe1700c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16f7fe640 1 -- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 msgr2=0x7fe17014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe17006cd80 0x7fe17014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16f7fe640 1 -- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170154bb0 msgr2=0x7fe1700c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe170154bb0 0x7fe1700c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16f7fe640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17014bc80 con 0x7fe1700cac90 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1700cac90 0x7fe1700c88d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe16400e9b0 tx=0x7fe16400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164017070 con 0x7fe1700cac90 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe16400c9a0 con 0x7fe1700cac90 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17014b9a0 con 0x7fe1700cac90 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16400cd90 con 0x7fe1700cac90 2024-09-19T05:44:51.679 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.675+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1700c7990 con 0x7fe1700cac90 2024-09-19T05:44:51.680 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.676+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe17015b3a0 con 0x7fe1700cac90 2024-09-19T05:44:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.677+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe16401a070 con 0x7fe1700cac90 2024-09-19T05:44:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.677+0000 7fe16cff9640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe15c06c2b0 0x7fe15c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.677+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5799+0+0 (secure 0 0 0) 0x7fe16408b910 con 0x7fe1700cac90 2024-09-19T05:44:51.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.677+0000 7fe16ffff640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe15c06c2b0 0x7fe15c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.677+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v77) ==== 54+0+16 (secure 0 0 0) 0x7fe16408c0c0 con 0x7fe1700cac90 2024-09-19T05:44:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.678+0000 7fe17519d640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe1700c7b60 0x7fe17014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.678+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1700cffb0 con 0x7fe1700c7b60 2024-09-19T05:44:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.678+0000 7fe17499c640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe1700c7b60 0x7fe17014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.678+0000 7fe16ffff640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe15c06c2b0 0x7fe15c06e770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe160034000 tx=0x7fe16002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.678+0000 7fe17499c640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fe1700c7b60 0x7fe17014eb00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.679+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1700cffb0 con 0x7fe1700c7b60 2024-09-19T05:44:51.683 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:51.681+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:51.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.693+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe17014f040 con 0x7fe1700c7b60 2024-09-19T05:44:51.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.695+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe17014f040 con 0x7fe1700c7b60 2024-09-19T05:44:51.697 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.695+0000 7fe17519d640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe17014b190 0x7fe17014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.695+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe1700c8e10 con 0x7fe17014b190 2024-09-19T05:44:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.695+0000 7fe16ffff640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe17014b190 0x7fe17014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.696+0000 7fe16ffff640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe17014b190 0x7fe17014fbb0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.696+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1700c8e10 con 0x7fe17014b190 2024-09-19T05:44:51.713 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.711+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe1700ca3a0 con 0x7fe17014b190 2024-09-19T05:44:51.714 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.712+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe1700ca3a0 con 0x7fe17014b190 2024-09-19T05:44:51.715 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:51.715 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:51.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.713+0000 7fe17519d640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe170150600 0x7fe170062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.713+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe1701509e0 con 0x7fe170150600 2024-09-19T05:44:51.715 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.713+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe170150600 0x7fe170062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.713+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fe170150600 0x7fe170062630 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.716 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.714+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1701509e0 con 0x7fe170150600 2024-09-19T05:44:51.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.727+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe170062dc0 con 0x7fe170150600 2024-09-19T05:44:51.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.729+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe170062dc0 con 0x7fe170150600 2024-09-19T05:44:51.731 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:51.732 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:51.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.729+0000 7fe17519d640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe170063390 0x7fe170158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.729+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe1701591e0 con 0x7fe170063390 2024-09-19T05:44:51.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.729+0000 7fe17499c640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe170063390 0x7fe170158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.729+0000 7fe17499c640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fe170063390 0x7fe170158ca0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.730+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1701591e0 con 0x7fe170063390 2024-09-19T05:44:51.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.744+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe170159660 con 0x7fe170063390 2024-09-19T05:44:51.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.745+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe170159660 con 0x7fe170063390 2024-09-19T05:44:51.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.745+0000 7fe17519d640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe17002cb50 0x7fe17002cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.745+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe1701595e0 con 0x7fe17002cb50 2024-09-19T05:44:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.746+0000 7fe16ffff640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe17002cb50 0x7fe17002cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.746+0000 7fe16ffff640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fe17002cb50 0x7fe17002cfd0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.747+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1701595e0 con 0x7fe17002cb50 2024-09-19T05:44:51.759 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:51.759 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:51.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.760+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe1701595e0 con 0x7fe17002cb50 2024-09-19T05:44:51.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.762+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe1701595e0 con 0x7fe17002cb50 2024-09-19T05:44:51.764 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.762+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=78}) -- 0x7fe17002d510 con 0x7fe1700cac90 2024-09-19T05:44:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.762+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe17002d8e0 con 0x7fe1700cac90 2024-09-19T05:44:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.763+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe164021040 con 0x7fe1700cac90 2024-09-19T05:44:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-19T05:44:51.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.763+0000 7fe17519d640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe170205110 0x7fe170207500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.763+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fe17020a2d0 con 0x7fe170205110 2024-09-19T05:44:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.763+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe170205110 0x7fe170207500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.763+0000 7fe16f7fe640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fe170205110 0x7fe170207500 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.764+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe17020a2d0 con 0x7fe170205110 2024-09-19T05:44:51.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:51.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.778+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe170207a40 con 0x7fe170205110 2024-09-19T05:44:51.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.780+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe170207a40 con 0x7fe170205110 2024-09-19T05:44:51.782 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:51.783 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:51.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.780+0000 7fe17519d640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe1702080c0 0x7fe17020a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:51.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.780+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fe17020d380 con 0x7fe1702080c0 2024-09-19T05:44:51.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.780+0000 7fe17499c640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe1702080c0 0x7fe17020a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:51.783 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.781+0000 7fe17499c640 1 --2- 172.21.15.2:0/2704386775 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fe1702080c0 0x7fe17020a4b0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:51.784 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.781+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe17020d380 con 0x7fe1702080c0 2024-09-19T05:44:51.784 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:51.783+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.796+0000 7fe17519d640 1 -- 172.21.15.2:0/2704386775 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe17020d350 con 0x7fe1702080c0 2024-09-19T05:44:51.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:51.798+0000 7fe16cff9640 1 -- 172.21.15.2:0/2704386775 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe17020d350 con 0x7fe1702080c0 2024-09-19T05:44:51.800 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:51.800 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:51.803 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-19T05:44:51.804 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-19T05:44:51.877 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:51.878 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:51.878 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:51.878 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:51.885 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:51.883+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:51.886 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:51.986 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:51.984+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:52.035 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:52.035 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:52.035 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:52.035 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:52.035 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:52.046 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:52.088 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:52.086+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:52.189 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:52.187+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:52.200 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2637 3.3 3:c38cdf66:::100000001f8.0000064a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.368210573999999, 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.86377266600000002, 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2637 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829833+0000", 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4700000000000004e-06 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834195+0000", 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043615379999999999 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834217+0000", 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1713e-05 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834229+0000", 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2418e-05 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.139987+0000", 2024-09-19T05:44:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.305758321 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140183+0000", 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195577 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140385+0000", 2024-09-19T05:44:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020212799999999999 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.465853+0000", 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32546820700000001 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693468+0000", 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22761451199999999 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693549+0000", 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1267999999999995e-05 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693602+0000", 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2513999999999999e-05 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2681 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 20.215719838999998, 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.077969779, 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2681 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982323+0000", 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2370000000000001e-06 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986715+0000", 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004392618 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986751+0000", 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5398999999999998e-05 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986762+0000", 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1545999999999999e-05 2024-09-19T05:44:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706073+0000", 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71931063299999998 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706251+0000", 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017771099999999999 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060111+0000", 2024-09-19T05:44:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35386035900000001 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060200+0000", 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9259999999999996e-05 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060289+0000", 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9016000000000003e-05 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:3095 3.33 e71/20 3:cd66a262:::100000001f8.00000814:head v 71'69, mlcod=70'60)", 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.886902888000002, 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80810778000000005, 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311141+0000", 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.284e-06 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318008+0000", 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068671770000000004 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318032+0000", 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4467000000000001e-05 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318051+0000", 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8284e-05 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693668+0000", 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37561766600000002 2024-09-19T05:44:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693691+0000", 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2875e-05 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116967+0000", 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42327609900000002 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.119244+0000", 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022769280000000001 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5152 3.2c 3:350c5391:::100000001f8.0000061d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3450122229999995, 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80414443700000005, 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5152 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853033+0000", 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6400000000000002e-06 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856724+0000", 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036910649999999999 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856747+0000", 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2719999999999999e-05 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856759+0000", 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2679e-05 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948007+0000", 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091247606999999994 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948133+0000", 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012556500000000001 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948378+0000", 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024540299999999999 2024-09-19T05:44:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.240715+0000", 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29233733899999997 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657035+0000", 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41632016100000002 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657101+0000", 2024-09-19T05:44:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691999999999994e-05 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657172+0000", 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0566000000000002e-05 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5153 3.2c 3:35ced6af:::100000001f8.0000061e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3412503250000007, 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84790699300000005, 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5153 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856791+0000", 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2239999999999998e-06 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859552+0000", 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027601930000000002 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859564+0000", 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2880999999999999e-05 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859579+0000", 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4925e-05 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145653+0000", 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.286073313 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145820+0000", 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166937 2024-09-19T05:44:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.167125+0000", 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021305001 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.270722+0000", 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103597041 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704576+0000", 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43385462400000002 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704637+0000", 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1118000000000007e-05 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704696+0000", 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8736000000000001e-05 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5229 3.34 3:2ebd7ea6:::100000001f8.0000066a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.2654007010000008, 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90816846299999998, 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5229 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932642+0000", 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5820000000000002e-06 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937356+0000", 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047141400000000003 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937389+0000", 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2929999999999998e-05 2024-09-19T05:44:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937402+0000", 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2855e-05 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268202+0000", 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.330799856 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268403+0000", 2024-09-19T05:44:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200464 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268639+0000", 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000236598 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.656059+0000", 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.387419719 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840676+0000", 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18461746900000001 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840747+0000", 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0462000000000004e-05 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840807+0000", 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0387999999999998e-05 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5231 3.3b 3:ddf1e7a4:::100000001f8.0000066c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.1437775880000007, 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1076528050000001, 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5231 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054273+0000", 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1253999999999999e-05 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059647+0000", 2024-09-19T05:44:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053742479999999999 2024-09-19T05:44:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059676+0000", 2024-09-19T05:44:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8699000000000001e-05 2024-09-19T05:44:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059689+0000", 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2476999999999999e-05 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.863904+0000", 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.804215652 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.864024+0000", 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119848 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161715+0000", 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.297691133 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161795+0000", 2024-09-19T05:44:52.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9671000000000003e-05 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161915+0000", 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011982299999999999 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.243 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5241 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.0989526319999996, 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.063036227, 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5241 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.244 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099090+0000", 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105109+0000", 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006018809 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105132+0000", 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2908000000000002e-05 2024-09-19T05:44:52.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105143+0000", 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1826e-05 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.924904+0000", 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81976026999999996 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925014+0000", 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011063 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162023+0000", 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23700855300000001 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162060+0000", 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6715999999999999e-05 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162123+0000", 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3480999999999997e-05 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5250 3.3 3:c0cd83f8:::100000001f8.0000067f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.0630163049999997, 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026991652, 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5250 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:52.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135025+0000", 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3379999999999999e-06 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137223+0000", 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021978639999999999 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137247+0000", 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3978999999999999e-05 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.144008+0000", 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067608770000000002 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915821+0000", 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77181293399999995 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915914+0000", 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2493999999999999e-05 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.916430+0000", 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00051667599999999998 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.122065+0000", 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20563482999999999 2024-09-19T05:44:52.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161930+0000", 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039865356999999997 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161977+0000", 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6873999999999999e-05 2024-09-19T05:44:52.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162015+0000", 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7428999999999997e-05 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5293 3.3 3:c2b9c98f:::100000001f8.000006aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.9863827240000003, 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95068157399999997, 2024-09-19T05:44:52.255 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5293 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211659+0000", 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215320+0000", 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003660862 2024-09-19T05:44:52.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215339+0000", 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8868999999999998e-05 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215351+0000", 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1802e-05 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939166+0000", 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72381503599999997 2024-09-19T05:44:52.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939258+0000", 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1911999999999997e-05 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939406+0000", 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014874699999999999 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162132+0000", 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.222725545 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162260+0000", 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012829700000000001 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162304+0000", 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4215000000000003e-05 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162338+0000", 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3819000000000003e-05 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5301 3.c 3:3307abed:::100000001f8.000006b2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.9773969460000007, 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75017270800000002, 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5301 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220645+0000", 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0509999999999999e-06 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238885+0000", 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018240014999999998 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238906+0000", 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1498999999999999e-05 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238919+0000", 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2565000000000001e-05 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841065+0000", 2024-09-19T05:44:52.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60214654400000001 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841146+0000", 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0688999999999999e-05 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841342+0000", 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019650000000000001 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878211+0000", 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036868724999999998 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970700+0000", 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092489022000000004 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970754+0000", 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3480999999999998e-05 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970815+0000", 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1617000000000003e-05 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5312 3.2c 3:36326779:::100000001f8.000006bd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.9393896670000004, 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71229407300000003, 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.267 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5312 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258653+0000", 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277446+0000", 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018793994000000001 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277464+0000", 2024-09-19T05:44:52.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7181e-05 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277476+0000", 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2167999999999999e-05 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878243+0000", 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60076709500000003 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878346+0000", 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103516 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878493+0000", 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000146205 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970686+0000", 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092192838999999999 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970867+0000", 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018116700000000001 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970904+0000", 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7474000000000001e-05 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970944+0000", 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9711e-05 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5315 3.3 3:c11688a5:::100000001f8.000006c0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.9205365850000007, 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1629463150000001, 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5315 2024-09-19T05:44:52.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277505+0000", 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-06 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280350+0000", 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028443909999999999 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280373+0000", 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3711999999999999e-05 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280386+0000", 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2252999999999999e-05 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188178+0000", 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90779182800000002 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188297+0000", 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119034 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188468+0000", 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000171139 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.319232+0000", 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13076428000000001 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434608+0000", 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115376337 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440394+0000", 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057860999999999998 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440449+0000", 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4761e-05 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5319 3.3b 3:de9a4780:::100000001f8.000006c4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.917628551, 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1541835229999999, 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5319 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280414+0000", 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9969999999999999e-06 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285680+0000", 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052661560000000001 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285697+0000", 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6883000000000001e-05 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285712+0000", 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4892000000000001e-05 2024-09-19T05:44:52.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253316+0000", 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96760461799999997 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253448+0000", 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013193499999999999 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434435+0000", 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18098683800000001 2024-09-19T05:44:52.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434490+0000", 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4833000000000002e-05 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434594+0000", 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010437099999999999 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.283 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5325 3.c 3:328b8b05:::100000001f8.000006ca:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.9055349110000002, 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75472665100000003, 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5325 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292508+0000", 2024-09-19T05:44:52.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3699999999999999e-06 2024-09-19T05:44:52.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301580+0000", 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0090716010000000003 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301598+0000", 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8278999999999999e-05 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301610+0000", 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.184e-05 2024-09-19T05:44:52.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893481+0000", 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59187088300000001 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893784+0000", 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030305399999999998 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893972+0000", 2024-09-19T05:44:52.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018840600000000001 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.972882+0000", 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.078910441999999997 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047120+0000", 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074237769999999995 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047176+0000", 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6224000000000002e-05 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047231+0000", 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4781999999999998e-05 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5327 3.2c 3:347123bb:::100000001f8.000006cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.382 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.8964011799999998, 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93214956400000004, 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5327 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301641+0000", 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.903e-06 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.304411+0000", 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027695879999999999 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336856+0000", 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032444796999999997 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336884+0000", 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8246999999999999e-05 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970976+0000", 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.634091768 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971067+0000", 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1092000000000001e-05 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971228+0000", 2024-09-19T05:44:52.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016114800000000001 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047103+0000", 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075875451999999996 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.221630+0000", 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17452633000000001 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233731+0000", 2024-09-19T05:44:52.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012101569 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233788+0000", 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6669999999999998e-05 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5355 3.1c 3:38412458:::100000001f8.000006e8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:52.388 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7195505460000007, 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89709229800000001, 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5355 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:52.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478493+0000", 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9929999999999997e-06 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488576+0000", 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010082702000000001 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488591+0000", 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5607000000000002e-05 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488601+0000", 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0035e-05 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047402+0000", 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55880116199999996 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047498+0000", 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5828000000000005e-05 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047655+0000", 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015719799999999999 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180902+0000", 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13324624500000001 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375474+0000", 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19457263499999999 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375539+0000", 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4295999999999996e-05 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375581+0000", 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2596999999999999e-05 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5365 3.34 3:2ec8418e:::100000001f8.000006f2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.7094140259999993, 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88704814600000004, 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5365 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488628+0000", 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4760000000000001e-06 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505248+0000", 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016620078999999999 2024-09-19T05:44:52.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505273+0000", 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4527000000000001e-05 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505283+0000", 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0351e-05 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073161+0000", 2024-09-19T05:44:52.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56787809499999997 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073310+0000", 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014909799999999999 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073513+0000", 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203088 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180947+0000", 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10743341100000001 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375593+0000", 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19464604999999999 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375634+0000", 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1115999999999997e-05 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375674+0000", 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9854999999999997e-05 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5367 3.1c 3:38083d86:::100000001f8.000006f4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.6927382560000002, 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87052496000000001, 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5367 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:52.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505303+0000", 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.198e-06 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527093+0000", 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021790053 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527114+0000", 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0774000000000001e-05 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527127+0000", 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2292e-05 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110650+0000", 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58352302099999998 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110816+0000", 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166174 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.113130+0000", 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023145470000000001 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180979+0000", 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067848435999999998 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375735+0000", 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19475654100000001 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375783+0000", 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7478000000000002e-05 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375826+0000", 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3445999999999997e-05 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.405 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5404 3.2c 3:357d9333:::100000001f8.00000719:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.5468965170000004, 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75485259100000002, 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5404 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651146+0000", 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7750000000000001e-06 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.666480+0000", 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015334166999999999 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709279+0000", 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042798738000000003 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709297+0000", 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8119999999999999e-05 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242624+0000", 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53332776400000004 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242737+0000", 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112262 2024-09-19T05:44:52.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242935+0000", 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019831499999999999 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405710+0000", 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.162774789 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405918+0000", 2024-09-19T05:44:52.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020817599999999999 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405962+0000", 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3606999999999998e-05 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405996+0000", 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3878000000000003e-05 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.412 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:52.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:52.414 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:52.411+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:52.514 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:52.512+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:52.574 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:52.574 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:52.574 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:52.575 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:52.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:52.627 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:52.625+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:52.716 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:52.714+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:52.746 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:52.747 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:52.758 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:52.817 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:52.816+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:52.918 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:52.916+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:52.920 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:52.920 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:52.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:52.920 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5482 3.27 3:e79b25dc:::100000001f8.00000767:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:44:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8869181879999992, 2024-09-19T05:44:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71036533999999996, 2024-09-19T05:44:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5482 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030320+0000", 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3929999999999998e-06 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034225+0000", 2024-09-19T05:44:52.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003904678 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034278+0000", 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2534999999999998e-05 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034303+0000", 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.565e-05 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.583640+0000", 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54933696700000001 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.583850+0000", 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000210165 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740511+0000", 2024-09-19T05:44:52.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.156660363 2024-09-19T05:44:52.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740567+0000", 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5918999999999998e-05 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740683+0000", 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011667000000000001 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5507 3.27 3:e4f54558:::100000001f8.00000780:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8681502279999993, 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71559931600000004, 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5507 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049088+0000", 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4990000000000001e-06 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.070701+0000", 2024-09-19T05:44:52.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021612672999999999 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087613+0000", 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016911942999999999 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087634+0000", 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0835e-05 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.707112+0000", 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61947850400000004 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.707259+0000", 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014680800000000001 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764542+0000", 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057282971000000002 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764599+0000", 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6934000000000001e-05 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764685+0000", 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6149000000000005e-05 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5521 3.27 3:e73f7143:::100000001f8.0000078e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8295804830000009, 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76359052999999999, 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5521 2024-09-19T05:44:52.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087659+0000", 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9229999999999998e-06 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107649+0000", 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019990396000000001 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107667+0000", 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7958999999999999e-05 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107678+0000", 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1286e-05 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.742986+0000", 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.635308188 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.743091+0000", 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010469200000000001 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851137+0000", 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.108045961 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851180+0000", 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3192999999999998e-05 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851246+0000", 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5932000000000005e-05 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5725 3.27 3:e65abc03:::100000001f8.0000085a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.6121992939999998, 2024-09-19T05:44:52.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.755317554, 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5725 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305041+0000", 2024-09-19T05:44:52.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1030000000000004e-06 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307067+0000", 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020258870000000001 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307085+0000", 2024-09-19T05:44:52.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7931000000000002e-05 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307098+0000", 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3426e-05 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.914327+0000", 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60722910699999999 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.914435+0000", 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010719 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060107+0000", 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.145671988 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060199+0000", 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2615999999999996e-05 2024-09-19T05:44:52.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060354+0000", 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015530599999999999 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5745 3.27 3:e6f7c8d2:::100000001f8.0000086e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.6101113680000001, 2024-09-19T05:44:52.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75335722800000005, 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5745 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307128+0000", 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.308414+0000", 2024-09-19T05:44:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001286389 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.309604+0000", 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001189884 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.309621+0000", 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6572999999999999e-05 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925094+0000", 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61547286800000001 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925185+0000", 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1120999999999997e-05 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060376+0000", 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13519163200000001 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060418+0000", 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1703999999999999e-05 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060482+0000", 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4083000000000003e-05 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5763 3.27 3:e74ad6a4:::100000001f8.00000880:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.6045705120000004, 2024-09-19T05:44:52.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76551648800000005, 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5763 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312668+0000", 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.571e-06 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316538+0000", 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038699429999999998 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316577+0000", 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9180000000000001e-05 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316596+0000", 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8332999999999999e-05 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.950346+0000", 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63375050600000005 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.950512+0000", 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016611800000000001 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078037+0000", 2024-09-19T05:44:52.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12752486599999999 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078090+0000", 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2540000000000002e-05 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078182+0000", 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2430999999999996e-05 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5824 3.1f 3:fb07d980:::100000001f8.000008bd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5794476260000003, 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73478956900000003, 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5824 2024-09-19T05:44:52.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337792+0000", 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0419999999999998e-06 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347160+0000", 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093682149999999992 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347177+0000", 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6806000000000001e-05 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347187+0000", 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0033e-05 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.949530+0000", 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60234381400000003 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.949666+0000", 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013539300000000001 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070735+0000", 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.121069458 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.072015+0000", 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012792369999999999 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.072578+0000", 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056357100000000002 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.970 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5844 3.1f 3:fac155ab:::100000001f8.000008d1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.557369349, 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73934379100000003, 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5844 2024-09-19T05:44:52.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359869+0000", 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.543e-06 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363862+0000", 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039926709999999997 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363876+0000", 2024-09-19T05:44:52.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3951000000000001e-05 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363891+0000", 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4951e-05 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.975926+0000", 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61203488500000003 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.976042+0000", 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011658300000000001 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099026+0000", 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.122983839 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099101+0000", 2024-09-19T05:44:52.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4606999999999999e-05 2024-09-19T05:44:52.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099211+0000", 2024-09-19T05:44:52.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010976099999999999 2024-09-19T05:44:52.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5846 3.27 3:e6e86654:::100000001f8.000008d3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5533269870000002, 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71445892200000005, 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5846 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363912+0000", 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2819999999999999e-06 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367743+0000", 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038313359999999999 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367760+0000", 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6892e-05 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367774+0000", 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4248e-05 2024-09-19T05:44:52.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.016385+0000", 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.64861128400000001 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.016504+0000", 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118838 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078254+0000", 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.061749399000000003 2024-09-19T05:44:52.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078303+0000", 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9301000000000001e-05 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078368+0000", 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5341999999999999e-05 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5863 3.27 3:e5e0177a:::100000001f8.000008e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5494363670000002, 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.74712701699999995, 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5863 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367802+0000", 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6479999999999999e-06 2024-09-19T05:44:52.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372984+0000", 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051814299999999999 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373015+0000", 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0611000000000002e-05 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373054+0000", 2024-09-19T05:44:52.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9690000000000001e-05 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.031807+0000", 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65875312100000005 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.038739+0000", 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069312100000000001 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114756+0000", 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076017243999999998 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114836+0000", 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0644999999999998e-05 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114927+0000", 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0418000000000001e-05 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5876 3.27 3:e58ad412:::100000001f8.000008f1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5441619499999995, 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.74196762800000005, 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5876 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373077+0000", 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7520000000000001e-06 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378577+0000", 2024-09-19T05:44:52.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054999849999999998 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378607+0000", 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9850999999999999e-05 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378625+0000", 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8148e-05 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060530+0000", 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68190477100000002 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060633+0000", 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010288700000000001 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114938+0000", 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054305327 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114977+0000", 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8602000000000001e-05 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115042+0000", 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5304999999999999e-05 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5918 3.27 3:e7c6e76b:::100000001f8.0000091b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5233733730000001, 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72128804099999999, 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5918 2024-09-19T05:44:52.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393866+0000", 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2550000000000002e-06 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397678+0000", 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038118190000000001 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397703+0000", 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4592000000000001e-05 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397720+0000", 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7348999999999999e-05 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070763+0000", 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.67304348999999997 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070867+0000", 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103608 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115051+0000", 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044183605000000001 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115088+0000", 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7394999999999997e-05 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115151+0000", 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2928000000000005e-05 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:52.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5925 3.1f 3:f886b5f6:::100000001f8.00000922:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5194793149999999, 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77643015699999995, 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5925 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:44:53.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397761+0000", 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.963e-06 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421926+0000", 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024164986999999999 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421953+0000", 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6874000000000001e-05 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421966+0000", 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3413e-05 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.077691+0000", 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65572456599999995 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.077798+0000", 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010718299999999999 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.173642+0000", 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.095844253000000004 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174071+0000", 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000428658 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174187+0000", 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011626 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5932 3.1f 3:f86d9493:::100000001f8.00000929:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.4952409800000002, 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.85551307099999996, 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5932 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421998+0000", 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7429999999999998e-06 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432488+0000", 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010489777 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432514+0000", 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6315000000000001e-05 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432526+0000", 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1926999999999999e-05 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.102287+0000", 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66976068499999997 2024-09-19T05:44:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.102472+0000", 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018525900000000001 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277214+0000", 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17474197899999999 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277291+0000", 2024-09-19T05:44:53.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7194999999999995e-05 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277508+0000", 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021719099999999999 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6011 3.27 3:e774c081:::100000001f8.00000978:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.4747277929999996, 2024-09-19T05:44:53.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78469899799999998, 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6011 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442511+0000", 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7300000000000001e-06 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.511020+0000", 2024-09-19T05:44:53.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068508736000000001 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517290+0000", 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062700580000000002 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517329+0000", 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8649000000000002e-05 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099275+0000", 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58194679299999996 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099398+0000", 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012280099999999999 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.226947+0000", 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12754907100000001 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.227099+0000", 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151707 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.227207+0000", 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108453 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6019 3.1f 3:f909e746:::100000001f8.00000980:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3998621310000008, 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76027384200000003, 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6019 2024-09-19T05:44:53.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517381+0000", 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4439999999999996e-06 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.519989+0000", 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026080410000000002 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520018+0000", 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9522e-05 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520033+0000", 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5097e-05 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.151257+0000", 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63122408200000002 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.151415+0000", 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158197 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277523+0000", 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.126107891 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277569+0000", 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6066999999999998e-05 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277648+0000", 2024-09-19T05:44:53.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8500999999999999e-05 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6020 3.1f 3:f8dd987c:::100000001f8.00000981:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3971698450000005, 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75784574900000001, 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6020 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:44:53.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520069+0000", 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.540152+0000", 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020082744999999999 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.582892+0000", 2024-09-19T05:44:53.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042740527 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.582906+0000", 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162437+0000", 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57953022099999996 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162537+0000", 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9968000000000002e-05 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277661+0000", 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11512412700000001 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277747+0000", 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6367999999999997e-05 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277912+0000", 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016495299999999999 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6052 3.27 3:e7b93799:::100000001f8.000009a1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3255982379999995, 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70507024600000001, 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.026 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6052 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:44:53.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591641+0000", 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.695e-06 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595294+0000", 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00365373 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595319+0000", 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4411999999999999e-05 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595333+0000", 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4157e-05 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174616+0000", 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57928295299999999 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174703+0000", 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7268000000000005e-05 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296561+0000", 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.121857365 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296626+0000", 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5643999999999997e-05 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296708+0000", 2024-09-19T05:44:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2021999999999995e-05 2024-09-19T05:44:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6053 3.27 3:e77fb6c3:::100000001f8.000009a2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3218745129999991, 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70595110000000005, 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6053 2024-09-19T05:44:53.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595364+0000", 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.452e-06 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596835+0000", 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001470636 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596860+0000", 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4909000000000002e-05 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596874+0000", 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4321000000000001e-05 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.211679+0000", 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61480495499999999 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.211794+0000", 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114678 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301141+0000", 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.089347139000000006 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301215+0000", 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4632e-05 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301313+0000", 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7378000000000002e-05 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6061 3.27 3:e4baa440:::100000001f8.000009aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.3203339370000009, 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70453015299999999, 2024-09-19T05:44:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6061 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596905+0000", 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6000000000000001e-06 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598385+0000", 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001479878 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598409+0000", 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3751999999999999e-05 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598418+0000", 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.218690+0000", 2024-09-19T05:44:53.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.62027168499999996 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.218818+0000", 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000127983 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301324+0000", 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082505675000000001 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301366+0000", 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1947999999999999e-05 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301432+0000", 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6905999999999999e-05 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.042 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:53.044 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:53.044 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:53.042+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:53.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-19T05:44:53.144 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:53.142+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.206+0000 7f6db1a74640 1 Processor -- start 2024-09-19T05:44:53.208 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.206+0000 7f6db1a74640 1 -- start start 2024-09-19T05:44:53.209 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.207+0000 7f6db1a74640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6dac06c040 0x7f6dac15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.207+0000 7f6db1a74640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dac0739e0 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.207+0000 7f6dabfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6dac0c8ee0 0x7f6dac1654f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:50858/0 (socket says 172.21.15.2:50858) 2024-09-19T05:44:53.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.207+0000 7f6dabfff640 1 -- 172.21.15.2:0/2872899289 learned_addr learned my addr 172.21.15.2:0/2872899289 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:53.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.207+0000 7f6da2ffd640 1 --2- 172.21.15.2:0/2872899289 >> v2:172.21.15.2:3300/0 conn(0x7f6dac06c040 0x7f6dac15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.207+0000 7f6db1a74640 1 -- 172.21.15.2:0/2872899289 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dac15e160 con 0x7f6dac154d60 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6db1a74640 1 -- 172.21.15.2:0/2872899289 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f6dac15ed70 con 0x7f6dac06c040 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6daaffd640 1 -- 172.21.15.2:0/2872899289 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1017376465 0 0) 0x7f6dac15e160 con 0x7f6dac154d60 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6daaffd640 1 -- 172.21.15.2:0/2872899289 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d98003520 con 0x7f6dac154d60 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6daaffd640 1 -- 172.21.15.2:0/2872899289 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1011823554 0 0) 0x7f6dac0739e0 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6daaffd640 1 -- 172.21.15.2:0/2872899289 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dac15e160 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2872899289 >> v1:172.21.15.123:6789/0 conn(0x7f6dac154d60 legacy=0x7f6dac0c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2872899289 >> v1:172.21.15.2:6789/0 conn(0x7f6dac0c8ee0 legacy=0x7f6dac1654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.208+0000 7f6da2ffd640 1 -- 172.21.15.2:0/2872899289 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac15e440 con 0x7f6dac06c040 2024-09-19T05:44:53.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.209+0000 7f6da2ffd640 1 --2- 172.21.15.2:0/2872899289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac06c040 0x7f6dac15ab70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6dac15eef0 tx=0x7f6d9402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db09a0b33e2f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.209+0000 7f6daaffd640 1 -- 172.21.15.2:0/2872899289 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94037070 con 0x7f6dac06c040 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.209+0000 7f6daaffd640 1 -- 172.21.15.2:0/2872899289 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94032440 con 0x7f6dac06c040 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.209+0000 7f6daaffd640 1 -- 172.21.15.2:0/2872899289 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94032810 con 0x7f6dac06c040 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.210+0000 7f6db1a74640 1 -- 172.21.15.2:0/2872899289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac06c040 msgr2=0x7f6dac15ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.210+0000 7f6db1a74640 1 --2- 172.21.15.2:0/2872899289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac06c040 0x7f6dac15ab70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6dac15eef0 tx=0x7f6d9402a210 comp rx=0 tx=0).stop 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.210+0000 7f6db1a74640 1 -- 172.21.15.2:0/2872899289 shutdown_connections 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.210+0000 7f6db1a74640 1 --2- 172.21.15.2:0/2872899289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac06c040 0x7f6dac15ab70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.210+0000 7f6db1a74640 1 -- 172.21.15.2:0/2872899289 >> 172.21.15.2:0/2872899289 conn(0x7f6dac06f3d0 msgr2=0x7f6dac0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.210+0000 7f6db1a74640 1 -- 172.21.15.2:0/2872899289 shutdown_connections 2024-09-19T05:44:53.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.210+0000 7f6db1a74640 1 -- 172.21.15.2:0/2872899289 wait complete. 2024-09-19T05:44:53.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.212+0000 7f6db1a74640 1 Processor -- start 2024-09-19T05:44:53.215 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.213+0000 7f6db1a74640 1 -- start start 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6db1a74640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dac06c040 0x7f6dac155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6db1a74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac0c8ee0 0x7f6dac14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6db1a74640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dac154d60 0x7f6dac1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac0c8ee0 0x7f6dac14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6dab7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dac06c040 0x7f6dac155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac0c8ee0 0x7f6dac14a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39952/0 (socket says 172.21.15.2:39952) 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6dab7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dac06c040 0x7f6dac155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33410/0 (socket says 172.21.15.2:33410) 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3981186230 learned_addr learned my addr 172.21.15.2:0/3981186230 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac0c8b80 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac15ea00 con 0x7f6dac06c040 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.214+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6dac0c8d00 con 0x7f6dac154d60 2024-09-19T05:44:53.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6dabfff640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dac154d60 0x7f6dac1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dac154d60 msgr2=0x7f6dac1f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6da2ffd640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dac154d60 0x7f6dac1f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dac06c040 msgr2=0x7f6dac155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6da2ffd640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dac06c040 0x7f6dac155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6da2ffd640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac148fb0 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6dabfff640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dac154d60 0x7f6dac1f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6dab7fe640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dac06c040 0x7f6dac155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.215+0000 7f6da2ffd640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac0c8ee0 0x7f6dac14a930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009890 tx=0x7f6d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.219 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:53.220 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:53.220 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.216+0000 7f6da8ff9640 1 -- 172.21.15.2:0/3981186230 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94041020 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.216+0000 7f6da8ff9640 1 -- 172.21.15.2:0/3981186230 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d9403d700 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.216+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac1583d0 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.216+0000 7f6da8ff9640 1 -- 172.21.15.2:0/3981186230 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94037040 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.216+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac149700 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.217+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dac0ca7e0 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.218+0000 7f6da8ff9640 1 -- 172.21.15.2:0/3981186230 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6d94044020 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.218+0000 7f6da8ff9640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d8c06c2e0 0x7f6d8c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.218+0000 7f6da8ff9640 1 -- 172.21.15.2:0/3981186230 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5874+0+0 (secure 0 0 0) 0x7f6d940aba90 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.218+0000 7f6dab7fe640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d8c06c2e0 0x7f6d8c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.219+0000 7f6dab7fe640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d8c06c2e0 0x7f6d8c06e7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0011e0 tx=0x7f6d9c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.230 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.227+0000 7f6da8ff9640 1 -- 172.21.15.2:0/3981186230 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f6dac0ca7e0 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.233 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:53.245 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:53.243+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:53.346 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:53.344+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:53.409 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:53.409 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:53.409 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:53.409 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:53.409 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:53.423 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:53.429 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:44:53.429 INFO:tasks.workunit.client.0.smithi002.stdout: Children see throughput for 1 re-readers = 8702505.00 kB/sec 2024-09-19T05:44:53.429 INFO:tasks.workunit.client.0.smithi002.stdout: Parent sees throughput for 1 re-readers = 8700633.49 kB/sec 2024-09-19T05:44:53.429 INFO:tasks.workunit.client.0.smithi002.stdout: Min throughput per process = 8702505.00 kB/sec 2024-09-19T05:44:53.429 INFO:tasks.workunit.client.0.smithi002.stdout: Max throughput per process = 8702505.00 kB/sec 2024-09-19T05:44:53.429 INFO:tasks.workunit.client.0.smithi002.stdout: Avg throughput per process = 8702505.00 kB/sec 2024-09-19T05:44:53.430 INFO:tasks.workunit.client.0.smithi002.stdout: Min xfer = 10485760.00 kB 2024-09-19T05:44:53.456 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.454+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f6dac0735c0 con 0x7f6dac0c8ee0 2024-09-19T05:44:53.456 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:53.454+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:53.549 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:53.547+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2140 3.3c 3:3ebd280f:::100000001f8.00000459:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.960410132, 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0038021129999999, 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2140 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641694+0000", 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049504800000000002 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646652+0000", 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049580040000000002 2024-09-19T05:44:53.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646670+0000", 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646681+0000", 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1555e-05 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429025+0000", 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.782343268 2024-09-19T05:44:53.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429156+0000", 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013093299999999999 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.435554+0000", 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063981860000000002 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549237+0000", 2024-09-19T05:44:53.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113683458 2024-09-19T05:44:53.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595633+0000", 2024-09-19T05:44:53.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046396021000000003 2024-09-19T05:44:53.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595950+0000", 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000316514 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595991+0000", 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1430000000000001e-05 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2232 2.14 2:2e2fa760:::200.00000001:head [write 2396859~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.749695769999999, 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9689986130000001, 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802904+0000", 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-07 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802912+0000", 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9489999999999996e-06 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802920+0000", 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5970000000000003e-06 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802925+0000", 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5040000000000002e-06 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730249+0000", 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9273239709999999 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730352+0000", 2024-09-19T05:44:53.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102648 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730514+0000", 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016198499999999999 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755135+0000", 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024621601 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771776+0000", 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016640773000000001 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771841+0000", 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5178000000000005e-05 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771902+0000", 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0825999999999997e-05 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2233 2.14 2:2e2fa760:::200.00000001:head [write 2398939~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.749665673999999, 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969058076, 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807300+0000", 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043666679999999998 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807312+0000", 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2199e-05 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807327+0000", 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4260999999999999e-05 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807339+0000", 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737540+0000", 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9302006619999998 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737660+0000", 2024-09-19T05:44:53.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119881 2024-09-19T05:44:53.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737878+0000", 2024-09-19T05:44:53.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218614 2024-09-19T05:44:53.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755190+0000", 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017311756000000001 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771913+0000", 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016722954000000002 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771958+0000", 2024-09-19T05:44:53.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123000000000001e-05 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771992+0000", 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3657e-05 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2234 2.14 2:2e2fa760:::200.00000001:head [write 2401019~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:53.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.745211921999999, 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9661180190000001, 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:53.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807388+0000", 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-07 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807399+0000", 2024-09-19T05:44:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1338999999999999e-05 2024-09-19T05:44:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807409+0000", 2024-09-19T05:44:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:44:53.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807416+0000", 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1180000000000003e-06 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743599+0000", 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9361828380000001 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743702+0000", 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103303 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743868+0000", 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771998+0000", 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028130420999999999 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773391+0000", 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013930699999999999 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773448+0000", 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6672999999999998e-05 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773505+0000", 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7445000000000003e-05 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2235 2.14 2:2e2fa760:::200.00000001:head [write 2403099~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.745172707999998, 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9662057229999998, 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4000000000000002e-07 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807435+0000", 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.072e-06 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807478+0000", 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2460999999999999e-05 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807486+0000", 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5350000000000004e-06 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.745914+0000", 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9384274700000002 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.747669+0000", 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017549239999999999 2024-09-19T05:44:53.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.748294+0000", 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00062547500000000001 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773338+0000", 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025043950999999998 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773551+0000", 2024-09-19T05:44:53.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021271199999999999 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773599+0000", 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7876e-05 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773632+0000", 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3707000000000001e-05 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.589 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2236 2.14 2:2e2fa760:::200.00000001:head [write 2405179~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.745102590999998, 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9899550850000001, 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.590 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:53.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807502+0000", 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4850000000000003e-06 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807509+0000", 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0859999999999996e-06 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807515+0000", 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9179999999999999e-06 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755230+0000", 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9477156679999998 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755342+0000", 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111703 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755522+0000", 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000179937 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.783617+0000", 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028095472 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797341+0000", 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013723506999999999 2024-09-19T05:44:53.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797404+0000", 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797452+0000", 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7567e-05 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2237 2.14 2:2e2fa760:::200.00000001:head [write 2407259~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.745077918, 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0092068749999998, 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807522+0000", 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807530+0000", 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8930000000000005e-06 2024-09-19T05:44:53.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807577+0000", 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7064999999999997e-05 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807606+0000", 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8963e-05 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761580+0000", 2024-09-19T05:44:53.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.953974901 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761685+0000", 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010435800000000001 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761827+0000", 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142313 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792591+0000", 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030764073999999999 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816632+0000", 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024040371000000001 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816688+0000", 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5951e-05 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816728+0000", 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0636000000000002e-05 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2238 2.14 2:2e2fa760:::200.00000001:head [write 2409339~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:53.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.744980063, 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0122208439999998, 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:53.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834275+0000", 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026656197999999999 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834287+0000", 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1691e-05 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834301+0000", 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3974e-05 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834310+0000", 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8659999999999995e-06 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772040+0000", 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9377302950000002 2024-09-19T05:44:53.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772148+0000", 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108136 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772314+0000", 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016526400000000001 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792597+0000", 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020283483000000001 2024-09-19T05:44:53.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819691+0000", 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027094025000000001 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819771+0000", 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0022000000000001e-05 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819840+0000", 2024-09-19T05:44:53.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8889999999999999e-05 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2239 2.14 2:2e2fa760:::200.00000001:head [write 2411419~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.71827008, 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.985646596, 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851429+0000", 2024-09-19T05:44:53.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017099436999999999 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851444+0000", 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5449000000000001e-05 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851460+0000", 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.556e-05 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851470+0000", 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0001e-05 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773666+0000", 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9221966369999999 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773757+0000", 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1160000000000001e-05 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773902+0000", 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014419500000000001 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795500+0000", 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021598346000000001 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819898+0000", 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024398474 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819942+0000", 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3151000000000001e-05 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819976+0000", 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4186000000000001e-05 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2240 2.14 2:2e2fa760:::200.00000001:head [write 2413499~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.701111357999999, 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9685921560000001, 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862188+0000", 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010699774 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862213+0000", 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5536000000000001e-05 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862225+0000", 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1328000000000001e-05 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862231+0000", 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.55e-06 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.780995+0000", 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.918763854 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781115+0000", 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120538 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781261+0000", 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145785 2024-09-19T05:44:53.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795505+0000", 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014244158 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820012+0000", 2024-09-19T05:44:53.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024506607 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820051+0000", 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8597000000000003e-05 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820080+0000", 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9428999999999999e-05 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2245 2.14 2:2e2fa760:::200.00000001:head [write 2423899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.690235442999999, 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969609441, 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:53.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.08e-07 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862368+0000", 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7929999999999999e-06 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862375+0000", 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6680000000000004e-06 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862380+0000", 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797491+0000", 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9351107490000001 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797581+0000", 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9559000000000001e-05 2024-09-19T05:44:53.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797723+0000", 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014230199999999999 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820590+0000", 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022867386 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827763+0000", 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071727190000000001 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.828797+0000", 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010343570000000001 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831973+0000", 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003175925 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2246 2.14 2:2e2fa760:::200.00000001:head [write 2425979~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.690211528999999, 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9696936549999999, 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.625 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.96e-07 2024-09-19T05:44:53.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862397+0000", 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862404+0000", 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862409+0000", 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1320000000000002e-06 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804733+0000", 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.94232443 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804840+0000", 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106724 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804998+0000", 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015801500000000001 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826045+0000", 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021046664999999999 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831983+0000", 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059387570000000002 2024-09-19T05:44:53.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832038+0000", 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4475999999999998e-05 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832081+0000", 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3591000000000003e-05 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2247 2.14 2:2e2fa760:::200.00000001:head [write 2428059~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.690184448, 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969738048, 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:53.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862426+0000", 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1090999999999999e-05 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862433+0000", 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8070000000000002e-06 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862438+0000", 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1599999999999997e-06 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806006+0000", 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.943567885 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806105+0000", 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8782000000000004e-05 2024-09-19T05:44:53.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806289+0000", 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183678 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827739+0000", 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021450258999999999 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832087+0000", 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043483280000000003 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832124+0000", 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6816000000000001e-05 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832153+0000", 2024-09-19T05:44:53.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.887e-05 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2248 2.14 2:2e2fa760:::200.00000001:head [write 2430139~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.690154450000001, 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9698627229999999, 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877451+0000", 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015005956000000001 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877461+0000", 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0034000000000001e-05 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.896137+0000", 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018675967000000002 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909217+0000", 2024-09-19T05:44:53.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013080629999999999 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807553+0000", 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8983351339999999 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807647+0000", 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4289000000000005e-05 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807794+0000", 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000147467 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832160+0000", 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024365753 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832224+0000", 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3689000000000006e-05 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832274+0000", 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0649000000000002e-05 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832308+0000", 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3154999999999997e-05 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2250 2.14 2:2e2fa760:::200.00000001:head [write 2434299~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.6427136, 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9595307040000001, 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939724+0000", 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029838587 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939743+0000", 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9077e-05 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941640+0000", 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018968310000000001 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941651+0000", 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0379e-05 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814230+0000", 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.872579934 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814430+0000", 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199481 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814571+0000", 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014074399999999999 2024-09-19T05:44:53.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832173+0000", 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017602199999999998 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869325+0000", 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037152223999999998 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869374+0000", 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9149e-05 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869416+0000", 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2098000000000002e-05 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2337 3.3c 3:3eebf982:::100000001f8.0000051e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:53.648 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.540567459999998, 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.014058033, 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2337 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:53.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.018343+0000", 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063114770000000002 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030374+0000", 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012031086999999999 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030396+0000", 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.156e-05 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030411+0000", 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4875000000000001e-05 2024-09-19T05:44:53.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897205+0000", 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8667940299999999 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897418+0000", 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021300400000000001 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897573+0000", 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015481300000000001 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.943251+0000", 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045678424000000002 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026021+0000", 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082769669000000004 2024-09-19T05:44:53.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026059+0000", 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8296000000000001e-05 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026090+0000", 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0797999999999999e-05 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2267 2.14 2:2e2fa760:::200.00000001:head [write 2469659~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.483925192000001, 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9703313549999999, 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076283+0000", 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076092460000000001 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076295+0000", 2024-09-19T05:44:53.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1857e-05 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076309+0000", 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3661e-05 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076319+0000", 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0589e-05 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963409+0000", 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.887089915 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963490+0000", 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1137000000000006e-05 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963615+0000", 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012412999999999999 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012633+0000", 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049018171999999999 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038886+0000", 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026253216999999999 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038950+0000", 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3876999999999998e-05 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039005+0000", 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5553999999999997e-05 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2268 2.14 2:2e2fa760:::200.00000001:head [write 2471739~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.476249694, 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962740785, 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076361+0000", 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076368+0000", 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4240000000000004e-06 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076374+0000", 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0179999999999998e-06 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975236+0000", 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.898862345 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975332+0000", 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5217999999999995e-05 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975458+0000", 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125986 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012673+0000", 2024-09-19T05:44:53.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037215224999999998 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039014+0000", 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026341256 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039056+0000", 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1944999999999999e-05 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039090+0000", 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4301999999999998e-05 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2269 2.14 2:2e2fa760:::200.00000001:head [write 2473819~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.476216136000001, 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9627868230000001, 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076384+0000", 2024-09-19T05:44:53.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076392+0000", 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.174e-06 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076398+0000", 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4309999999999999e-06 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076403+0000", 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6399999999999996e-06 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976688+0000", 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9002852180000001 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976785+0000", 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6963e-05 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976951+0000", 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016594799999999999 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012708+0000", 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035757376 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039097+0000", 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026388551 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039137+0000", 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0025999999999999e-05 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039170+0000", 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3065000000000003e-05 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.671 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2270 2.14 2:2e2fa760:::200.00000001:head [write 2475899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.476190170999999, 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962839365, 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102351+0000", 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025941703999999999 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102359+0000", 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-19T05:44:53.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102372+0000", 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3749e-05 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102382+0000", 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0229999999999998e-06 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978315+0000", 2024-09-19T05:44:53.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.875933294 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978422+0000", 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107007 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978575+0000", 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015296099999999999 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012746+0000", 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034170849000000003 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039176+0000", 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026430282999999999 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039215+0000", 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9368e-05 2024-09-19T05:44:53.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039248+0000", 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3157000000000001e-05 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:53.677 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:53.679 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:53.679 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:53.677+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:53.779 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:53.777+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:53.805 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:53.843 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:53.843 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:53.843 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:53.843 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:53.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:53.895 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:53.893+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.929+0000 7f3f20dd8640 1 Processor -- start 2024-09-19T05:44:53.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.929+0000 7f3f20dd8640 1 -- start start 2024-09-19T05:44:53.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f20dd8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 0x7f3f1c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f20dd8640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f1c15aa90 0x7f3f1c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f20dd8640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f1c15ebe0 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f20dd8640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1c15eec0 con 0x7f3f1c15aa90 2024-09-19T05:44:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f20dd8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1c0b2e00 con 0x7f3f1c0cd4e0 2024-09-19T05:44:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f19d9b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f1c15aa90 0x7f3f1c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f19d9b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3f1c15aa90 0x7f3f1c0b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33414/0 (socket says 172.21.15.2:33414) 2024-09-19T05:44:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f1ad9d640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3f1c0cd8c0 0x7f3f1c15df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:51962/0 (socket says 172.21.15.2:51962) 2024-09-19T05:44:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f19d9b640 1 -- 172.21.15.2:0/99446197 learned_addr learned my addr 172.21.15.2:0/99446197 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:53.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.930+0000 7f3f1a59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 0x7f3f1c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.931+0000 7f3f19d9b640 1 -- 172.21.15.2:0/99446197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 msgr2=0x7f3f1c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.931+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/99446197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 0x7f3f1c0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.931+0000 7f3f19d9b640 1 -- 172.21.15.2:0/99446197 >> v1:172.21.15.123:6789/0 conn(0x7f3f1c0cd8c0 legacy=0x7f3f1c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.931+0000 7f3f19d9b640 1 -- 172.21.15.2:0/99446197 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c0b30e0 con 0x7f3f1c15aa90 2024-09-19T05:44:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.932+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/99446197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 0x7f3f1c0b4100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009870 tx=0x7f3f1002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3237130a32aa55b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.932+0000 7f3f1959a640 1 -- 172.21.15.2:0/99446197 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037070 con 0x7f3f1c15aa90 2024-09-19T05:44:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.932+0000 7f3f1959a640 1 -- 172.21.15.2:0/99446197 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f10032440 con 0x7f3f1c15aa90 2024-09-19T05:44:53.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.932+0000 7f3f1959a640 1 -- 172.21.15.2:0/99446197 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10032810 con 0x7f3f1c15aa90 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 -- 172.21.15.2:0/99446197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 msgr2=0x7f3f1c0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/99446197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 0x7f3f1c0b4100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009870 tx=0x7f3f1002a270 comp rx=0 tx=0).stop 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 -- 172.21.15.2:0/99446197 shutdown_connections 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/99446197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 0x7f3f1c0b4100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/99446197 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 0x7f3f1c0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 -- 172.21.15.2:0/99446197 >> 172.21.15.2:0/99446197 conn(0x7f3f1c067b30 msgr2=0x7f3f1c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 -- 172.21.15.2:0/99446197 shutdown_connections 2024-09-19T05:44:53.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.933+0000 7f3f20dd8640 1 -- 172.21.15.2:0/99446197 wait complete. 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.934+0000 7f3f20dd8640 1 Processor -- start 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f20dd8640 1 -- start start 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f20dd8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 0x7f3f1c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f20dd8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f1c0cd8c0 0x7f3f1c0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f20dd8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 0x7f3f1c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f20dd8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c15e640 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f20dd8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c0b36a0 con 0x7f3f1c15aa90 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f20dd8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1c15e7c0 con 0x7f3f1c0cd4e0 2024-09-19T05:44:53.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.935+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f1c0cd8c0 0x7f3f1c0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f1c0cd8c0 0x7f3f1c0ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39968/0 (socket says 172.21.15.2:39968) 2024-09-19T05:44:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 -- 172.21.15.2:0/315210310 learned_addr learned my addr 172.21.15.2:0/315210310 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f1a59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 0x7f3f1c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f1ad9d640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 0x7f3f1c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 msgr2=0x7f3f1c158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3f1c0cd4e0 0x7f3f1c158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 msgr2=0x7f3f1c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f1c15aa90 0x7f3f1c15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c15a330 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f1c0cd8c0 0x7f3f1c0ce280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a2b0 tx=0x7f3f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10041030 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f10030440 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.936+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c153ac0 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.937+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037040 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.937+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c15a5b0 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.938+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f3f1c0b6240 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.938+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3f10044070 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.939+0000 7f3f0affd640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3efc06c2b0 0x7f3efc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.939+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5874+0+0 (secure 0 0 0) 0x7f3f100ab9f0 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.939+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3efc06c2b0 0x7f3efc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.939+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v79) ==== 54+0+16 (secure 0 0 0) 0x7f3f1c0b6240 con 0x7f3f1c0cd8c0 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.939+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f1c153630 0x7f3f1c14b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.939+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c0ca640 con 0x7f3f1c153630 2024-09-19T05:44:53.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.940+0000 7f3f1ad9d640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f1c153630 0x7f3f1c14b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.940+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3efc06c2b0 0x7f3efc06e770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009b40 tx=0x7f3f0c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.940+0000 7f3f1ad9d640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f1c153630 0x7f3f1c14b640 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.941+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1c0ca640 con 0x7f3f1c153630 2024-09-19T05:44:53.960 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.958+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c06ce80 con 0x7f3f1c153630 2024-09-19T05:44:53.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.960+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f1c06ce80 con 0x7f3f1c153630 2024-09-19T05:44:53.962 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:53.963 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:53.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.960+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f1c1595a0 0x7f3f1c1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.960+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c14c090 con 0x7f3f1c1595a0 2024-09-19T05:44:53.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.960+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f1c1595a0 0x7f3f1c1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.960+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f1c1595a0 0x7f3f1c1f39a0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.961+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1c14c090 con 0x7f3f1c1595a0 2024-09-19T05:44:53.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.976+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c0c6fb0 con 0x7f3f1c1595a0 2024-09-19T05:44:53.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.978+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f1c0c6fb0 con 0x7f3f1c1595a0 2024-09-19T05:44:53.980 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.978+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f1c15d1b0 0x7f3f1c1f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.978+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c0cea10 con 0x7f3f1c15d1b0 2024-09-19T05:44:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.978+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f1c15d1b0 0x7f3f1c1f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.978+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f1c15d1b0 0x7f3f1c1f43f0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.982 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.979+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1c0cea10 con 0x7f3f1c15d1b0 2024-09-19T05:44:53.982 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:53.980+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:53.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.993+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c062900 con 0x7f3f1c15d1b0 2024-09-19T05:44:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.995+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f1c062900 con 0x7f3f1c15d1b0 2024-09-19T05:44:53.997 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.995+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f1c151c10 0x7f3f1c152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.995+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c0633e0 con 0x7f3f1c151c10 2024-09-19T05:44:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.995+0000 7f3f1ad9d640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f1c151c10 0x7f3f1c152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.995+0000 7f3f1ad9d640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f1c151c10 0x7f3f1c152050 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:53.996+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1c0633e0 con 0x7f3f1c151c10 2024-09-19T05:44:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.012+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c152860 con 0x7f3f1c151c10 2024-09-19T05:44:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.014+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f1c152860 con 0x7f3f1c151c10 2024-09-19T05:44:54.017 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:54.018 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:54.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.014+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f1c02cb50 0x7f3f1c02cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:54.018 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.014+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c0635f0 con 0x7f3f1c02cb50 2024-09-19T05:44:54.018 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:54.018 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:54.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.015+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f1c02cb50 0x7f3f1c02cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:54.019 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.015+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f1c02cb50 0x7f3f1c02cfd0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:54.020 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.016+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f0c004140 con 0x7f3f1c02cb50 2024-09-19T05:44:54.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.031+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c063410 con 0x7f3f1c02cb50 2024-09-19T05:44:54.036 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.033+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f1c063410 con 0x7f3f1c02cb50 2024-09-19T05:44:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.034+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f3f1c204d20 0x7f3f1c207110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.034+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c02db20 con 0x7f3f1c204d20 2024-09-19T05:44:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.034+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f3f1c204d20 0x7f3f1c207110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.035+0000 7f3f19d9b640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f3f1c204d20 0x7f3f1c207110 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.036+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.5 v2:172.21.15.123:6800/2989037926 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1c02db20 con 0x7f3f1c204d20 2024-09-19T05:44:54.053 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.051+0000 7f6da8ff9640 1 -- 172.21.15.2:0/3981186230 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v80) ==== 97+0+0 (secure 0 0 0) 0x7f6d9404b030 con 0x7f6dac0c8ee0 2024-09-19T05:44:54.054 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.051+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c1525f0 con 0x7f3f1c204d20 2024-09-19T05:44:54.054 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-19T05:44:54.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d8c06c2e0 msgr2=0x7f6d8c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d8c06c2e0 0x7f6d8c06e7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0011e0 tx=0x7f6d9c009040 comp rx=0 tx=0).stop 2024-09-19T05:44:54.056 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac0c8ee0 msgr2=0x7f6dac14a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac0c8ee0 0x7f6dac14a930 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009890 tx=0x7f6d940046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 shutdown_connections 2024-09-19T05:44:54.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f6d8c06c2e0 0x7f6d8c06e7a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dac154d60 0x7f6dac1f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6dac0c8ee0 0x7f6dac14a930 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 --2- 172.21.15.2:0/3981186230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dac06c040 0x7f6dac155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 >> 172.21.15.2:0/3981186230 conn(0x7f6dac06f3d0 msgr2=0x7f6dac15bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:54.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.054+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 shutdown_connections 2024-09-19T05:44:54.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.055+0000 7f6db1a74640 1 -- 172.21.15.2:0/3981186230 wait complete. 2024-09-19T05:44:54.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.056+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.5 v2:172.21.15.123:6800/2989037926 2 ==== command_reply(tid 12: 0 osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' ) ==== 503+0+3 (crc 0 0 0) 0x7f3f1c1525f0 con 0x7f3f1c204d20 2024-09-19T05:44:54.058 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:54.059 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-19T05:44:54.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.057+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f1c207b60 0x7f3f1c207f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:54.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.057+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c209bd0 con 0x7f3f1c207b60 2024-09-19T05:44:54.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.058+0000 7f3f1ad9d640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f1c207b60 0x7f3f1c207f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:54.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.059+0000 7f3f1ad9d640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f1c207b60 0x7f3f1c207f80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.061+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1c209bd0 con 0x7f3f1c207b60 2024-09-19T05:44:54.078 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.076+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c2084c0 con 0x7f3f1c207b60 2024-09-19T05:44:54.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.078+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f1c2084c0 con 0x7f3f1c207b60 2024-09-19T05:44:54.081 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:54.081 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.079+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f1c208b40 0x7f3f1c208f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.079+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f3f1c2095f0 con 0x7f3f1c208b40 2024-09-19T05:44:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.079+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f1c208b40 0x7f3f1c208f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.080+0000 7f3f1a59c640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f1c208b40 0x7f3f1c208f80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:54.083 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:54.081+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.081+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1c2095f0 con 0x7f3f1c208b40 2024-09-19T05:44:54.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.100+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3f1c075590 con 0x7f3f1c208b40 2024-09-19T05:44:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f0affd640 1 -- 172.21.15.2:0/315210310 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3f1c075590 con 0x7f3f1c208b40 2024-09-19T05:44:54.104 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f1c153630 msgr2=0x7f3f1c14b640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f3f1c153630 0x7f3f1c14b640 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f1c1595a0 msgr2=0x7f3f1c1f39a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f3f1c1595a0 0x7f3f1c1f39a0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f1c15d1b0 msgr2=0x7f3f1c1f43f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f3f1c15d1b0 0x7f3f1c1f43f0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f1c151c10 msgr2=0x7f3f1c152050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f3f1c151c10 0x7f3f1c152050 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f1c02cb50 msgr2=0x7f3f1c02cfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f3f1c02cb50 0x7f3f1c02cfd0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f3f1c204d20 msgr2=0x7f3f1c207110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f3f1c204d20 0x7f3f1c207110 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f1ad9d640 1 -- 172.21.15.2:0/315210310 reap_dead start 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f1c207b60 msgr2=0x7f3f1c207f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f3f1c207b60 0x7f3f1c207f80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f1ad9d640 1 -- 172.21.15.2:0/315210310 reap_dead start 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f1ad9d640 1 -- 172.21.15.2:0/315210310 reap_dead start 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f1c208b40 msgr2=0x7f3f1c208f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f1c208b40 0x7f3f1c208f80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.102+0000 7f3f1ad9d640 1 -- 172.21.15.2:0/315210310 reap_dead start 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3efc06c2b0 msgr2=0x7f3efc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3efc06c2b0 0x7f3efc06e770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009b40 tx=0x7f3f0c009190 comp rx=0 tx=0).stop 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f1c0cd8c0 msgr2=0x7f3f1c0ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f1c0cd8c0 0x7f3f1c0ce280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a2b0 tx=0x7f3f100046c0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 shutdown_connections 2024-09-19T05:44:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f3f1c208b40 0x7f3f1c208f80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3efc06c2b0 0x7f3efc06e770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 --2- 172.21.15.2:0/315210310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f1c0cd8c0 0x7f3f1c0ce280 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 >> 172.21.15.2:0/315210310 conn(0x7f3f1c067b30 msgr2=0x7f3f1c0cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 shutdown_connections 2024-09-19T05:44:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:54.103+0000 7f3f20dd8640 1 -- 172.21.15.2:0/315210310 wait complete. 2024-09-19T05:44:54.167 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:54.167 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:54.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:54.184 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:54.182+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:54.285 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:54.283+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:54.311 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:54.311 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:54.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:54.409 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:54.407+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:54.487 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:54.485+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:54.488 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:54.488 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:54.499 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:54.588 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:54.586+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:54.679 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:54.679 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:54.688 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:54.687+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:54.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:54.789 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:54.788+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:54.862 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:54.862 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:54.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:54.915 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:54.913+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:54.991 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:54.990+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:55.037 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:55.037 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:55.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:55.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:55.092 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:55.091+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:55.193 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:55.192+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:55.207 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:55.207 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:55.214 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:55.214 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:55.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:55.309 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:55.308+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:55.390 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:55.390 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:55.393 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:55.392+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:55.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:55.494 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:55.493+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:55.575 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:55.575 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:55.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:55.627 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:55.626+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:55.696 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:55.694+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:55.741 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:55.741 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:55.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:44:55.797 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:55.796+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:55.898 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:55.896+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:55.902 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:55.902 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:55.911 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:44:55.998 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:55.997+0000 7f9b2dcb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 51427) UID: 0 2024-09-19T05:44:56.066 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:56.066 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:56.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:44:56.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:44:56.117 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:56.115+0000 7f9b2dcb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 51427) UID: 0 2024-09-19T05:44:56.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:44:56.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.186+0000 7f2a50eb7640 1 Processor -- start 2024-09-19T05:44:56.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.186+0000 7f2a50eb7640 1 -- start start 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a50eb7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c06ea80 0x7f2a4c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a50eb7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2a4c071110 0x7f2a4c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a50eb7640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4c0c98d0 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a50eb7640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f2a4c070cf0 con 0x7f2a4c071110 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a50eb7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4c0706a0 con 0x7f2a4c06ea80 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a4ad9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2a4c071110 0x7f2a4c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a4ad9d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2a4c071110 0x7f2a4c0c53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39982/0 (socket says 172.21.15.2:39982) 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.187+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/1047798699 learned_addr learned my addr 172.21.15.2:0/1047798699 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:56.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.188+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/1047798699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c06ea80 msgr2=0x7f2a4c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:56.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.188+0000 7f2a4ad9d640 1 --2- 172.21.15.2:0/1047798699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c06ea80 0x7f2a4c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.188+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/1047798699 >> v1:172.21.15.123:6789/0 conn(0x7f2a4c0b0ee0 legacy=0x7f2a4c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:56.190 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.188+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/1047798699 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c070980 con 0x7f2a4c071110 2024-09-19T05:44:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.188+0000 7f2a4ad9d640 1 --2- 172.21.15.2:0/1047798699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c071110 0x7f2a4c0c53c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009e50 tx=0x7f2a4402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad09507dc8f7fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.188+0000 7f2a49d9b640 1 -- 172.21.15.2:0/1047798699 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44036070 con 0x7f2a4c071110 2024-09-19T05:44:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a49d9b640 1 -- 172.21.15.2:0/1047798699 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a44031600 con 0x7f2a4c071110 2024-09-19T05:44:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a49d9b640 1 -- 172.21.15.2:0/1047798699 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a440319d0 con 0x7f2a4c071110 2024-09-19T05:44:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 -- 172.21.15.2:0/1047798699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c071110 msgr2=0x7f2a4c0c53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/1047798699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c071110 0x7f2a4c0c53c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009e50 tx=0x7f2a4402a6c0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 -- 172.21.15.2:0/1047798699 shutdown_connections 2024-09-19T05:44:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/1047798699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c071110 0x7f2a4c0c53c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/1047798699 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c06ea80 0x7f2a4c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 -- 172.21.15.2:0/1047798699 >> 172.21.15.2:0/1047798699 conn(0x7f2a4c06c240 msgr2=0x7f2a4c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 -- 172.21.15.2:0/1047798699 shutdown_connections 2024-09-19T05:44:56.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.189+0000 7f2a50eb7640 1 -- 172.21.15.2:0/1047798699 wait complete. 2024-09-19T05:44:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.191+0000 7f2a50eb7640 1 Processor -- start 2024-09-19T05:44:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.191+0000 7f2a50eb7640 1 -- start start 2024-09-19T05:44:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a50eb7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c06ea80 0x7f2a4c0c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a50eb7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c071110 0x7f2a4c0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a50eb7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c0b0ee0 0x7f2a4c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a50eb7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c0c9bb0 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a50eb7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c0b12c0 con 0x7f2a4c06ea80 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a50eb7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2a4c0ceca0 con 0x7f2a4c071110 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c0b0ee0 0x7f2a4c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c0b0ee0 0x7f2a4c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:39994/0 (socket says 172.21.15.2:39994) 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/519203775 learned_addr learned my addr 172.21.15.2:0/519203775 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a43fff640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c071110 0x7f2a4c0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4a59c640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c06ea80 0x7f2a4c0c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c071110 msgr2=0x7f2a4c0c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c071110 0x7f2a4c0c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c06ea80 msgr2=0x7f2a4c0c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c06ea80 0x7f2a4c0c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 -- 172.21.15.2:0/519203775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c0c6140 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a43fff640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c071110 0x7f2a4c0c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:56.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4a59c640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c06ea80 0x7f2a4c0c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:44:56.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.192+0000 7f2a4ad9d640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c0b0ee0 0x7f2a4c0b2160 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2a44034040 tx=0x7f2a44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.193+0000 7f2a42ffd640 1 -- 172.21.15.2:0/519203775 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44041020 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.193+0000 7f2a42ffd640 1 -- 172.21.15.2:0/519203775 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a44007780 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.193+0000 7f2a42ffd640 1 -- 172.21.15.2:0/519203775 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a44036040 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.193+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4c1f17a0 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.196 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.193+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4c1f14c0 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.194+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4c15b630 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.195+0000 7f2a42ffd640 1 -- 172.21.15.2:0/519203775 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2a44044020 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.195+0000 7f2a42ffd640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a2c06c2b0 0x7f2a2c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.195+0000 7f2a42ffd640 1 -- 172.21.15.2:0/519203775 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5799+0+0 (secure 0 0 0) 0x7f2a440ac070 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.195+0000 7f2a4a59c640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a2c06c2b0 0x7f2a2c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.197 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.196+0000 7f2a4a59c640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a2c06c2b0 0x7f2a2c06e770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c006fd0 tx=0x7f2a3c009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.201 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:56.199+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:56.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.200+0000 7f2a42ffd640 1 -- 172.21.15.2:0/519203775 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2a4404b030 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.230+0000 7fb58305e640 1 Processor -- start 2024-09-19T05:44:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.230+0000 7fb58305e640 1 -- start start 2024-09-19T05:44:56.232 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:56.232 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb58305e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd4e0 0x7fb57c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb58305e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb57c15ebe0 con 0x7fb57c0cd8c0 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb58305e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb57c0b2e00 con 0x7fb57c15aa90 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb58305e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c15eec0 con 0x7fb57c0cd4e0 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb5815fb640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb57c0cd8c0 0x7fb57c0c9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:50862/0 (socket says 172.21.15.2:50862) 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb5815fb640 1 -- 172.21.15.2:0/1882204049 learned_addr learned my addr 172.21.15.2:0/1882204049 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1882204049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd4e0 0x7fb57c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb57b7fe640 1 -- 172.21.15.2:0/1882204049 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718994721 0 0) 0x7fb57c15ebe0 con 0x7fb57c0cd8c0 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb57b7fe640 1 -- 172.21.15.2:0/1882204049 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb56c003520 con 0x7fb57c0cd8c0 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.231+0000 7fb57b7fe640 1 -- 172.21.15.2:0/1882204049 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239611575 0 0) 0x7fb57c0b2e00 con 0x7fb57c15aa90 2024-09-19T05:44:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.232+0000 7fb57b7fe640 1 -- 172.21.15.2:0/1882204049 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb57c15ebe0 con 0x7fb57c15aa90 2024-09-19T05:44:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.232+0000 7fb580dfa640 1 -- 172.21.15.2:0/1882204049 >> v1:172.21.15.123:6789/0 conn(0x7fb57c15aa90 legacy=0x7fb57c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.232+0000 7fb580dfa640 1 -- 172.21.15.2:0/1882204049 >> v1:172.21.15.2:6789/0 conn(0x7fb57c0cd8c0 legacy=0x7fb57c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.232+0000 7fb580dfa640 1 -- 172.21.15.2:0/1882204049 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c0b30e0 con 0x7fb57c0cd4e0 2024-09-19T05:44:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.232+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1882204049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd4e0 0x7fb57c0b3bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb5700097d0 tx=0x7fb57002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6568d50af553e672 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb57b7fe640 1 -- 172.21.15.2:0/1882204049 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570037070 con 0x7fb57c0cd4e0 2024-09-19T05:44:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb57b7fe640 1 -- 172.21.15.2:0/1882204049 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570032440 con 0x7fb57c0cd4e0 2024-09-19T05:44:56.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb57b7fe640 1 -- 172.21.15.2:0/1882204049 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570032810 con 0x7fb57c0cd4e0 2024-09-19T05:44:56.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb58305e640 1 -- 172.21.15.2:0/1882204049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd4e0 msgr2=0x7fb57c0b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb58305e640 1 --2- 172.21.15.2:0/1882204049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd4e0 0x7fb57c0b3bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb5700097d0 tx=0x7fb57002a210 comp rx=0 tx=0).stop 2024-09-19T05:44:56.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb58305e640 1 -- 172.21.15.2:0/1882204049 shutdown_connections 2024-09-19T05:44:56.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb58305e640 1 --2- 172.21.15.2:0/1882204049 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd4e0 0x7fb57c0b3bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.236 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb58305e640 1 -- 172.21.15.2:0/1882204049 >> 172.21.15.2:0/1882204049 conn(0x7fb57c067b30 msgr2=0x7fb57c067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:56.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.233+0000 7fb58305e640 1 -- 172.21.15.2:0/1882204049 shutdown_connections 2024-09-19T05:44:56.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.234+0000 7fb58305e640 1 -- 172.21.15.2:0/1882204049 wait complete. 2024-09-19T05:44:56.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.235+0000 7fb58305e640 1 Processor -- start 2024-09-19T05:44:56.237 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.235+0000 7fb58305e640 1 -- start start 2024-09-19T05:44:56.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb58305e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0cd4e0 0x7fb57c0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb58305e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd8c0 0x7fb57c14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb58305e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb57c15aa90 0x7fb57c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb58305e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15e640 con 0x7fb57c15aa90 2024-09-19T05:44:56.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb58305e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c0b36a0 con 0x7fb57c0cd4e0 2024-09-19T05:44:56.243 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb58305e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c15e7c0 con 0x7fb57c0cd8c0 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb57bfff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd8c0 0x7fb57c14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb5815fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb57c15aa90 0x7fb57c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb580dfa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0cd4e0 0x7fb57c0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb57bfff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd8c0 0x7fb57c14cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:59990/0 (socket says 172.21.15.2:59990) 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb580dfa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0cd4e0 0x7fb57c0cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:33446/0 (socket says 172.21.15.2:33446) 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb57bfff640 1 -- 172.21.15.2:0/1223598747 learned_addr learned my addr 172.21.15.2:0/1223598747 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb5815fb640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd8c0 msgr2=0x7fb57c14cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb5815fb640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb57c0cd8c0 0x7fb57c14cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb5815fb640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0cd4e0 msgr2=0x7fb57c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb5815fb640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb57c0cd4e0 0x7fb57c0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.236+0000 7fb5815fb640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c1f4470 con 0x7fb57c15aa90 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.237+0000 7fb5815fb640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb57c15aa90 0x7fb57c152c60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0cdcd0 tx=0x7fb57400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.244 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.237+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574017070 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.237+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57400aba0 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.237+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c14de00 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.237+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574012400 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.237+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c0c8a90 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.238+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb57c14cbe0 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.239+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb57401a070 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.239+0000 7fb5797fa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb56006c2b0 0x7fb56006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.239+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5799+0+0 (secure 0 0 0) 0x7fb57408b840 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.239+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v82) ==== 54+0+16 (secure 0 0 0) 0x7fb57408c0c0 con 0x7fb57c15aa90 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.239+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb57c0c70b0 0x7fb57c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.239+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb57c14dbb0 con 0x7fb57c0c70b0 2024-09-19T05:44:56.245 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.239+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb56006c2b0 0x7fb56006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.240+0000 7fb57bfff640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb57c0c70b0 0x7fb57c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.240+0000 7fb57bfff640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb57c0c70b0 0x7fb57c1531a0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.240+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb56006c2b0 0x7fb56006e770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb570034000 tx=0x7fb57002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.246 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.241+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c14dbb0 con 0x7fb57c0c70b0 2024-09-19T05:44:56.247 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:44:56.260 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.259+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c15df30 con 0x7fb57c0c70b0 2024-09-19T05:44:56.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.262+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c15df30 con 0x7fb57c0c70b0 2024-09-19T05:44:56.263 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:56.264 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:56.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.262+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb57c14d4e0 0x7fb57c14b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.262+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb57c0680b0 con 0x7fb57c14d4e0 2024-09-19T05:44:56.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.262+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb57c14d4e0 0x7fb57c14b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.263+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb57c14d4e0 0x7fb57c14b310 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.264+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c0680b0 con 0x7fb57c14d4e0 2024-09-19T05:44:56.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.280+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c154ac0 con 0x7fb57c14d4e0 2024-09-19T05:44:56.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.282+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c154ac0 con 0x7fb57c14d4e0 2024-09-19T05:44:56.283 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:56.284 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:56.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.282+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb57c14bd60 0x7fb57c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.282+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb57c14e700 con 0x7fb57c14bd60 2024-09-19T05:44:56.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.282+0000 7fb57bfff640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb57c14bd60 0x7fb57c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.283+0000 7fb57bfff640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb57c14bd60 0x7fb57c062630 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.284+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c14e700 con 0x7fb57c14bd60 2024-09-19T05:44:56.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.298+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c14c140 con 0x7fb57c14bd60 2024-09-19T05:44:56.301 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:56.300+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:56.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.301+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c14c140 con 0x7fb57c14bd60 2024-09-19T05:44:56.303 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:56.303 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:56.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.301+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb57c0632d0 0x7fb57c0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.302+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb57c0c7e50 con 0x7fb57c0632d0 2024-09-19T05:44:56.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.302+0000 7fb5815fb640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb57c0632d0 0x7fb57c0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.302+0000 7fb5815fb640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb57c0632d0 0x7fb57c0c7910 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.303+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c0c7e50 con 0x7fb57c0632d0 2024-09-19T05:44:56.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.339+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c0c8300 con 0x7fb57c0632d0 2024-09-19T05:44:56.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.340+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c0c8300 con 0x7fb57c0632d0 2024-09-19T05:44:56.341 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:56.342 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:56.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.341+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb57c02c640 0x7fb57c02ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.341+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb57c0c8280 con 0x7fb57c02c640 2024-09-19T05:44:56.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.341+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb57c02c640 0x7fb57c02ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.342+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb57c02c640 0x7fb57c02ca80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.343 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.342+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c0c8280 con 0x7fb57c02c640 2024-09-19T05:44:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.358+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c02cfc0 con 0x7fb57c02c640 2024-09-19T05:44:56.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.359+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c02cfc0 con 0x7fb57c02c640 2024-09-19T05:44:56.361 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.360+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb57c02d4d0 0x7fb57c206ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.360+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb57c02d9e0 con 0x7fb57c02d4d0 2024-09-19T05:44:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.360+0000 7fb57bfff640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb57c02d4d0 0x7fb57c206ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.360+0000 7fb57bfff640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb57c02d4d0 0x7fb57c206ca0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.361+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.5 v2:172.21.15.123:6800/2989037926 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c02d9e0 con 0x7fb57c02d4d0 2024-09-19T05:44:56.363 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:56.364 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:56.364 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:56.364 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:56.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:44:56.378 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.377+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c02d930 con 0x7fb57c02d4d0 2024-09-19T05:44:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.379+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.5 v2:172.21.15.123:6800/2989037926 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c02d930 con 0x7fb57c02d4d0 2024-09-19T05:44:56.380 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.379+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb57c2076f0 0x7fb57c209ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.379+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb57c20ca30 con 0x7fb57c2076f0 2024-09-19T05:44:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.379+0000 7fb5815fb640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb57c2076f0 0x7fb57c209ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.380+0000 7fb5815fb640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb57c2076f0 0x7fb57c209ae0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.381+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c20ca30 con 0x7fb57c2076f0 2024-09-19T05:44:56.402 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:56.400+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:56.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.415+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c0c8080 con 0x7fb57c2076f0 2024-09-19T05:44:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.417+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c0c8080 con 0x7fb57c2076f0 2024-09-19T05:44:56.418 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:56.419 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:56.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.417+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb57c20a580 0x7fb57c20a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:56.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.417+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb57c20fe00 con 0x7fb57c20a580 2024-09-19T05:44:56.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.417+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb57c20a580 0x7fb57c20a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:56.419 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.418+0000 7fb580dfa640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb57c20a580 0x7fb57c20a9e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:56.420 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.419+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb57c20fe00 con 0x7fb57c20a580 2024-09-19T05:44:56.453 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.452+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb57c20afe0 con 0x7fb57c20a580 2024-09-19T05:44:56.455 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb5797fa640 1 -- 172.21.15.2:0/1223598747 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb57c20afe0 con 0x7fb57c20a580 2024-09-19T05:44:56.455 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:56.456 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb57c0c70b0 msgr2=0x7fb57c1531a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb57c0c70b0 0x7fb57c1531a0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb57c14d4e0 msgr2=0x7fb57c14b310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb57c14d4e0 0x7fb57c14b310 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb57c14bd60 msgr2=0x7fb57c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb57c14bd60 0x7fb57c062630 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb57c0632d0 msgr2=0x7fb57c0c7910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb57c0632d0 0x7fb57c0c7910 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb57c02c640 msgr2=0x7fb57c02ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb5815fb640 1 -- 172.21.15.2:0/1223598747 reap_dead start 2024-09-19T05:44:56.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb57c02c640 0x7fb57c02ca80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb5815fb640 1 -- 172.21.15.2:0/1223598747 reap_dead start 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb57c02d4d0 msgr2=0x7fb57c206ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb57c02d4d0 0x7fb57c206ca0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb57c2076f0 msgr2=0x7fb57c209ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb57c2076f0 0x7fb57c209ae0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb57c20a580 msgr2=0x7fb57c20a9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb57c20a580 0x7fb57c20a9e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb56006c2b0 msgr2=0x7fb56006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb56006c2b0 0x7fb56006e770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb570034000 tx=0x7fb57002e000 comp rx=0 tx=0).stop 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb57c15aa90 msgr2=0x7fb57c152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 --2- 172.21.15.2:0/1223598747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb57c15aa90 0x7fb57c152c60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0cdcd0 tx=0x7fb57400cf00 comp rx=0 tx=0).stop 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb5815fb640 1 -- 172.21.15.2:0/1223598747 reap_dead start 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb5815fb640 1 -- 172.21.15.2:0/1223598747 reap_dead start 2024-09-19T05:44:56.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 shutdown_connections 2024-09-19T05:44:56.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 >> 172.21.15.2:0/1223598747 conn(0x7fb57c067b30 msgr2=0x7fb57c153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:56.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.454+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 shutdown_connections 2024-09-19T05:44:56.459 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.455+0000 7fb58305e640 1 -- 172.21.15.2:0/1223598747 wait complete. 2024-09-19T05:44:56.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.474+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f2a4c070ef0 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:56.496 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:56.496 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:56.496 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:56.496 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:56.502 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:56.501+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:56.505 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:44:56.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.557+0000 7f2a42ffd640 1 -- 172.21.15.2:0/519203775 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v83) ==== 91+0+0 (secure 0 0 0) 0x7f2a4407a810 con 0x7f2a4c0b0ee0 2024-09-19T05:44:56.558 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:44:56.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a2c06c2b0 msgr2=0x7f2a2c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a2c06c2b0 0x7f2a2c06e770 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c006fd0 tx=0x7f2a3c009060 comp rx=0 tx=0).stop 2024-09-19T05:44:56.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c0b0ee0 msgr2=0x7f2a4c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:56.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c0b0ee0 0x7f2a4c0b2160 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2a44034040 tx=0x7f2a44004a40 comp rx=0 tx=0).stop 2024-09-19T05:44:56.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 shutdown_connections 2024-09-19T05:44:56.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a2c06c2b0 0x7f2a2c06e770 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a4c0b0ee0 0x7f2a4c0b2160 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a4c071110 0x7f2a4c0c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 --2- 172.21.15.2:0/519203775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a4c06ea80 0x7f2a4c0c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:56.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 >> 172.21.15.2:0/519203775 conn(0x7f2a4c06c240 msgr2=0x7f2a4c0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:56.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 shutdown_connections 2024-09-19T05:44:56.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:56.560+0000 7f2a50eb7640 1 -- 172.21.15.2:0/519203775 wait complete. 2024-09-19T05:44:56.603 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:56.602+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:56.660 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2637 3.3 3:c38cdf66:::100000001f8.0000064a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.827237189999998, 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.86377266600000002, 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2637 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829833+0000", 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4700000000000004e-06 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834195+0000", 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043615379999999999 2024-09-19T05:44:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834217+0000", 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1713e-05 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834229+0000", 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2418e-05 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.139987+0000", 2024-09-19T05:44:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.305758321 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140183+0000", 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195577 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140385+0000", 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020212799999999999 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.465853+0000", 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32546820700000001 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693468+0000", 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22761451199999999 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693549+0000", 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1267999999999995e-05 2024-09-19T05:44:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693602+0000", 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2513999999999999e-05 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2681 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.674746455000001, 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.077969779, 2024-09-19T05:44:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2681 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982323+0000", 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2370000000000001e-06 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986715+0000", 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004392618 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986751+0000", 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5398999999999998e-05 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986762+0000", 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1545999999999999e-05 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706073+0000", 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71931063299999998 2024-09-19T05:44:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706251+0000", 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017771099999999999 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060111+0000", 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35386035900000001 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060200+0000", 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9259999999999996e-05 2024-09-19T05:44:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060289+0000", 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9016000000000003e-05 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:3095 3.33 e71/20 3:cd66a262:::100000001f8.00000814:head v 71'69, mlcod=70'60)", 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 24.345929504000001, 2024-09-19T05:44:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80810778000000005, 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311141+0000", 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.284e-06 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318008+0000", 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068671770000000004 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318032+0000", 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4467000000000001e-05 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318051+0000", 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8284e-05 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693668+0000", 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37561766600000002 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693691+0000", 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2875e-05 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116967+0000", 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42327609900000002 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.119244+0000", 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022769280000000001 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5152 3.2c 3:350c5391:::100000001f8.0000061d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.804038839, 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80414443700000005, 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5152 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853033+0000", 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6400000000000002e-06 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856724+0000", 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036910649999999999 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856747+0000", 2024-09-19T05:44:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2719999999999999e-05 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856759+0000", 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2679e-05 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948007+0000", 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091247606999999994 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948133+0000", 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012556500000000001 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948378+0000", 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024540299999999999 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.240715+0000", 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29233733899999997 2024-09-19T05:44:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657035+0000", 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41632016100000002 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657101+0000", 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691999999999994e-05 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657172+0000", 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0566000000000002e-05 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5153 3.2c 3:35ced6af:::100000001f8.0000061e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.800276941, 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84790699300000005, 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5153 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856791+0000", 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2239999999999998e-06 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859552+0000", 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027601930000000002 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859564+0000", 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2880999999999999e-05 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859579+0000", 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4925e-05 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145653+0000", 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.286073313 2024-09-19T05:44:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145820+0000", 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166937 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.167125+0000", 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021305001 2024-09-19T05:44:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.270722+0000", 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103597041 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704576+0000", 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43385462400000002 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704637+0000", 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1118000000000007e-05 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704696+0000", 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8736000000000001e-05 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5229 3.34 3:2ebd7ea6:::100000001f8.0000066a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.724427317, 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90816846299999998, 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5229 2024-09-19T05:44:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932642+0000", 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5820000000000002e-06 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937356+0000", 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047141400000000003 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937389+0000", 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2929999999999998e-05 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937402+0000", 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2855e-05 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268202+0000", 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.330799856 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268403+0000", 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200464 2024-09-19T05:44:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268639+0000", 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000236598 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.656059+0000", 2024-09-19T05:44:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.387419719 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840676+0000", 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18461746900000001 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840747+0000", 2024-09-19T05:44:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0462000000000004e-05 2024-09-19T05:44:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840807+0000", 2024-09-19T05:44:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0387999999999998e-05 2024-09-19T05:44:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5231 3.3b 3:ddf1e7a4:::100000001f8.0000066c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.602804204, 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1076528050000001, 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5231 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:44:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054273+0000", 2024-09-19T05:44:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1253999999999999e-05 2024-09-19T05:44:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059647+0000", 2024-09-19T05:44:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053742479999999999 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059676+0000", 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8699000000000001e-05 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059689+0000", 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2476999999999999e-05 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.863904+0000", 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.804215652 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.864024+0000", 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119848 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161715+0000", 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.297691133 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161795+0000", 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9671000000000003e-05 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161915+0000", 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011982299999999999 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5241 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.557979248000001, 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.063036227, 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5241 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099090+0000", 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:44:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105109+0000", 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006018809 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105132+0000", 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2908000000000002e-05 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105143+0000", 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1826e-05 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.924904+0000", 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81976026999999996 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925014+0000", 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011063 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162023+0000", 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23700855300000001 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162060+0000", 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6715999999999999e-05 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162123+0000", 2024-09-19T05:44:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3480999999999997e-05 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5250 3.3 3:c0cd83f8:::100000001f8.0000067f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.522042921000001, 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026991652, 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5250 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135025+0000", 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3379999999999999e-06 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137223+0000", 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021978639999999999 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137247+0000", 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3978999999999999e-05 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.144008+0000", 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067608770000000002 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915821+0000", 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77181293399999995 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915914+0000", 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2493999999999999e-05 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.916430+0000", 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00051667599999999998 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.122065+0000", 2024-09-19T05:44:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20563482999999999 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161930+0000", 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039865356999999997 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161977+0000", 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6873999999999999e-05 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162015+0000", 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7428999999999997e-05 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5293 3.3 3:c2b9c98f:::100000001f8.000006aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.445409339999999, 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95068157399999997, 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5293 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211659+0000", 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215320+0000", 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003660862 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215339+0000", 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8868999999999998e-05 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215351+0000", 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1802e-05 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939166+0000", 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72381503599999997 2024-09-19T05:44:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939258+0000", 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1911999999999997e-05 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939406+0000", 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014874699999999999 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162132+0000", 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.222725545 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162260+0000", 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012829700000000001 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162304+0000", 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4215000000000003e-05 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162338+0000", 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3819000000000003e-05 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5301 3.c 3:3307abed:::100000001f8.000006b2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.436423562, 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75017270800000002, 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5301 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220645+0000", 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0509999999999999e-06 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238885+0000", 2024-09-19T05:44:56.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018240014999999998 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238906+0000", 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1498999999999999e-05 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238919+0000", 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2565000000000001e-05 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841065+0000", 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60214654400000001 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841146+0000", 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0688999999999999e-05 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841342+0000", 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019650000000000001 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878211+0000", 2024-09-19T05:44:56.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036868724999999998 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970700+0000", 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092489022000000004 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970754+0000", 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3480999999999998e-05 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970815+0000", 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1617000000000003e-05 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5312 3.2c 3:36326779:::100000001f8.000006bd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.398416283, 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71229407300000003, 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5312 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258653+0000", 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277446+0000", 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018793994000000001 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277464+0000", 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7181e-05 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277476+0000", 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2167999999999999e-05 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878243+0000", 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60076709500000003 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878346+0000", 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103516 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878493+0000", 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000146205 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970686+0000", 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092192838999999999 2024-09-19T05:44:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970867+0000", 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018116700000000001 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970904+0000", 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7474000000000001e-05 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970944+0000", 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9711e-05 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5315 3.3 3:c11688a5:::100000001f8.000006c0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.379563201, 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1629463150000001, 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5315 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277505+0000", 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-06 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280350+0000", 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028443909999999999 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280373+0000", 2024-09-19T05:44:56.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3711999999999999e-05 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280386+0000", 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2252999999999999e-05 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188178+0000", 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90779182800000002 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188297+0000", 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119034 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188468+0000", 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000171139 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.319232+0000", 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13076428000000001 2024-09-19T05:44:56.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434608+0000", 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115376337 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440394+0000", 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057860999999999998 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440449+0000", 2024-09-19T05:44:56.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4761e-05 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5319 3.3b 3:de9a4780:::100000001f8.000006c4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.376655166999999, 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1541835229999999, 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5319 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:44:56.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280414+0000", 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9969999999999999e-06 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285680+0000", 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052661560000000001 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285697+0000", 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6883000000000001e-05 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285712+0000", 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4892000000000001e-05 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253316+0000", 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96760461799999997 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253448+0000", 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013193499999999999 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434435+0000", 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18098683800000001 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434490+0000", 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4833000000000002e-05 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434594+0000", 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010437099999999999 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5325 3.c 3:328b8b05:::100000001f8.000006ca:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.364561526999999, 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75472665100000003, 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5325 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292508+0000", 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3699999999999999e-06 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301580+0000", 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0090716010000000003 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301598+0000", 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8278999999999999e-05 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301610+0000", 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.184e-05 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893481+0000", 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59187088300000001 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893784+0000", 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030305399999999998 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893972+0000", 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018840600000000001 2024-09-19T05:44:56.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.972882+0000", 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.078910441999999997 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047120+0000", 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074237769999999995 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047176+0000", 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6224000000000002e-05 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047231+0000", 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4781999999999998e-05 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5327 3.2c 3:347123bb:::100000001f8.000006cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:56.758 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.355427796000001, 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93214956400000004, 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5327 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:56.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301641+0000", 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.903e-06 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.304411+0000", 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027695879999999999 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336856+0000", 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032444796999999997 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336884+0000", 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8246999999999999e-05 2024-09-19T05:44:56.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970976+0000", 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.634091768 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971067+0000", 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1092000000000001e-05 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971228+0000", 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016114800000000001 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047103+0000", 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075875451999999996 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.221630+0000", 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17452633000000001 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233731+0000", 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012101569 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233788+0000", 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6669999999999998e-05 2024-09-19T05:44:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5355 3.1c 3:38412458:::100000001f8.000006e8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.178577162, 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89709229800000001, 2024-09-19T05:44:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5355 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478493+0000", 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9929999999999997e-06 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488576+0000", 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010082702000000001 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488591+0000", 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5607000000000002e-05 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488601+0000", 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0035e-05 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047402+0000", 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55880116199999996 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047498+0000", 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5828000000000005e-05 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047655+0000", 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015719799999999999 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180902+0000", 2024-09-19T05:44:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13324624500000001 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375474+0000", 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19457263499999999 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375539+0000", 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4295999999999996e-05 2024-09-19T05:44:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375581+0000", 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2596999999999999e-05 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5365 3.34 3:2ec8418e:::100000001f8.000006f2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.168440642, 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88704814600000004, 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5365 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488628+0000", 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4760000000000001e-06 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505248+0000", 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016620078999999999 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505273+0000", 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4527000000000001e-05 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505283+0000", 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0351e-05 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073161+0000", 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56787809499999997 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073310+0000", 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014909799999999999 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073513+0000", 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203088 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180947+0000", 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10743341100000001 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375593+0000", 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19464604999999999 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375634+0000", 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1115999999999997e-05 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375674+0000", 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9854999999999997e-05 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5367 3.1c 3:38083d86:::100000001f8.000006f4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.151764871999999, 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87052496000000001, 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5367 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505303+0000", 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.198e-06 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527093+0000", 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021790053 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527114+0000", 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0774000000000001e-05 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527127+0000", 2024-09-19T05:44:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2292e-05 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110650+0000", 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58352302099999998 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110816+0000", 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166174 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.113130+0000", 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023145470000000001 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180979+0000", 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067848435999999998 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375735+0000", 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19475654100000001 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375783+0000", 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7478000000000002e-05 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375826+0000", 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3445999999999997e-05 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5404 3.2c 3:357d9333:::100000001f8.00000719:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.005923133, 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75485259100000002, 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5404 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651146+0000", 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7750000000000001e-06 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.666480+0000", 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015334166999999999 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709279+0000", 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042798738000000003 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709297+0000", 2024-09-19T05:44:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8119999999999999e-05 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242624+0000", 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53332776400000004 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242737+0000", 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112262 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:44:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242935+0000", 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019831499999999999 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405710+0000", 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.162774789 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405918+0000", 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020817599999999999 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405962+0000", 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3606999999999998e-05 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405996+0000", 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3878000000000003e-05 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:56.791 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:56.793 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:44:56.794 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:56.791+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:56.894 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:44:56.892+0000 7f9b2dcb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 51427) UID: 0 2024-09-19T05:44:56.902 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:56.902 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:56.902 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:56.910 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:44:56.995 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:56.994+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:57.019 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:57.020 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:57.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:44:57.095 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:57.093+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:57.187 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5482 3.27 3:e79b25dc:::100000001f8.00000767:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.154430440000001, 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71036533999999996, 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5482 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030320+0000", 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3929999999999998e-06 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034225+0000", 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003904678 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034278+0000", 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2534999999999998e-05 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034303+0000", 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.565e-05 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.583640+0000", 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54933696700000001 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.583850+0000", 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000210165 2024-09-19T05:44:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740511+0000", 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.156660363 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740567+0000", 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5918999999999998e-05 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740683+0000", 2024-09-19T05:44:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011667000000000001 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5507 3.27 3:e4f54558:::100000001f8.00000780:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.135662480000001, 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71559931600000004, 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5507 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049088+0000", 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4990000000000001e-06 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.070701+0000", 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021612672999999999 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087613+0000", 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016911942999999999 2024-09-19T05:44:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087634+0000", 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0835e-05 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.707112+0000", 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61947850400000004 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.707259+0000", 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014680800000000001 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764542+0000", 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057282971000000002 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764599+0000", 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6934000000000001e-05 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764685+0000", 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6149000000000005e-05 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5521 3.27 3:e73f7143:::100000001f8.0000078e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.097092735, 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76359052999999999, 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5521 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:44:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087659+0000", 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9229999999999998e-06 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107649+0000", 2024-09-19T05:44:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019990396000000001 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107667+0000", 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7958999999999999e-05 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107678+0000", 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1286e-05 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.742986+0000", 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.635308188 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.743091+0000", 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010469200000000001 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851137+0000", 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.108045961 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851180+0000", 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3192999999999998e-05 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851246+0000", 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5932000000000005e-05 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5725 3.27 3:e65abc03:::100000001f8.0000085a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:44:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.879711545999999, 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.755317554, 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5725 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:44:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305041+0000", 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1030000000000004e-06 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307067+0000", 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020258870000000001 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307085+0000", 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7931000000000002e-05 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307098+0000", 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3426e-05 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.914327+0000", 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60722910699999999 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.914435+0000", 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010719 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060107+0000", 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.145671988 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060199+0000", 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2615999999999996e-05 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060354+0000", 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015530599999999999 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5745 3.27 3:e6f7c8d2:::100000001f8.0000086e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.87762362, 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75335722800000005, 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5745 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:44:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307128+0000", 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.308414+0000", 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001286389 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.309604+0000", 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001189884 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.309621+0000", 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6572999999999999e-05 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925094+0000", 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61547286800000001 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925185+0000", 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1120999999999997e-05 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060376+0000", 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13519163200000001 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060418+0000", 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1703999999999999e-05 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060482+0000", 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4083000000000003e-05 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5763 3.27 3:e74ad6a4:::100000001f8.00000880:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.872082764, 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76551648800000005, 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5763 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312668+0000", 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.571e-06 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316538+0000", 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038699429999999998 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316577+0000", 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9180000000000001e-05 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316596+0000", 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8332999999999999e-05 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.950346+0000", 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63375050600000005 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.950512+0000", 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016611800000000001 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078037+0000", 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12752486599999999 2024-09-19T05:44:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078090+0000", 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2540000000000002e-05 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078182+0000", 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2430999999999996e-05 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5824 3.1f 3:fb07d980:::100000001f8.000008bd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.846959878, 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73478956900000003, 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5824 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337792+0000", 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0419999999999998e-06 2024-09-19T05:44:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347160+0000", 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093682149999999992 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347177+0000", 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6806000000000001e-05 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347187+0000", 2024-09-19T05:44:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0033e-05 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.949530+0000", 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60234381400000003 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.949666+0000", 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013539300000000001 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070735+0000", 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.121069458 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.072015+0000", 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012792369999999999 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.072578+0000", 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056357100000000002 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5844 3.1f 3:fac155ab:::100000001f8.000008d1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.824881601, 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73934379100000003, 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5844 2024-09-19T05:44:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359869+0000", 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.543e-06 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363862+0000", 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039926709999999997 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363876+0000", 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3951000000000001e-05 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363891+0000", 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4951e-05 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.975926+0000", 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61203488500000003 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.976042+0000", 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011658300000000001 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099026+0000", 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.122983839 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099101+0000", 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4606999999999999e-05 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099211+0000", 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010976099999999999 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5846 3.27 3:e6e86654:::100000001f8.000008d3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.820839239, 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71445892200000005, 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5846 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363912+0000", 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2819999999999999e-06 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367743+0000", 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038313359999999999 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367760+0000", 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6892e-05 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367774+0000", 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4248e-05 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.016385+0000", 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.64861128400000001 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.016504+0000", 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118838 2024-09-19T05:44:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078254+0000", 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.061749399000000003 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078303+0000", 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9301000000000001e-05 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078368+0000", 2024-09-19T05:44:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5341999999999999e-05 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5863 3.27 3:e5e0177a:::100000001f8.000008e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.816948619, 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.74712701699999995, 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5863 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:44:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367802+0000", 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6479999999999999e-06 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372984+0000", 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051814299999999999 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373015+0000", 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0611000000000002e-05 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373054+0000", 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9690000000000001e-05 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.031807+0000", 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65875312100000005 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.038739+0000", 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069312100000000001 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114756+0000", 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076017243999999998 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114836+0000", 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0644999999999998e-05 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114927+0000", 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0418000000000001e-05 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5876 3.27 3:e58ad412:::100000001f8.000008f1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.811674202000001, 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.74196762800000005, 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5876 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373077+0000", 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7520000000000001e-06 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378577+0000", 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054999849999999998 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378607+0000", 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9850999999999999e-05 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378625+0000", 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8148e-05 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060530+0000", 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68190477100000002 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060633+0000", 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010288700000000001 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114938+0000", 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054305327 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114977+0000", 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8602000000000001e-05 2024-09-19T05:44:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115042+0000", 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5304999999999999e-05 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5918 3.27 3:e7c6e76b:::100000001f8.0000091b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.790885625, 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72128804099999999, 2024-09-19T05:44:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5918 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393866+0000", 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2550000000000002e-06 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397678+0000", 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038118190000000001 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397703+0000", 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4592000000000001e-05 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397720+0000", 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7348999999999999e-05 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070763+0000", 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.67304348999999997 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070867+0000", 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103608 2024-09-19T05:44:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115051+0000", 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044183605000000001 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115088+0000", 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7394999999999997e-05 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115151+0000", 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2928000000000005e-05 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5925 3.1f 3:f886b5f6:::100000001f8.00000922:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.786991566999999, 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77643015699999995, 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5925 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397761+0000", 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.963e-06 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421926+0000", 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024164986999999999 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421953+0000", 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6874000000000001e-05 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421966+0000", 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3413e-05 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.077691+0000", 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65572456599999995 2024-09-19T05:44:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.077798+0000", 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010718299999999999 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.173642+0000", 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.095844253000000004 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174071+0000", 2024-09-19T05:44:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000428658 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174187+0000", 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011626 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5932 3.1f 3:f86d9493:::100000001f8.00000929:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:44:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.762753232, 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.85551307099999996, 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5932 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:44:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421998+0000", 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7429999999999998e-06 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432488+0000", 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010489777 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432514+0000", 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6315000000000001e-05 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432526+0000", 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1926999999999999e-05 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.102287+0000", 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66976068499999997 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.102472+0000", 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018525900000000001 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277214+0000", 2024-09-19T05:44:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17474197899999999 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277291+0000", 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7194999999999995e-05 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277508+0000", 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021719099999999999 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6011 3.27 3:e774c081:::100000001f8.00000978:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.742240044999999, 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78469899799999998, 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6011 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442511+0000", 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7300000000000001e-06 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.511020+0000", 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068508736000000001 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517290+0000", 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062700580000000002 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517329+0000", 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8649000000000002e-05 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099275+0000", 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58194679299999996 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099398+0000", 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012280099999999999 2024-09-19T05:44:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.226947+0000", 2024-09-19T05:44:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12754907100000001 2024-09-19T05:44:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.227099+0000", 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151707 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.227207+0000", 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108453 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6019 3.1f 3:f909e746:::100000001f8.00000980:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.667374383, 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76027384200000003, 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6019 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517381+0000", 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4439999999999996e-06 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.519989+0000", 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026080410000000002 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520018+0000", 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9522e-05 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520033+0000", 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5097e-05 2024-09-19T05:44:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.151257+0000", 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63122408200000002 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.151415+0000", 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158197 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277523+0000", 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.126107891 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277569+0000", 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6066999999999998e-05 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277648+0000", 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8500999999999999e-05 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6020 3.1f 3:f8dd987c:::100000001f8.00000981:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.664682097, 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75784574900000001, 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6020 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520069+0000", 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.540152+0000", 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020082744999999999 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.582892+0000", 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042740527 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.582906+0000", 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162437+0000", 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57953022099999996 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162537+0000", 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9968000000000002e-05 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277661+0000", 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11512412700000001 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277747+0000", 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6367999999999997e-05 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277912+0000", 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016495299999999999 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6052 3.27 3:e7b93799:::100000001f8.000009a1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.593110490000001, 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70507024600000001, 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6052 2024-09-19T05:44:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591641+0000", 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.695e-06 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595294+0000", 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00365373 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595319+0000", 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4411999999999999e-05 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595333+0000", 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4157e-05 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174616+0000", 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57928295299999999 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174703+0000", 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7268000000000005e-05 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296561+0000", 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.121857365 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296626+0000", 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5643999999999997e-05 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296708+0000", 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2021999999999995e-05 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6053 3.27 3:e77fb6c3:::100000001f8.000009a2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.589386765, 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70595110000000005, 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6053 2024-09-19T05:44:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595364+0000", 2024-09-19T05:44:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.452e-06 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596835+0000", 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001470636 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596860+0000", 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4909000000000002e-05 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596874+0000", 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4321000000000001e-05 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.211679+0000", 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61480495499999999 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.211794+0000", 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114678 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301141+0000", 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.089347139000000006 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301215+0000", 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4632e-05 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301313+0000", 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7378000000000002e-05 2024-09-19T05:44:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6061 3.27 3:e4baa440:::100000001f8.000009aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.587846189, 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70453015299999999, 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6061 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596905+0000", 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6000000000000001e-06 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598385+0000", 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001479878 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598409+0000", 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3751999999999999e-05 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598418+0000", 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.218690+0000", 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.62027168499999996 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.218818+0000", 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000127983 2024-09-19T05:44:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301324+0000", 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082505675000000001 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301366+0000", 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1947999999999999e-05 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301432+0000", 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6905999999999999e-05 2024-09-19T05:44:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.295 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:57.296 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:44:57.297 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:57.295+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:57.397 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:57.396+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:57.447 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:57.448 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:57.448 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:57.457 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:44:57.498 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:57.497+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:57.598 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:57.597+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:57.617 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:57.617 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:57.617 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:57.617 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:57.617 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:57.627 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:44:57.699 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:57.698+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:57.745 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:57.745 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2140 3.3c 3:3ebd280f:::100000001f8.00000459:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.150254168, 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0038021129999999, 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2140 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641694+0000", 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049504800000000002 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646652+0000", 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049580040000000002 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646670+0000", 2024-09-19T05:44:57.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646681+0000", 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1555e-05 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429025+0000", 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.782343268 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429156+0000", 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013093299999999999 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.435554+0000", 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063981860000000002 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549237+0000", 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113683458 2024-09-19T05:44:57.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595633+0000", 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046396021000000003 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595950+0000", 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000316514 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595991+0000", 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1430000000000001e-05 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2232 2.14 2:2e2fa760:::200.00000001:head [write 2396859~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.939539805999999, 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9689986130000001, 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.752 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802904+0000", 2024-09-19T05:44:57.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-07 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802912+0000", 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9489999999999996e-06 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802920+0000", 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5970000000000003e-06 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802925+0000", 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5040000000000002e-06 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730249+0000", 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9273239709999999 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730352+0000", 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102648 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730514+0000", 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016198499999999999 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755135+0000", 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024621601 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771776+0000", 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016640773000000001 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771841+0000", 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5178000000000005e-05 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771902+0000", 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0825999999999997e-05 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2233 2.14 2:2e2fa760:::200.00000001:head [write 2398939~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.939509709999999, 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969058076, 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807300+0000", 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043666679999999998 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807312+0000", 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2199e-05 2024-09-19T05:44:57.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807327+0000", 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4260999999999999e-05 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807339+0000", 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737540+0000", 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9302006619999998 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737660+0000", 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119881 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737878+0000", 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218614 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755190+0000", 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017311756000000001 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771913+0000", 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016722954000000002 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771958+0000", 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123000000000001e-05 2024-09-19T05:44:57.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771992+0000", 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3657e-05 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2234 2.14 2:2e2fa760:::200.00000001:head [write 2401019~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.935055958, 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9661180190000001, 2024-09-19T05:44:57.763 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807388+0000", 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-07 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807399+0000", 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1338999999999999e-05 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807409+0000", 2024-09-19T05:44:57.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807416+0000", 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1180000000000003e-06 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743599+0000", 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9361828380000001 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743702+0000", 2024-09-19T05:44:57.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103303 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743868+0000", 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771998+0000", 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028130420999999999 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773391+0000", 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013930699999999999 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773448+0000", 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6672999999999998e-05 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773505+0000", 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7445000000000003e-05 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2235 2.14 2:2e2fa760:::200.00000001:head [write 2403099~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.935016743999999, 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9662057229999998, 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4000000000000002e-07 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807435+0000", 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.072e-06 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807478+0000", 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2460999999999999e-05 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807486+0000", 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5350000000000004e-06 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.745914+0000", 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9384274700000002 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.747669+0000", 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017549239999999999 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.748294+0000", 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00062547500000000001 2024-09-19T05:44:57.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773338+0000", 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025043950999999998 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773551+0000", 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021271199999999999 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773599+0000", 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7876e-05 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773632+0000", 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3707000000000001e-05 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2236 2.14 2:2e2fa760:::200.00000001:head [write 2405179~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.934946626999999, 2024-09-19T05:44:57.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9899550850000001, 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807502+0000", 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4850000000000003e-06 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807509+0000", 2024-09-19T05:44:57.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0859999999999996e-06 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807515+0000", 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9179999999999999e-06 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755230+0000", 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9477156679999998 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755342+0000", 2024-09-19T05:44:57.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111703 2024-09-19T05:44:57.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755522+0000", 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000179937 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.783617+0000", 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028095472 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797341+0000", 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013723506999999999 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797404+0000", 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797452+0000", 2024-09-19T05:44:57.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7567e-05 2024-09-19T05:44:57.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.781 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2237 2.14 2:2e2fa760:::200.00000001:head [write 2407259~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.934921954, 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0092068749999998, 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807522+0000", 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807530+0000", 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8930000000000005e-06 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807577+0000", 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7064999999999997e-05 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807606+0000", 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8963e-05 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761580+0000", 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.953974901 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761685+0000", 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010435800000000001 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761827+0000", 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142313 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792591+0000", 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030764073999999999 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816632+0000", 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024040371000000001 2024-09-19T05:44:57.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816688+0000", 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5951e-05 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816728+0000", 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0636000000000002e-05 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2238 2.14 2:2e2fa760:::200.00000001:head [write 2409339~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.934824099, 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0122208439999998, 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834275+0000", 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026656197999999999 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834287+0000", 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1691e-05 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834301+0000", 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3974e-05 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834310+0000", 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8659999999999995e-06 2024-09-19T05:44:57.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772040+0000", 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9377302950000002 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772148+0000", 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108136 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772314+0000", 2024-09-19T05:44:57.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016526400000000001 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792597+0000", 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020283483000000001 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819691+0000", 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027094025000000001 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819771+0000", 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0022000000000001e-05 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819840+0000", 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8889999999999999e-05 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2239 2.14 2:2e2fa760:::200.00000001:head [write 2411419~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.908114116, 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.985646596, 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851429+0000", 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017099436999999999 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851444+0000", 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5449000000000001e-05 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851460+0000", 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.556e-05 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851470+0000", 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0001e-05 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773666+0000", 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9221966369999999 2024-09-19T05:44:57.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773757+0000", 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1160000000000001e-05 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773902+0000", 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014419500000000001 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795500+0000", 2024-09-19T05:44:57.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021598346000000001 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819898+0000", 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024398474 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819942+0000", 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3151000000000001e-05 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819976+0000", 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4186000000000001e-05 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2240 2.14 2:2e2fa760:::200.00000001:head [write 2413499~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.890955393999999, 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9685921560000001, 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862188+0000", 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010699774 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862213+0000", 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5536000000000001e-05 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862225+0000", 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1328000000000001e-05 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862231+0000", 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.55e-06 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.780995+0000", 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.918763854 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781115+0000", 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120538 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781261+0000", 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145785 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795505+0000", 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014244158 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820012+0000", 2024-09-19T05:44:57.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024506607 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820051+0000", 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8597000000000003e-05 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820080+0000", 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9428999999999999e-05 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.804 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2245 2.14 2:2e2fa760:::200.00000001:head [write 2423899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.880079478999999, 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969609441, 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.08e-07 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862368+0000", 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7929999999999999e-06 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862375+0000", 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6680000000000004e-06 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862380+0000", 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797491+0000", 2024-09-19T05:44:57.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9351107490000001 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797581+0000", 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9559000000000001e-05 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797723+0000", 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014230199999999999 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820590+0000", 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022867386 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827763+0000", 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071727190000000001 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.828797+0000", 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010343570000000001 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831973+0000", 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003175925 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2246 2.14 2:2e2fa760:::200.00000001:head [write 2425979~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.880055564999999, 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9696936549999999, 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:44:57.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.96e-07 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862397+0000", 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862404+0000", 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862409+0000", 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1320000000000002e-06 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804733+0000", 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.94232443 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804840+0000", 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106724 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804998+0000", 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015801500000000001 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826045+0000", 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021046664999999999 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831983+0000", 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059387570000000002 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832038+0000", 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4475999999999998e-05 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832081+0000", 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3591000000000003e-05 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2247 2.14 2:2e2fa760:::200.00000001:head [write 2428059~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.880028484, 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969738048, 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862426+0000", 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1090999999999999e-05 2024-09-19T05:44:57.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862433+0000", 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8070000000000002e-06 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862438+0000", 2024-09-19T05:44:57.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1599999999999997e-06 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806006+0000", 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.943567885 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806105+0000", 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8782000000000004e-05 2024-09-19T05:44:57.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806289+0000", 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183678 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827739+0000", 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021450258999999999 2024-09-19T05:44:57.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832087+0000", 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043483280000000003 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832124+0000", 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6816000000000001e-05 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832153+0000", 2024-09-19T05:44:57.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.887e-05 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2248 2.14 2:2e2fa760:::200.00000001:head [write 2430139~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.879998486000002, 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9698627229999999, 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.825 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877451+0000", 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015005956000000001 2024-09-19T05:44:57.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877461+0000", 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0034000000000001e-05 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.896137+0000", 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018675967000000002 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909217+0000", 2024-09-19T05:44:57.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013080629999999999 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807553+0000", 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8983351339999999 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807647+0000", 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4289000000000005e-05 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807794+0000", 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000147467 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832160+0000", 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024365753 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832224+0000", 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3689000000000006e-05 2024-09-19T05:44:57.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832274+0000", 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0649000000000002e-05 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832308+0000", 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3154999999999997e-05 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2250 2.14 2:2e2fa760:::200.00000001:head [write 2434299~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.832557636000001, 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9595307040000001, 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939724+0000", 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029838587 2024-09-19T05:44:57.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939743+0000", 2024-09-19T05:44:57.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9077e-05 2024-09-19T05:44:57.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941640+0000", 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018968310000000001 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941651+0000", 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0379e-05 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814230+0000", 2024-09-19T05:44:57.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.872579934 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814430+0000", 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199481 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814571+0000", 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014074399999999999 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832173+0000", 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017602199999999998 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869325+0000", 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037152223999999998 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869374+0000", 2024-09-19T05:44:57.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9149e-05 2024-09-19T05:44:57.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869416+0000", 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2098000000000002e-05 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2337 3.3c 3:3eebf982:::100000001f8.0000051e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.730411495999999, 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.014058033, 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2337 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.018343+0000", 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063114770000000002 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030374+0000", 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012031086999999999 2024-09-19T05:44:57.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030396+0000", 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.156e-05 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030411+0000", 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4875000000000001e-05 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897205+0000", 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8667940299999999 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897418+0000", 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021300400000000001 2024-09-19T05:44:57.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897573+0000", 2024-09-19T05:44:57.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015481300000000001 2024-09-19T05:44:57.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.943251+0000", 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045678424000000002 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026021+0000", 2024-09-19T05:44:57.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082769669000000004 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026059+0000", 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8296000000000001e-05 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026090+0000", 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0797999999999999e-05 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2267 2.14 2:2e2fa760:::200.00000001:head [write 2469659~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.673769228000001, 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9703313549999999, 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076283+0000", 2024-09-19T05:44:57.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076092460000000001 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076295+0000", 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1857e-05 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076309+0000", 2024-09-19T05:44:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3661e-05 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076319+0000", 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0589e-05 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963409+0000", 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.887089915 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963490+0000", 2024-09-19T05:44:57.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1137000000000006e-05 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963615+0000", 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012412999999999999 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012633+0000", 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049018171999999999 2024-09-19T05:44:57.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038886+0000", 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026253216999999999 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038950+0000", 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3876999999999998e-05 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039005+0000", 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5553999999999997e-05 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2268 2.14 2:2e2fa760:::200.00000001:head [write 2471739~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.66609373, 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962740785, 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.853 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:57.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076361+0000", 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076368+0000", 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4240000000000004e-06 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076374+0000", 2024-09-19T05:44:57.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0179999999999998e-06 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975236+0000", 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.898862345 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975332+0000", 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5217999999999995e-05 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975458+0000", 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125986 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012673+0000", 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037215224999999998 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039014+0000", 2024-09-19T05:44:57.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026341256 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039056+0000", 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1944999999999999e-05 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039090+0000", 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4301999999999998e-05 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2269 2.14 2:2e2fa760:::200.00000001:head [write 2473819~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.666060172000002, 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9627868230000001, 2024-09-19T05:44:57.859 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076384+0000", 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076392+0000", 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.174e-06 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076398+0000", 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4309999999999999e-06 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076403+0000", 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6399999999999996e-06 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976688+0000", 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9002852180000001 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976785+0000", 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6963e-05 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976951+0000", 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016594799999999999 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012708+0000", 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035757376 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039097+0000", 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026388551 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039137+0000", 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0025999999999999e-05 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039170+0000", 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3065000000000003e-05 2024-09-19T05:44:57.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2270 2.14 2:2e2fa760:::200.00000001:head [write 2475899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.666034206999999, 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962839365, 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102351+0000", 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025941703999999999 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102359+0000", 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102372+0000", 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3749e-05 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102382+0000", 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0229999999999998e-06 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978315+0000", 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.875933294 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978422+0000", 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107007 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978575+0000", 2024-09-19T05:44:57.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015296099999999999 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012746+0000", 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034170849000000003 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039176+0000", 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026430282999999999 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:57.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039215+0000", 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9368e-05 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039248+0000", 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3157000000000001e-05 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:57.870 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:57.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:44:57.872 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:57.870+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:57.973 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:57.972+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:58.023 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:58.023 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:58.023 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:44:58.023 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:58.032 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:44:58.073 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:58.072+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:58.174 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:58.173+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:58.184 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:58.184 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:44:58.184 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:44:58.184 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:44:58.184 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:58.193 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:44:58.275 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:44:58.273+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:44:58.357 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:44:58.357 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:44:58.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:44:58.357 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2666 3.9 3:92ca8d9b:::100000001f8.00000667:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.459579798, 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3293169570000001, 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2666 2024-09-19T05:44:58.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:44:58.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894727+0000", 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8439999999999998e-06 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897799+0000", 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030718199999999998 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897817+0000", 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7847999999999999e-05 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897831+0000", 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3845999999999999e-05 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897938+0000", 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010742399999999999 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898111+0000", 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017279099999999999 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898360+0000", 2024-09-19T05:44:58.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024885800000000002 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956644+0000", 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058284217999999999 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223960+0000", 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.267316119 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224004+0000", 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4039000000000003e-05 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224042+0000", 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7150000000000002e-05 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.364 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2667 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.45643995, 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3258909560000001, 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2667 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897867+0000", 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9299999999999999e-06 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902567+0000", 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004699356 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902588+0000", 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1631e-05 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902602+0000", 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3478e-05 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.906894+0000", 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042925760000000002 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907078+0000", 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183067 2024-09-19T05:44:58.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907289+0000", 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021123499999999999 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956671+0000", 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049382078000000003 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223652+0000", 2024-09-19T05:44:58.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.266981122 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223704+0000", 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2244999999999999e-05 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223755+0000", 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1238e-05 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2673 3.2d 3:b45a3f48:::100000001f8.0000066e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.443442523000002, 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.373980822, 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2673 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910864+0000", 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:44:58.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915100+0000", 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042352259999999999 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915123+0000", 2024-09-19T05:44:58.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3334999999999999e-05 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915137+0000", 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4178999999999999e-05 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915220+0000", 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2693000000000003e-05 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915395+0000", 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017551099999999999 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915574+0000", 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178357 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951064+0000", 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035490116000000002 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284764+0000", 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.333700259 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284808+0000", 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3813999999999998e-05 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284843+0000", 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4807000000000001e-05 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.376 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2700 3.30 3:0e790ae4:::100000001f8.00000689:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.410740421, 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.25937777, 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2700 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943567+0000", 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.083e-06 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946402+0000", 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028349439999999998 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946419+0000", 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7105999999999999e-05 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946429+0000", 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8709999999999992e-06 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.947951+0000", 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015219960000000001 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948153+0000", 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020153400000000001 2024-09-19T05:44:58.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948415+0000", 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026250200000000002 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.064312+0000", 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115897369 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202811+0000", 2024-09-19T05:44:58.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.138498891 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202872+0000", 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0529000000000003e-05 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202942+0000", 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9944999999999995e-05 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2711 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.403222811999999, 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.27282242, 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2711 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951084+0000", 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.553e-06 2024-09-19T05:44:58.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956906+0000", 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058222400000000002 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956926+0000", 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9619e-05 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956937+0000", 2024-09-19T05:44:58.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0594999999999999e-05 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958546+0000", 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001609369 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958792+0000", 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024587600000000002 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.959831+0000", 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010394079999999999 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293509+0000", 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33367725100000001 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223817+0000", 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93030886999999995 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223864+0000", 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6465000000000003e-05 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223904+0000", 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0173999999999999e-05 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2732 3.9 3:92ca8380:::100000001f8.000006a9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.388 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.346353208, 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2619013319999999, 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2732 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007954+0000", 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018132+0000", 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010177734000000001 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018148+0000", 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6555999999999999e-05 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018187+0000", 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8760000000000002e-05 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020347+0000", 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021597589999999998 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020514+0000", 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016697899999999999 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020695+0000", 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018172199999999999 2024-09-19T05:44:58.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.375158+0000", 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35446218200000001 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269697+0000", 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89453943599999997 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269768+0000", 2024-09-19T05:44:58.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0619999999999998e-05 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269853+0000", 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4876000000000005e-05 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2746 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.318815996000001, 2024-09-19T05:44:58.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1890195509999999, 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2746 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035491+0000", 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043360+0000", 2024-09-19T05:44:58.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0078693240000000005 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043383+0000", 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2853e-05 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.045231+0000", 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001847476 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.122884+0000", 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077653244999999996 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123275+0000", 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039126200000000002 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123895+0000", 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061987200000000004 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293518+0000", 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.169623147 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224430+0000", 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93091173699999996 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224474+0000", 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3816000000000002e-05 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224508+0000", 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4369000000000002e-05 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2753 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.292793158999999, 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.188208317, 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2753 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061513+0000", 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.871e-06 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078405+0000", 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016891461 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078433+0000", 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8436e-05 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078447+0000", 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3607e-05 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137285+0000", 2024-09-19T05:44:58.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058837911 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137457+0000", 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017199399999999999 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137637+0000", 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180173 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.460405+0000", 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.322768426 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249610+0000", 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78920498100000003 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249664+0000", 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3668000000000002e-05 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249720+0000", 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5788999999999997e-05 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2808 3.11 3:8bd5b656:::100000001f8.000006f5:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.114176896, 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.160580451, 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2808 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:44:58.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240131+0000", 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244285+0000", 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041546919999999998 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244303+0000", 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7646999999999999e-05 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244316+0000", 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2931999999999999e-05 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373296+0000", 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12897989000000001 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373463+0000", 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166847 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373621+0000", 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015846400000000001 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.585995+0000", 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.212374013 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400581+0000", 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81458587900000001 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400646+0000", 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5036999999999994e-05 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400708+0000", 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1937e-05 2024-09-19T05:44:58.411 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5105 3.11 3:89c7a7f0:::100000001f8.000005ee:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.633885943999999, 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2072587750000001, 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.412 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5105 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720422+0000", 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.241e-06 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723834+0000", 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003412389 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723847+0000", 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2404e-05 2024-09-19T05:44:58.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723856+0000", 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6609999999999995e-06 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723919+0000", 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2928999999999993e-05 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.724040+0000", 2024-09-19T05:44:58.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120505 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.724239+0000", 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019924600000000001 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.782736+0000", 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058496758000000003 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927592+0000", 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1448568539999999 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927639+0000", 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6495000000000001e-05 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927677+0000", 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8293000000000001e-05 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5127 3.11 3:8b184820:::100000001f8.00000604:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.589883503999999, 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.163370802, 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5127 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:44:58.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764428+0000", 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.778132+0000", 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013703958 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.790747+0000", 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012615167 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.790767+0000", 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9216999999999999e-05 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.793911+0000", 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031443019999999999 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.794025+0000", 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011406200000000001 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.794225+0000", 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001997 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.836693+0000", 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042467799000000001 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927716+0000", 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0910237540000001 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927758+0000", 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1959999999999998e-05 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927792+0000", 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3470000000000003e-05 2024-09-19T05:44:58.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5160 3.14 3:2aefce75:::100000001f8.00000625:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.478788762000001, 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3210729219999999, 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5160 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875518+0000", 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.880298+0000", 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047798010000000002 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.888744+0000", 2024-09-19T05:44:58.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084454649999999992 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.888764+0000", 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0517e-05 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889116+0000", 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000351391 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889240+0000", 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012396099999999999 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889440+0000", 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020088800000000001 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.939338+0000", 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049897237999999997 2024-09-19T05:44:58.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196475+0000", 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.257137682 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196526+0000", 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0680999999999998e-05 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196589+0000", 2024-09-19T05:44:58.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2575000000000003e-05 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5205 3.2d 3:b48c1fbb:::100000001f8.00000652:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.303900511, 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1819966100000001, 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5205 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:44:58.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050407+0000", 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5149999999999998e-06 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053361+0000", 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029540410000000001 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053381+0000", 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9471999999999999e-05 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053393+0000", 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372360+0000", 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.318966784 2024-09-19T05:44:58.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372504+0000", 2024-09-19T05:44:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014391600000000001 2024-09-19T05:44:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372718+0000", 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021371699999999999 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.499375+0000", 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12665742599999999 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232287+0000", 2024-09-19T05:44:58.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73291224399999999 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232347+0000", 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9920000000000002e-05 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232401+0000", 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3273999999999998e-05 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5227 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.269390194, 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.532865806, 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5227 2024-09-19T05:44:58.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084919+0000", 2024-09-19T05:44:58.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7650000000000001e-06 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091910+0000", 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069906250000000003 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091939+0000", 2024-09-19T05:44:58.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9274000000000001e-05 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091955+0000", 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6280999999999999e-05 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165174+0000", 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073219124999999996 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165300+0000", 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012565699999999999 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165490+0000", 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000190519 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.233620+0000", 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068129713999999994 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617666+0000", 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.384046068 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617724+0000", 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7410000000000001e-05 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617780+0000", 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6368e-05 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5271 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.109377762999999, 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.372975533, 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5271 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244929+0000", 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6460000000000002e-06 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.251094+0000", 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061647539999999997 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258025+0000", 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00693065 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258041+0000", 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6708999999999999e-05 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258150+0000", 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010840000000000001 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258311+0000", 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000160688 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.260274+0000", 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001963027 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359521+0000", 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099247940000000007 2024-09-19T05:44:58.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617826+0000", 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.258304525 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617868+0000", 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2454999999999999e-05 2024-09-19T05:44:58.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617902+0000", 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3738999999999997e-05 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5306 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.960762147000001, 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.224487935, 2024-09-19T05:44:58.449 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5306 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393546+0000", 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6389999999999998e-06 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.398380+0000", 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048341859999999999 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.433026+0000", 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034645428999999998 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.433052+0000", 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6945e-05 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.435900+0000", 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028475969999999999 2024-09-19T05:44:58.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.436007+0000", 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010702 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.436183+0000", 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017633300000000001 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531619+0000", 2024-09-19T05:44:58.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.095435107000000005 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617951+0000", 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0863320190000001 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617995+0000", 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4474999999999997e-05 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.618030+0000", 2024-09-19T05:44:58.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5185000000000003e-05 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5313 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.916278449, 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2520088149999999, 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5313 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438028+0000", 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1610000000000001e-06 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440687+0000", 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026591789999999998 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440706+0000", 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8236999999999998e-05 2024-09-19T05:44:58.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440717+0000", 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.171e-05 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444350+0000", 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036329819999999999 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444438+0000", 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7292000000000004e-05 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444578+0000", 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013099999999999 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.684423+0000", 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.239845633 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.689914+0000", 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.005491175 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.689975+0000", 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0943999999999998e-05 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690035+0000", 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9370999999999998e-05 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.460 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5329 3.11 3:8be9d197:::100000001f8.000006ce:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.907589327, 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.24346679, 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5329 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446718+0000", 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2169999999999999e-06 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453551+0000", 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068323860000000002 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453567+0000", 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6538e-05 2024-09-19T05:44:58.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453579+0000", 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.218e-05 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.475998+0000", 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022418223000000001 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.477939+0000", 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001941639 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478106+0000", 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016716699999999999 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.780733+0000", 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.302626495 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690099+0000", 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90936588600000001 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690144+0000", 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5019999999999999e-05 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690182+0000", 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8039e-05 2024-09-19T05:44:58.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5333 3.9 3:9102af4b:::100000001f8.000006d2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.896315389, 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1967455760000001, 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5333 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457991+0000", 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.08e-06 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462463+0000", 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044715370000000003 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462495+0000", 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2617999999999998e-05 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462510+0000", 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4452e-05 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531565+0000", 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069055635000000004 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531664+0000", 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8634999999999993e-05 2024-09-19T05:44:58.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531822+0000", 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015805600000000001 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.867710+0000", 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33588781299999998 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654623+0000", 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78691265499999996 2024-09-19T05:44:58.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654683+0000", 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0380000000000001e-05 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654735+0000", 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1715000000000002e-05 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5356 3.9 3:9157a25d:::100000001f8.000006e9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.872495986000001, 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.173045817, 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5356 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481811+0000", 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7159999999999999e-06 2024-09-19T05:44:58.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493724+0000", 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011912668 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493759+0000", 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4968999999999997e-05 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493772+0000", 2024-09-19T05:44:58.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2845e-05 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645428+0000", 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15165590900000001 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645559+0000", 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013117000000000001 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645810+0000", 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000251432 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.867748+0000", 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22193777100000001 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654779+0000", 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78703058599999998 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654821+0000", 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2540999999999998e-05 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654854+0000", 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210000000000002e-05 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:44:58.478 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:44:58.479 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:44:58.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:44:58.481 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:58.479+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:58.576 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:44:58.601 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:44:58.602 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:44:58.602 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:44:58.602 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:44:58.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:44:58.621 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:58.620+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:58.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.623+0000 7fb63ad98640 1 Processor -- start 2024-09-19T05:44:58.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.623+0000 7fb63ad98640 1 -- start start 2024-09-19T05:44:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.623+0000 7fb63ad98640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb63415aa90 0x7fb63415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.623+0000 7fb63ad98640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb6340cd8c0 0x7fb63415e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.623+0000 7fb63ad98640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb63415ebe0 con 0x7fb6340cd4e0 2024-09-19T05:44:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.623+0000 7fb63ad98640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fb63415e9b0 con 0x7fb6340cd8c0 2024-09-19T05:44:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.623+0000 7fb63ad98640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb63415eec0 con 0x7fb63415aa90 2024-09-19T05:44:58.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb639335640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb6340cd8c0 0x7fb63415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb639335640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb6340cd8c0 0x7fb63415e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40008/0 (socket says 172.21.15.2:40008) 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb639335640 1 -- 172.21.15.2:0/4123638251 learned_addr learned my addr 172.21.15.2:0/4123638251 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb633fff640 1 --2- 172.21.15.2:0/4123638251 >> v2:172.21.15.123:3300/0 conn(0x7fb63415aa90 0x7fb63415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb639335640 1 -- 172.21.15.2:0/4123638251 >> v2:172.21.15.123:3300/0 conn(0x7fb63415aa90 msgr2=0x7fb63415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb639335640 1 --2- 172.21.15.2:0/4123638251 >> v2:172.21.15.123:3300/0 conn(0x7fb63415aa90 0x7fb63415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb639335640 1 -- 172.21.15.2:0/4123638251 >> v1:172.21.15.123:6789/0 conn(0x7fb6340cd4e0 legacy=0x7fb6340b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb6337fe640 1 -- 172.21.15.2:0/4123638251 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902294267 0 0) 0x7fb63415ebe0 con 0x7fb6340cd4e0 2024-09-19T05:44:58.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.624+0000 7fb639335640 1 -- 172.21.15.2:0/4123638251 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6340b2e00 con 0x7fb6340cd8c0 2024-09-19T05:44:58.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.625+0000 7fb639335640 1 --2- 172.21.15.2:0/4123638251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb63415e470 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb63415a940 tx=0x7fb62c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fa04b212ad1fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.625+0000 7fb6337fe640 1 -- 172.21.15.2:0/4123638251 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c036070 con 0x7fb6340cd8c0 2024-09-19T05:44:58.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.625+0000 7fb6337fe640 1 -- 172.21.15.2:0/4123638251 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c031600 con 0x7fb6340cd8c0 2024-09-19T05:44:58.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.625+0000 7fb6337fe640 1 -- 172.21.15.2:0/4123638251 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0319d0 con 0x7fb6340cd8c0 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 -- 172.21.15.2:0/4123638251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 msgr2=0x7fb63415e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 --2- 172.21.15.2:0/4123638251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb63415e470 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb63415a940 tx=0x7fb62c02a730 comp rx=0 tx=0).stop 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 -- 172.21.15.2:0/4123638251 shutdown_connections 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 --2- 172.21.15.2:0/4123638251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb63415e470 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 --2- 172.21.15.2:0/4123638251 >> v2:172.21.15.123:3300/0 conn(0x7fb63415aa90 0x7fb63415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 -- 172.21.15.2:0/4123638251 >> 172.21.15.2:0/4123638251 conn(0x7fb634067b30 msgr2=0x7fb634067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 -- 172.21.15.2:0/4123638251 shutdown_connections 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.626+0000 7fb63ad98640 1 -- 172.21.15.2:0/4123638251 wait complete. 2024-09-19T05:44:58.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.627+0000 7fb63ad98640 1 Processor -- start 2024-09-19T05:44:58.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.627+0000 7fb63ad98640 1 -- start start 2024-09-19T05:44:58.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb63ad98640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6340cd4e0 0x7fb6341e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb63ad98640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb6341e6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb63ad98640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63415aa90 0x7fb6341e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb63ad98640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fb6340b36a0 con 0x7fb6340cd8c0 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb63ad98640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6340b33c0 con 0x7fb63415aa90 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb63ad98640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb6340b42d0 con 0x7fb6340cd4e0 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb633fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb6341e6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb633fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb6341e6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40022/0 (socket says 172.21.15.2:40022) 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb633fff640 1 -- 172.21.15.2:0/515819163 learned_addr learned my addr 172.21.15.2:0/515819163 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb639335640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63415aa90 0x7fb6341e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.628+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6340cd4e0 0x7fb6341e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb633fff640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6340cd4e0 msgr2=0x7fb6341e68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb633fff640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb6340cd4e0 0x7fb6341e68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb633fff640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63415aa90 msgr2=0x7fb6341e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb633fff640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb63415aa90 0x7fb6341e47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb633fff640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6341f3600 con 0x7fb6340cd8c0 2024-09-19T05:44:58.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb633fff640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb6341e6e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e8a0 tx=0x7fb62000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620017070 con 0x7fb6340cd8c0 2024-09-19T05:44:58.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62000c940 con 0x7fb6340cd8c0 2024-09-19T05:44:58.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000cd30 con 0x7fb6340cd8c0 2024-09-19T05:44:58.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6341f38e0 con 0x7fb6340cd8c0 2024-09-19T05:44:58.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.629+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6341f3ea0 con 0x7fb6340cd8c0 2024-09-19T05:44:58.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.630+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb6341f4280 con 0x7fb6340cd8c0 2024-09-19T05:44:58.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.631+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb62001a070 con 0x7fb6340cd8c0 2024-09-19T05:44:58.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.631+0000 7fb6317fa640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb61806c2e0 0x7fb61806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.631+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5799+0+0 (secure 0 0 0) 0x7fb62008b230 con 0x7fb6340cd8c0 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.631+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb61806c2e0 0x7fb61806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.632+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v83) ==== 54+0+16 (secure 0 0 0) 0x7fb6341f4280 con 0x7fb6340cd8c0 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.632+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb6340c6dc0 0x7fb63406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.632+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6341f33a0 con 0x7fb6340c6dc0 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.632+0000 7fb639335640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb6340c6dc0 0x7fb63406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.632+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb61806c2e0 0x7fb61806e7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb628000f60 tx=0x7fb628009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.632+0000 7fb639335640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb6340c6dc0 0x7fb63406d190 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.633+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb62c037040 con 0x7fb6340c6dc0 2024-09-19T05:44:58.648 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.647+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb6340b3bc0 con 0x7fb6340c6dc0 2024-09-19T05:44:58.650 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.649+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb6340b3bc0 con 0x7fb6340c6dc0 2024-09-19T05:44:58.650 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:44:58.651 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:44:58.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.649+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb63406d840 0x7fb634157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.649+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb6340c84b0 con 0x7fb63406d840 2024-09-19T05:44:58.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.649+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb63406d840 0x7fb634157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.650+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb63406d840 0x7fb634157050 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.651+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6340c84b0 con 0x7fb63406d840 2024-09-19T05:44:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.665+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb6340c8910 con 0x7fb63406d840 2024-09-19T05:44:58.668 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.667+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb6340c8910 con 0x7fb63406d840 2024-09-19T05:44:58.668 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:44:58.669 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:44:58.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.667+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb634062630 0x7fb634062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.667+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb634062f90 con 0x7fb634062630 2024-09-19T05:44:58.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.667+0000 7fb633fff640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb634062630 0x7fb634062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.668+0000 7fb633fff640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb634062630 0x7fb634062a50 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.668+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb634062f90 con 0x7fb634062630 2024-09-19T05:44:58.680 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:58.680+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:58.683 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.682+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb634063470 con 0x7fb634062630 2024-09-19T05:44:58.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.683+0000 7fe046581640 1 Processor -- start 2024-09-19T05:44:58.684 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.683+0000 7fe046581640 1 -- start start 2024-09-19T05:44:58.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb634063470 con 0x7fb634062630 2024-09-19T05:44:58.685 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:44:58.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe046581640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe0400c9430 0x7fe0400c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.686 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:44:58.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe046581640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0400706a0 0x7fe040070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe046581640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe04015ad90 0x7fe04015b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe046581640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe04015b760 con 0x7fe0400c9430 2024-09-19T05:44:58.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe046581640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe04015e110 con 0x7fe0400706a0 2024-09-19T05:44:58.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe046581640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe040071170 con 0x7fe04015ad90 2024-09-19T05:44:58.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe03ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe0400c9430 0x7fe0400c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb63415c3d0 0x7fb63402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb63415c810 con 0x7fb63415c3d0 2024-09-19T05:44:58.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe03ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe0400c9430 0x7fe0400c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40030/0 (socket says 172.21.15.2:40030) 2024-09-19T05:44:58.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.684+0000 7fe03ffff640 1 -- 172.21.15.2:0/1136932836 learned_addr learned my addr 172.21.15.2:0/1136932836 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:58.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fe03f7fe640 1 --2- 172.21.15.2:0/1136932836 >> v2:172.21.15.123:3300/0 conn(0x7fe0400706a0 0x7fe040070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fe044b1e640 1 --2- 172.21.15.2:0/1136932836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe04015ad90 0x7fe04015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.688 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fb639335640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb63415c3d0 0x7fb63402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fe03ffff640 1 -- 172.21.15.2:0/1136932836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe04015ad90 msgr2=0x7fe04015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:58.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fe03ffff640 1 --2- 172.21.15.2:0/1136932836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe04015ad90 0x7fe04015b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fe03ffff640 1 -- 172.21.15.2:0/1136932836 >> v2:172.21.15.123:3300/0 conn(0x7fe0400706a0 msgr2=0x7fe040070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:44:58.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fe03ffff640 1 --2- 172.21.15.2:0/1136932836 >> v2:172.21.15.123:3300/0 conn(0x7fe0400706a0 0x7fe040070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fe03ffff640 1 -- 172.21.15.2:0/1136932836 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04015e390 con 0x7fe0400c9430 2024-09-19T05:44:58.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.685+0000 7fb639335640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb63415c3d0 0x7fb63402c640 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe03ffff640 1 --2- 172.21.15.2:0/1136932836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe0400c9810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe0400716f0 tx=0x7fe03002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a975b9a03a19ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb63415c810 con 0x7fb63415c3d0 2024-09-19T05:44:58.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe03effd640 1 -- 172.21.15.2:0/1136932836 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030037070 con 0x7fe0400c9430 2024-09-19T05:44:58.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe03effd640 1 -- 172.21.15.2:0/1136932836 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe030032440 con 0x7fe0400c9430 2024-09-19T05:44:58.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe03effd640 1 -- 172.21.15.2:0/1136932836 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030032810 con 0x7fe0400c9430 2024-09-19T05:44:58.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe046581640 1 -- 172.21.15.2:0/1136932836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 msgr2=0x7fe0400c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.693 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe046581640 1 --2- 172.21.15.2:0/1136932836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe0400c9810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe0400716f0 tx=0x7fe03002a270 comp rx=0 tx=0).stop 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe046581640 1 -- 172.21.15.2:0/1136932836 shutdown_connections 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe046581640 1 --2- 172.21.15.2:0/1136932836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe04015ad90 0x7fe04015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe046581640 1 --2- 172.21.15.2:0/1136932836 >> v2:172.21.15.123:3300/0 conn(0x7fe0400706a0 0x7fe040070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.686+0000 7fe046581640 1 --2- 172.21.15.2:0/1136932836 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe0400c9810 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.687+0000 7fe046581640 1 -- 172.21.15.2:0/1136932836 >> 172.21.15.2:0/1136932836 conn(0x7fe0400cd4d0 msgr2=0x7fe0400cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.687+0000 7fe046581640 1 -- 172.21.15.2:0/1136932836 shutdown_connections 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.687+0000 7fe046581640 1 -- 172.21.15.2:0/1136932836 wait complete. 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.688+0000 7fe046581640 1 Processor -- start 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.688+0000 7fe046581640 1 -- start start 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe046581640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0400706a0 0x7fe0400c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe046581640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe04015c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe046581640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04015ad90 0x7fe0401f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe046581640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe04015ec30 con 0x7fe0400c9430 2024-09-19T05:44:58.694 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe046581640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe04015e950 con 0x7fe04015ad90 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe046581640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe04015ef10 con 0x7fe0400706a0 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe04015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe04015c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40034/0 (socket says 172.21.15.2:40034) 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 -- 172.21.15.2:0/3932537514 learned_addr learned my addr 172.21.15.2:0/3932537514 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03ffff640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0400706a0 0x7fe0400c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe044b1e640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04015ad90 0x7fe0401f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 -- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0400706a0 msgr2=0x7fe0400c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0400706a0 0x7fe0400c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 -- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04015ad90 msgr2=0x7fe0401f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04015ad90 0x7fe0401f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 -- 172.21.15.2:0/3932537514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0401503e0 con 0x7fe0400c9430 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.689+0000 7fe03f7fe640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe04015c570 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe03400e8a0 tx=0x7fe03400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.690+0000 7fe03cff9640 1 -- 172.21.15.2:0/3932537514 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034017070 con 0x7fe0400c9430 2024-09-19T05:44:58.695 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.690+0000 7fe03cff9640 1 -- 172.21.15.2:0/3932537514 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03400ee90 con 0x7fe0400c9430 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.690+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0400cd010 con 0x7fe0400c9430 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.690+0000 7fe03cff9640 1 -- 172.21.15.2:0/3932537514 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03400cb20 con 0x7fe0400c9430 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.690+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0400cb780 con 0x7fe0400c9430 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.691+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0400c9090 con 0x7fe0400c9430 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.691+0000 7fe03cff9640 1 -- 172.21.15.2:0/3932537514 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe03401a070 con 0x7fe0400c9430 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.691+0000 7fe03cff9640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe02c06c2e0 0x7fe02c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.692+0000 7fe03cff9640 1 -- 172.21.15.2:0/3932537514 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5799+0+0 (secure 0 0 0) 0x7fe03408b4f0 con 0x7fe0400c9430 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.692+0000 7fe03ffff640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe02c06c2e0 0x7fe02c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.696 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.692+0000 7fe03ffff640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe02c06c2e0 0x7fe02c06e7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe030034000 tx=0x7fe03002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.697 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.696+0000 7fe03cff9640 1 -- 172.21.15.2:0/3932537514 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe0340595b0 con 0x7fe0400c9430 2024-09-19T05:44:58.701 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.701+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb63402cb80 con 0x7fb63415c3d0 2024-09-19T05:44:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.702+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb63402cb80 con 0x7fb63415c3d0 2024-09-19T05:44:58.703 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:44:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:44:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.702+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb63402d090 0x7fb63402d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.702+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb63402da30 con 0x7fb63402d090 2024-09-19T05:44:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.703+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb63402d090 0x7fb63402d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.703+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb63402d090 0x7fb63402d4f0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.705 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.704+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb63402da30 con 0x7fb63402d090 2024-09-19T05:44:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.717+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb63402da30 con 0x7fb63402d090 2024-09-19T05:44:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.719+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb63402da30 con 0x7fb63402d090 2024-09-19T05:44:58.722 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:44:58.722 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:44:58.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.719+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb634204c80 0x7fb634207070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.719+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb63402da30 con 0x7fb634204c80 2024-09-19T05:44:58.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.719+0000 7fb633fff640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb634204c80 0x7fb634207070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.720+0000 7fb633fff640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb634204c80 0x7fb634207070 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.720+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.5 v2:172.21.15.123:6800/2989037926 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb63402da30 con 0x7fb634204c80 2024-09-19T05:44:58.729 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:58.729 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.734+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb63402da30 con 0x7fb634204c80 2024-09-19T05:44:58.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.736+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.5 v2:172.21.15.123:6800/2989037926 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb63402da30 con 0x7fb634204c80 2024-09-19T05:44:58.737 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:44:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:44:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.736+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb634207ac0 0x7fb634209eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.736+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb63420cc80 con 0x7fb634207ac0 2024-09-19T05:44:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.736+0000 7fb639335640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb634207ac0 0x7fb634209eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.736+0000 7fb639335640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb634207ac0 0x7fb634209eb0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.737+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb63420cc80 con 0x7fb634207ac0 2024-09-19T05:44:58.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:44:58.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.751+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb63420a3f0 con 0x7fb634207ac0 2024-09-19T05:44:58.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.753+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb63420a3f0 con 0x7fb634207ac0 2024-09-19T05:44:58.754 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:44:58.754 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:44:58.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.753+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb63420aa70 0x7fb63420ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:44:58.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.753+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb63420fd80 con 0x7fb63420aa70 2024-09-19T05:44:58.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.753+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb63420aa70 0x7fb63420ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:44:58.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.753+0000 7fb638b34640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb63420aa70 0x7fb63420ce60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:44:58.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.754+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb63420fd80 con 0x7fb63420aa70 2024-09-19T05:44:58.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.768+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fb63420fd40 con 0x7fb63420aa70 2024-09-19T05:44:58.770 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.769+0000 7fb6317fa640 1 -- 172.21.15.2:0/515819163 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb63420fd40 con 0x7fb63420aa70 2024-09-19T05:44:58.771 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:44:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:44:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb6340c6dc0 msgr2=0x7fb63406d190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7fb6340c6dc0 0x7fb63406d190 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb63406d840 msgr2=0x7fb634157050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fb63406d840 0x7fb634157050 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb634062630 msgr2=0x7fb634062a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fb634062630 0x7fb634062a50 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb63415c3d0 msgr2=0x7fb63402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fb63415c3d0 0x7fb63402c640 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb63402d090 msgr2=0x7fb63402d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb639335640 1 -- 172.21.15.2:0/515819163 reap_dead start 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7fb63402d090 0x7fb63402d4f0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb639335640 1 -- 172.21.15.2:0/515819163 reap_dead start 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb634204c80 msgr2=0x7fb634207070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fb634204c80 0x7fb634207070 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb634207ac0 msgr2=0x7fb634209eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7fb634207ac0 0x7fb634209eb0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb63420aa70 msgr2=0x7fb63420ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7fb63420aa70 0x7fb63420ce60 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb61806c2e0 msgr2=0x7fb61806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb61806c2e0 0x7fb61806e7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb628000f60 tx=0x7fb628009040 comp rx=0 tx=0).stop 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb639335640 1 -- 172.21.15.2:0/515819163 reap_dead start 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 msgr2=0x7fb6341e6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb6341e6e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb62000e8a0 tx=0x7fb62000eca0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 shutdown_connections 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 --2- 172.21.15.2:0/515819163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6340cd8c0 0x7fb6341e6e00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 >> 172.21.15.2:0/515819163 conn(0x7fb634067b30 msgr2=0x7fb6340ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 shutdown_connections 2024-09-19T05:44:58.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.770+0000 7fb63ad98640 1 -- 172.21.15.2:0/515819163 wait complete. 2024-09-19T05:44:58.781 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:58.780+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:58.854 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:58.855 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:58.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:44:58.883 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:58.882+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:58.892+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fe040150680 con 0x7fe0400c9430 2024-09-19T05:44:58.974 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:58.974 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:58.983 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:44:58.984 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:58.983+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:59.085 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:59.084+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:59.117 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:59.118 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:59.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:44:59.186 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:59.185+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:59.287 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:59.286+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:59.346 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:59.346 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:59.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:44:59.387 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:44:59.386+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:44:59.488 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:44:59.486+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:44:59.531 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:59.531 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:59.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.560+0000 7fe03cff9640 1 -- 172.21.15.2:0/3932537514 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v84) ==== 101+0+0 (secure 0 0 0) 0x7fe03405d8b0 con 0x7fe0400c9430 2024-09-19T05:44:59.561 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:44:59.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe02c06c2e0 msgr2=0x7fe02c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:59.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe02c06c2e0 0x7fe02c06e7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe030034000 tx=0x7fe03002e000 comp rx=0 tx=0).stop 2024-09-19T05:44:59.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 msgr2=0x7fe04015c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:44:59.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe04015c570 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe03400e8a0 tx=0x7fe03400ed60 comp rx=0 tx=0).stop 2024-09-19T05:44:59.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 shutdown_connections 2024-09-19T05:44:59.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe02c06c2e0 0x7fe02c06e7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:59.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe04015ad90 0x7fe0401f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:59.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe0400c9430 0x7fe04015c570 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:59.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 --2- 172.21.15.2:0/3932537514 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0400706a0 0x7fe0400c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:44:59.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 >> 172.21.15.2:0/3932537514 conn(0x7fe0400cd4d0 msgr2=0x7fe04006c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:44:59.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 shutdown_connections 2024-09-19T05:44:59.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:44:59.564+0000 7fe046581640 1 -- 172.21.15.2:0/3932537514 wait complete. 2024-09-19T05:44:59.623 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:44:59.622+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:44:59.690 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:59.688+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:59.758 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:59.758 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:59.772 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:44:59.791 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:44:59.790+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:44:59.892 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:44:59.891+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:44:59.983 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:44:59.984 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:44:59.992 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:44:59.991+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:44:59.995 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:45:00.093 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:45:00.091+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:45:00.188 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:00.189 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:00.194 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:45:00.193+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:45:00.200 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:45:00.221 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 6, 7, 5] 2024-09-19T05:45:00.222 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-19T05:45:00.222 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-19T05:45:00.222 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-19T05:45:00.222 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-19T05:45:00.222 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-19T05:45:00.295 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:45:00.294+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:45:00.374 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:00.374 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "" 2024-09-19T05:45:00.374 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:00.380 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:00.380 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:00.394 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:45:00.395 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:45:00.394+0000 7f9b2dcb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 51427) UID: 0 2024-09-19T05:45:00.496 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:45:00.494+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:45:00.505 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:00.505 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:00.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-19T05:45:00.597 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:45:00.596+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:45:00.623 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:00.623 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:00.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-19T05:45:00.718 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:45:00.717+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:45:00.781 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:00.781 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:00.784 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-19T05:45:00.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-19T05:45:00.798 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:45:00.797+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:45:00.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.884+0000 7f7ba8d68640 1 Processor -- start 2024-09-19T05:45:00.886 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.885+0000 7f7ba8d68640 1 -- start start 2024-09-19T05:45:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.885+0000 7f7ba8d68640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ba415aa90 0x7f7ba415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.885+0000 7f7ba8d68640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba415ebe0 con 0x7f7ba40cd4e0 2024-09-19T05:45:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.885+0000 7f7ba8d68640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba40b2e00 con 0x7f7ba40cd8c0 2024-09-19T05:45:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.885+0000 7f7ba8d68640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba415eec0 con 0x7f7ba415aa90 2024-09-19T05:45:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba259c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f7ba40cd4e0 0x7f7ba40b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:50866/0 (socket says 172.21.15.2:50866) 2024-09-19T05:45:00.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba259c640 1 -- 172.21.15.2:0/228759618 learned_addr learned my addr 172.21.15.2:0/228759618 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092285127 0 0) 0x7f7ba415ebe0 con 0x7f7ba40cd4e0 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/228759618 >> v2:172.21.15.123:3300/0 conn(0x7f7ba415aa90 0x7f7ba415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b90003520 con 0x7f7ba40cd4e0 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2321511336 0 0) 0x7f7b90003520 con 0x7f7ba40cd4e0 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 >> v2:172.21.15.123:3300/0 conn(0x7f7ba415aa90 msgr2=0x7f7ba415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 --2- 172.21.15.2:0/228759618 >> v2:172.21.15.123:3300/0 conn(0x7f7ba415aa90 0x7f7ba415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 >> v1:172.21.15.123:6789/0 conn(0x7f7ba40cd8c0 legacy=0x7f7ba40c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba40b30e0 con 0x7f7ba40cd4e0 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f7b9c0019f0 con 0x7f7ba40cd4e0 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 >> v1:172.21.15.2:6789/0 conn(0x7f7ba40cd4e0 legacy=0x7f7ba40b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b900039b0 0x7f7b90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b90004800 0x7f7b90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b900051c0 0x7f7b9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba415eec0 con 0x7f7b900039b0 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9000c060 con 0x7f7b900051c0 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9000c260 con 0x7f7b90004800 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba259c640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b900039b0 0x7f7b90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.886+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b900051c0 0x7f7b9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b90004800 0x7f7b90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba259c640 1 -- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b90004800 msgr2=0x7f7b90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba259c640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b90004800 0x7f7b90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba259c640 1 -- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b900051c0 msgr2=0x7f7b9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba259c640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b900051c0 0x7f7b9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:00.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba259c640 1 -- 172.21.15.2:0/228759618 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b90003520 con 0x7f7b900039b0 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba2d9d640 1 -- 172.21.15.2:0/228759618 reap_dead start 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.887+0000 7f7ba259c640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b900039b0 0x7f7b90003dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00a840 tx=0x7f7b90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb806ceeeae19e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00a460 con 0x7f7b900039b0 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9c02f070 con 0x7f7b900039b0 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba159a640 1 -- 172.21.15.2:0/228759618 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c02bcf0 con 0x7f7b900039b0 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba8d68640 1 -- 172.21.15.2:0/228759618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b900039b0 msgr2=0x7f7b90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b900039b0 0x7f7b90003dd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00a840 tx=0x7f7b90005830 comp rx=0 tx=0).stop 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba8d68640 1 -- 172.21.15.2:0/228759618 shutdown_connections 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/228759618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b900039b0 0x7f7b90003dd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba8d68640 1 -- 172.21.15.2:0/228759618 >> 172.21.15.2:0/228759618 conn(0x7f7ba4067b30 msgr2=0x7f7ba4067f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba8d68640 1 -- 172.21.15.2:0/228759618 shutdown_connections 2024-09-19T05:45:00.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.888+0000 7f7ba8d68640 1 -- 172.21.15.2:0/228759618 wait complete. 2024-09-19T05:45:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.890+0000 7f7ba8d68640 1 Processor -- start 2024-09-19T05:45:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.890+0000 7f7ba8d68640 1 -- start start 2024-09-19T05:45:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba8d68640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b900039b0 0x7f7ba41f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba8d68640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ba4153dc0 0x7f7ba4150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba8d68640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba4150ff0 0x7f7ba415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba8d68640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40b3bc0 con 0x7f7ba4153dc0 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba8d68640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40b36a0 con 0x7f7ba4150ff0 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba8d68640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba40b3d40 con 0x7f7b900039b0 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ba4153dc0 0x7f7ba4150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ba4153dc0 0x7f7ba4150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:40040/0 (socket says 172.21.15.2:40040) 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 -- 172.21.15.2:0/2332330704 learned_addr learned my addr 172.21.15.2:0/2332330704 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b900039b0 0x7f7ba41f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba4150ff0 0x7f7ba415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b900039b0 msgr2=0x7f7ba41f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b900039b0 0x7f7ba41f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba4150ff0 msgr2=0x7f7ba415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ba4150ff0 0x7f7ba415bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba4150320 con 0x7f7ba4153dc0 2024-09-19T05:45:00.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ba4153dc0 0x7f7ba4150ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c011a70 tx=0x7f7b8c011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00fb50 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00fcf0 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba415d200 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.891+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00fea0 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.892+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba415d4a0 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.892+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7ba40b40d0 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.893+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7b8c012040 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.893+0000 7f7b9affd640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7b8406c2b0 0x7f7b8406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5799+0+0 (secure 0 0 0) 0x7f7b8c08ba50 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v84) ==== 54+0+16 (secure 0 0 0) 0x7f7b8c019030 con 0x7f7ba4153dc0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7b8406c2b0 0x7f7b8406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ba4150620 0x7f7ba4159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ba40b5d10 con 0x7f7ba4150620 2024-09-19T05:45:00.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ba4150620 0x7f7ba4159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7b8406c2b0 0x7f7b8406e770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c033000 tx=0x7f7b9c007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.894+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ba4150620 0x7f7ba4159350 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.895+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b940051f0 con 0x7f7ba4150620 2024-09-19T05:45:00.901 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:45:00.899+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:45:00.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.909+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba415a290 con 0x7f7ba4150620 2024-09-19T05:45:00.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.911+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba415a290 con 0x7f7ba4150620 2024-09-19T05:45:00.912 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:45:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:45:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.911+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ba4154240 0x7f7ba415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.911+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7ba4159fb0 con 0x7f7ba4154240 2024-09-19T05:45:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.912+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ba4154240 0x7f7ba415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.912+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ba4154240 0x7f7ba415c530 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:00.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.912+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba4159fb0 con 0x7f7ba4154240 2024-09-19T05:45:00.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.927+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba4152420 con 0x7f7ba4154240 2024-09-19T05:45:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.929+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba4152420 con 0x7f7ba4154240 2024-09-19T05:45:00.930 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:45:00.930 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:45:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.929+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ba414eb10 0x7f7ba4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.929+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f7ba414eef0 con 0x7f7ba414eb10 2024-09-19T05:45:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.929+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ba414eb10 0x7f7ba4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.929+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ba414eb10 0x7f7ba4062630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:00.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.930+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba414eef0 con 0x7f7ba414eb10 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2637 3.3 3:c38cdf66:::100000001f8.0000064a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 29.109441585999999, 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.86377266600000002, 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2637 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829829+0000", 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.829833+0000", 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4700000000000004e-06 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834195+0000", 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043615379999999999 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834217+0000", 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1713e-05 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.834229+0000", 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2418e-05 2024-09-19T05:45:00.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.139987+0000", 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.305758321 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140183+0000", 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000195577 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.140385+0000", 2024-09-19T05:45:00.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020212799999999999 2024-09-19T05:45:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.465853+0000", 2024-09-19T05:45:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.32546820700000001 2024-09-19T05:45:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693468+0000", 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22761451199999999 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693549+0000", 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1267999999999995e-05 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693602+0000", 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2513999999999999e-05 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2681 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.956950850999998, 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.077969779, 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2681 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982320+0000", 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.982323+0000", 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2370000000000001e-06 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986715+0000", 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004392618 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986751+0000", 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5398999999999998e-05 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.986762+0000", 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1545999999999999e-05 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706073+0000", 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71931063299999998 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.706251+0000", 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017771099999999999 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060111+0000", 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35386035900000001 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060200+0000", 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9259999999999996e-05 2024-09-19T05:45:00.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.060289+0000", 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9016000000000003e-05 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4534.0:3095 3.33 e71/20 3:cd66a262:::100000001f8.00000814:head v 71'69, mlcod=70'60)", 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 28.628133900000002, 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80810778000000005, 2024-09-19T05:45:00.954 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311137+0000", 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.311141+0000", 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.284e-06 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318008+0000", 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068671770000000004 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318032+0000", 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4467000000000001e-05 2024-09-19T05:45:00.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.318051+0000", 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8284e-05 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693668+0000", 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.37561766600000002 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.693691+0000", 2024-09-19T05:45:00.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2875e-05 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.116967+0000", 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.42327609900000002 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.119244+0000", 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022769280000000001 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5152 3.2c 3:350c5391:::100000001f8.0000061d:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.086243235000001, 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.80414443700000005, 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5152 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853027+0000", 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.853033+0000", 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6400000000000002e-06 2024-09-19T05:45:00.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856724+0000", 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036910649999999999 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856747+0000", 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2719999999999999e-05 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856759+0000", 2024-09-19T05:45:00.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2679e-05 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948007+0000", 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.091247606999999994 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948133+0000", 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012556500000000001 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:00.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.948378+0000", 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024540299999999999 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.240715+0000", 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.29233733899999997 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657035+0000", 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.41632016100000002 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657101+0000", 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5691999999999994e-05 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.657172+0000", 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0566000000000002e-05 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5153 3.2c 3:35ced6af:::100000001f8.0000061e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.964 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.082481337000001, 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.84790699300000005, 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5153 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.965 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856789+0000", 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.856791+0000", 2024-09-19T05:45:00.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2239999999999998e-06 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859552+0000", 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027601930000000002 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859564+0000", 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2880999999999999e-05 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.859579+0000", 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4925e-05 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145653+0000", 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.286073313 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.145820+0000", 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166937 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.167125+0000", 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021305001 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.270722+0000", 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.103597041 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704576+0000", 2024-09-19T05:45:00.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.43385462400000002 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704637+0000", 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1118000000000007e-05 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.704696+0000", 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8736000000000001e-05 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5229 3.34 3:2ebd7ea6:::100000001f8.0000066a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.006631713000001, 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90816846299999998, 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5229 2024-09-19T05:45:00.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932639+0000", 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.932642+0000", 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5820000000000002e-06 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937356+0000", 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047141400000000003 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937389+0000", 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2929999999999998e-05 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.937402+0000", 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2855e-05 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268202+0000", 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.330799856 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268403+0000", 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200464 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.268639+0000", 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000236598 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.656059+0000", 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.387419719 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840676+0000", 2024-09-19T05:45:00.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18461746900000001 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840747+0000", 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0462000000000004e-05 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.840807+0000", 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0387999999999998e-05 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5231 3.3b 3:ddf1e7a4:::100000001f8.0000066c:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.885008599999999, 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1076528050000001, 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5231 2024-09-19T05:45:00.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054262+0000", 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.054273+0000", 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1253999999999999e-05 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059647+0000", 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053742479999999999 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059676+0000", 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8699000000000001e-05 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.059689+0000", 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2476999999999999e-05 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.863904+0000", 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.804215652 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.864024+0000", 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119848 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161715+0000", 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.297691133 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161795+0000", 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9671000000000003e-05 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161915+0000", 2024-09-19T05:45:00.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011982299999999999 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5241 3.3b 3:dee7dda2:::100000001f8.00000676:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.840183644, 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.063036227, 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.982 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5241 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099087+0000", 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.099090+0000", 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0340000000000001e-06 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105109+0000", 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006018809 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105132+0000", 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2908000000000002e-05 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.105143+0000", 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1826e-05 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.924904+0000", 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81976026999999996 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925014+0000", 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011063 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162023+0000", 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.23700855300000001 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162060+0000", 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6715999999999999e-05 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162123+0000", 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3480999999999997e-05 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5250 3.3 3:c0cd83f8:::100000001f8.0000067f:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.804247317000002, 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.026991652, 2024-09-19T05:45:00.987 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5250 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135023+0000", 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.135025+0000", 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3379999999999999e-06 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137223+0000", 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021978639999999999 2024-09-19T05:45:00.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.137247+0000", 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3978999999999999e-05 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.144008+0000", 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067608770000000002 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915821+0000", 2024-09-19T05:45:00.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77181293399999995 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.915914+0000", 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2493999999999999e-05 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.916430+0000", 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00051667599999999998 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.122065+0000", 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.20563482999999999 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161930+0000", 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039865356999999997 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.161977+0000", 2024-09-19T05:45:00.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6873999999999999e-05 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162015+0000", 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7428999999999997e-05 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5293 3.3 3:c2b9c98f:::100000001f8.000006aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:45:00.993 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.727613735999999, 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.95068157399999997, 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5293 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211657+0000", 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.211659+0000", 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4700000000000001e-06 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215320+0000", 2024-09-19T05:45:00.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003660862 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215339+0000", 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8868999999999998e-05 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.215351+0000", 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1802e-05 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:00.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939166+0000", 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72381503599999997 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939258+0000", 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1911999999999997e-05 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.939406+0000", 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014874699999999999 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162132+0000", 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.222725545 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162260+0000", 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012829700000000001 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162304+0000", 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4215000000000003e-05 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162338+0000", 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3819000000000003e-05 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5301 3.c 3:3307abed:::100000001f8.000006b2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.718627957999999, 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75017270800000002, 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:00.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5301 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220643+0000", 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.220645+0000", 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0509999999999999e-06 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238885+0000", 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018240014999999998 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238906+0000", 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1498999999999999e-05 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.238919+0000", 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2565000000000001e-05 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841065+0000", 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60214654400000001 2024-09-19T05:45:01.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841146+0000", 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0688999999999999e-05 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.841342+0000", 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019650000000000001 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878211+0000", 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.036868724999999998 2024-09-19T05:45:01.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970700+0000", 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092489022000000004 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970754+0000", 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3480999999999998e-05 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970815+0000", 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1617000000000003e-05 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5312 3.2c 3:36326779:::100000001f8.000006bd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.680620679, 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71229407300000003, 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.005 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5312 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258650+0000", 2024-09-19T05:45:01.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258653+0000", 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277446+0000", 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018793994000000001 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277464+0000", 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7181e-05 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277476+0000", 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2167999999999999e-05 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878243+0000", 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60076709500000003 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878346+0000", 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103516 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.878493+0000", 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000146205 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970686+0000", 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.092192838999999999 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970867+0000", 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018116700000000001 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970904+0000", 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7474000000000001e-05 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970944+0000", 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9711e-05 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5315 3.3 3:c11688a5:::100000001f8.000006c0:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.661767597000001, 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1629463150000001, 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5315 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277503+0000", 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.277505+0000", 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-06 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280350+0000", 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028443909999999999 2024-09-19T05:45:01.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280373+0000", 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3711999999999999e-05 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280386+0000", 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2252999999999999e-05 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188178+0000", 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90779182800000002 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188297+0000", 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119034 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.188468+0000", 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000171139 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.319232+0000", 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13076428000000001 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434608+0000", 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115376337 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440394+0000", 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0057860999999999998 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.440449+0000", 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4761e-05 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5319 3.3b 3:de9a4780:::100000001f8.000006c4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.658859563, 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1541835229999999, 2024-09-19T05:45:01.017 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5319 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280411+0000", 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.280414+0000", 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9969999999999999e-06 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285680+0000", 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052661560000000001 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285697+0000", 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6883000000000001e-05 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.285712+0000", 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4892000000000001e-05 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253316+0000", 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.96760461799999997 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.253448+0000", 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013193499999999999 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434435+0000", 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.18098683800000001 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434490+0000", 2024-09-19T05:45:01.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4833000000000002e-05 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.434594+0000", 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010437099999999999 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5325 3.c 3:328b8b05:::100000001f8.000006ca:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.646765923, 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75472665100000003, 2024-09-19T05:45:01.022 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5325 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292505+0000", 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.292508+0000", 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3699999999999999e-06 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301580+0000", 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0090716010000000003 2024-09-19T05:45:01.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301598+0000", 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8278999999999999e-05 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301610+0000", 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.184e-05 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893481+0000", 2024-09-19T05:45:01.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.59187088300000001 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893784+0000", 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030305399999999998 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.893972+0000", 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018840600000000001 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.972882+0000", 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.078910441999999997 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047120+0000", 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.074237769999999995 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047176+0000", 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6224000000000002e-05 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047231+0000", 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4781999999999998e-05 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5327 3.2c 3:347123bb:::100000001f8.000006cc:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.637632192000002, 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93214956400000004, 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5327 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301638+0000", 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.301641+0000", 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.903e-06 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.304411+0000", 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027695879999999999 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336856+0000", 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.032444796999999997 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.336884+0000", 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8246999999999999e-05 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.970976+0000", 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.634091768 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971067+0000", 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1092000000000001e-05 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.971228+0000", 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016114800000000001 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047103+0000", 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.075875451999999996 2024-09-19T05:45:01.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.221630+0000", 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17452633000000001 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233731+0000", 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012101569 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.233788+0000", 2024-09-19T05:45:01.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6669999999999998e-05 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5355 3.1c 3:38412458:::100000001f8.000006e8:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.460781558000001, 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89709229800000001, 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5355 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478489+0000", 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478493+0000", 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9929999999999997e-06 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488576+0000", 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010082702000000001 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488591+0000", 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5607000000000002e-05 2024-09-19T05:45:01.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488601+0000", 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0035e-05 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047402+0000", 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.55880116199999996 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047498+0000", 2024-09-19T05:45:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5828000000000005e-05 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.047655+0000", 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015719799999999999 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180902+0000", 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13324624500000001 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375474+0000", 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19457263499999999 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375539+0000", 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4295999999999996e-05 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375581+0000", 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2596999999999999e-05 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5365 3.34 3:2ec8418e:::100000001f8.000006f2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.450645038000001, 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.88704814600000004, 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5365 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.040 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488625+0000", 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.488628+0000", 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4760000000000001e-06 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505248+0000", 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016620078999999999 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505273+0000", 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4527000000000001e-05 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505283+0000", 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0351e-05 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073161+0000", 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.56787809499999997 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073310+0000", 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014909799999999999 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.073513+0000", 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000203088 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180947+0000", 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.10743341100000001 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375593+0000", 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19464604999999999 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375634+0000", 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1115999999999997e-05 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375674+0000", 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9854999999999997e-05 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5367 3.1c 3:38083d86:::100000001f8.000006f4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.433969267999998, 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87052496000000001, 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5367 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505301+0000", 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.505303+0000", 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.198e-06 2024-09-19T05:45:01.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527093+0000", 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021790053 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527114+0000", 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0774000000000001e-05 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.527127+0000", 2024-09-19T05:45:01.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2292e-05 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110650+0000", 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58352302099999998 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.110816+0000", 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166174 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:01.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.113130+0000", 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023145470000000001 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.180979+0000", 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.067848435999999998 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375735+0000", 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.19475654100000001 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375783+0000", 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7478000000000002e-05 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.375826+0000", 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3445999999999997e-05 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5404 3.2c 3:357d9333:::100000001f8.00000719:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.288127529, 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75485259100000002, 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5404 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651143+0000", 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.651146+0000", 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7750000000000001e-06 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.666480+0000", 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015334166999999999 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709279+0000", 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042798738000000003 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.709297+0000", 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8119999999999999e-05 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242624+0000", 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.53332776400000004 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242737+0000", 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000112262 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.242935+0000", 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019831499999999999 2024-09-19T05:45:01.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405710+0000", 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.162774789 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405918+0000", 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020817599999999999 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405962+0000", 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3606999999999998e-05 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.405996+0000", 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3878000000000003e-05 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.057 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:01.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.944+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba414f0a0 con 0x7f7ba414eb10 2024-09-19T05:45:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.945+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba414f0a0 con 0x7f7ba414eb10 2024-09-19T05:45:01.059 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:45:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:45:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.946+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ba40cb2c0 0x7f7ba40cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.946+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f7ba40cbc40 con 0x7f7ba40cb2c0 2024-09-19T05:45:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.946+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ba40cb2c0 0x7f7ba40cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.946+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ba40cb2c0 0x7f7ba40cb700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.947+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba40cbc40 con 0x7f7ba40cb2c0 2024-09-19T05:45:01.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-19T05:45:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.961+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba40cbf10 con 0x7f7ba40cb2c0 2024-09-19T05:45:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.963+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba40cbf10 con 0x7f7ba40cb2c0 2024-09-19T05:45:01.062 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.963+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ba402c640 0x7f7ba402cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.963+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f7ba402d000 con 0x7f7ba402c640 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.964+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ba402c640 0x7f7ba402cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.964+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ba402c640 0x7f7ba402cac0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.965+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba402d000 con 0x7f7ba402c640 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.978+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba402d000 con 0x7f7ba402c640 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.980+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba402d000 con 0x7f7ba402c640 2024-09-19T05:45:01.063 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:45:01.064 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:45:01.064 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.980+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f7ba402d680 0x7f7ba4206f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.980+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f7ba4209d50 con 0x7f7ba402d680 2024-09-19T05:45:01.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.981+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f7ba402d680 0x7f7ba4206f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.981+0000 7f7ba1d9b640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f7ba402d680 0x7f7ba4206f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.982+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.5 v2:172.21.15.123:6800/2989037926 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba4209d50 con 0x7f7ba402d680 2024-09-19T05:45:01.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.996+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba42074c0 con 0x7f7ba402d680 2024-09-19T05:45:01.065 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.998+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.5 v2:172.21.15.123:6800/2989037926 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba42074c0 con 0x7f7ba402d680 2024-09-19T05:45:01.065 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:45:01.066 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:45:01.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.998+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ba4207b40 0x7f7ba4207f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.998+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f7ba420d0e0 con 0x7f7ba4207b40 2024-09-19T05:45:01.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.999+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ba4207b40 0x7f7ba4207f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.066 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:00.999+0000 7f7ba2d9d640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ba4207b40 0x7f7ba4207f80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.000+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba420d0e0 con 0x7f7ba4207b40 2024-09-19T05:45:01.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.013+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba42085d0 con 0x7f7ba4207b40 2024-09-19T05:45:01.067 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.015+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba42085d0 con 0x7f7ba4207b40 2024-09-19T05:45:01.067 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.015+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ba4208c50 0x7f7ba42090b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.015+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f7ba4209810 con 0x7f7ba4208c50 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.015+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ba4208c50 0x7f7ba42090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.016+0000 7f7ba259c640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ba4208c50 0x7f7ba42090b0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.016+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba4209810 con 0x7f7ba4208c50 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.030+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7ba4209790 con 0x7f7ba4208c50 2024-09-19T05:45:01.068 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7b9affd640 1 -- 172.21.15.2:0/2332330704 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7ba4209790 con 0x7f7ba4208c50 2024-09-19T05:45:01.069 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ba4150620 msgr2=0x7f7ba4159350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f7ba4150620 0x7f7ba4159350 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ba4154240 msgr2=0x7f7ba415c530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7f7ba4154240 0x7f7ba415c530 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ba414eb10 msgr2=0x7f7ba4062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7ba414eb10 0x7f7ba4062630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ba40cb2c0 msgr2=0x7f7ba40cb700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f7ba40cb2c0 0x7f7ba40cb700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ba402c640 msgr2=0x7f7ba402cac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.071 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f7ba402c640 0x7f7ba402cac0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f7ba402d680 msgr2=0x7f7ba4206f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba2d9d640 1 -- 172.21.15.2:0/2332330704 reap_dead start 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f7ba402d680 0x7f7ba4206f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba2d9d640 1 -- 172.21.15.2:0/2332330704 reap_dead start 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba2d9d640 1 -- 172.21.15.2:0/2332330704 reap_dead start 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ba4207b40 msgr2=0x7f7ba4207f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f7ba4207b40 0x7f7ba4207f80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ba4208c50 msgr2=0x7f7ba42090b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f7ba4208c50 0x7f7ba42090b0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7b8406c2b0 msgr2=0x7f7b8406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7b8406c2b0 0x7f7b8406e770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c033000 tx=0x7f7b9c007000 comp rx=0 tx=0).stop 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ba4153dc0 msgr2=0x7f7ba4150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.032+0000 7f7ba8d68640 1 --2- 172.21.15.2:0/2332330704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ba4153dc0 0x7f7ba4150ab0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c011a70 tx=0x7f7b8c011f30 comp rx=0 tx=0).stop 2024-09-19T05:45:01.072 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.033+0000 7f7ba2d9d640 1 -- 172.21.15.2:0/2332330704 reap_dead start 2024-09-19T05:45:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.033+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 shutdown_connections 2024-09-19T05:45:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.033+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 >> 172.21.15.2:0/2332330704 conn(0x7f7ba4067b30 msgr2=0x7f7ba40c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.033+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 shutdown_connections 2024-09-19T05:45:01.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.033+0000 7f7ba8d68640 1 -- 172.21.15.2:0/2332330704 wait complete. 2024-09-19T05:45:01.073 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:45:01.058+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:45:01.159 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:45:01.159+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:45:01.209 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:01.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:45:01.209 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:45:01.209 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:01.218 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-19T05:45:01.260 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-09-19T05:45:01.259+0000 7fb959a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37601) UID: 0 2024-09-19T05:45:01.361 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:45:01.360+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:45:01.365 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:45:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:45:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:45:01.365 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:01.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-19T05:45:01.501 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:45:01.462+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:45:01.524 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:45:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:45:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:45:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5482 3.27 3:e79b25dc:::100000001f8.00000767:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:45:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.491222445999998, 2024-09-19T05:45:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71036533999999996, 2024-09-19T05:45:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5482 2024-09-19T05:45:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030318+0000", 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.030320+0000", 2024-09-19T05:45:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3929999999999998e-06 2024-09-19T05:45:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034225+0000", 2024-09-19T05:45:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003904678 2024-09-19T05:45:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034278+0000", 2024-09-19T05:45:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2534999999999998e-05 2024-09-19T05:45:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.034303+0000", 2024-09-19T05:45:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.565e-05 2024-09-19T05:45:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.583640+0000", 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.54933696700000001 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.583850+0000", 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000210165 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740511+0000", 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.156660363 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740567+0000", 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5918999999999998e-05 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.740683+0000", 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011667000000000001 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5507 3.27 3:e4f54558:::100000001f8.00000780:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.472454486, 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71559931600000004, 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5507 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049086+0000", 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.049088+0000", 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4990000000000001e-06 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.070701+0000", 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021612672999999999 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087613+0000", 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016911942999999999 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087634+0000", 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0835e-05 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.707112+0000", 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61947850400000004 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.707259+0000", 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014680800000000001 2024-09-19T05:45:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764542+0000", 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.057282971000000002 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764599+0000", 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6934000000000001e-05 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.764685+0000", 2024-09-19T05:45:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6149000000000005e-05 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5521 3.27 3:e73f7143:::100000001f8.0000078e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.433884741, 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76359052999999999, 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5521 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087656+0000", 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.087659+0000", 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9229999999999998e-06 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107649+0000", 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.019990396000000001 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107667+0000", 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7958999999999999e-05 2024-09-19T05:45:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.107678+0000", 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1286e-05 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.742986+0000", 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.635308188 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.743091+0000", 2024-09-19T05:45:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010469200000000001 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851137+0000", 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.108045961 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851180+0000", 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3192999999999998e-05 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.851246+0000", 2024-09-19T05:45:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5932000000000005e-05 2024-09-19T05:45:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5725 3.27 3:e65abc03:::100000001f8.0000085a:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:45:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.216503551999999, 2024-09-19T05:45:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.755317554, 2024-09-19T05:45:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5725 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:45:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305037+0000", 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.305041+0000", 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1030000000000004e-06 2024-09-19T05:45:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307067+0000", 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020258870000000001 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307085+0000", 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7931000000000002e-05 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307098+0000", 2024-09-19T05:45:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3426e-05 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.914327+0000", 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60722910699999999 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.914435+0000", 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010719 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060107+0000", 2024-09-19T05:45:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.145671988 2024-09-19T05:45:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060199+0000", 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2615999999999996e-05 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060354+0000", 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015530599999999999 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5745 3.27 3:e6f7c8d2:::100000001f8.0000086e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.214415626000001, 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75335722800000005, 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5745 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307125+0000", 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.307128+0000", 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9739999999999998e-06 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.308414+0000", 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001286389 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.309604+0000", 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001189884 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.309621+0000", 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6572999999999999e-05 2024-09-19T05:45:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925094+0000", 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61547286800000001 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.925185+0000", 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1120999999999997e-05 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060376+0000", 2024-09-19T05:45:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.13519163200000001 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060418+0000", 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1703999999999999e-05 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060482+0000", 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4083000000000003e-05 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5763 3.27 3:e74ad6a4:::100000001f8.00000880:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.208874770000001, 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76551648800000005, 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5763 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312666+0000", 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.312668+0000", 2024-09-19T05:45:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.571e-06 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316538+0000", 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038699429999999998 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316577+0000", 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9180000000000001e-05 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.316596+0000", 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8332999999999999e-05 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.950346+0000", 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63375050600000005 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.950512+0000", 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016611800000000001 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078037+0000", 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12752486599999999 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078090+0000", 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2540000000000002e-05 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078182+0000", 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2430999999999996e-05 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5824 3.1f 3:fb07d980:::100000001f8.000008bd:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.183751883999999, 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73478956900000003, 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5824 2024-09-19T05:45:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337789+0000", 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.337792+0000", 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0419999999999998e-06 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347160+0000", 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093682149999999992 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347177+0000", 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6806000000000001e-05 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.347187+0000", 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0033e-05 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.949530+0000", 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.60234381400000003 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.949666+0000", 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013539300000000001 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070735+0000", 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.121069458 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.072015+0000", 2024-09-19T05:45:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012792369999999999 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.072578+0000", 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00056357100000000002 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5844 3.1f 3:fac155ab:::100000001f8.000008d1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.161673607000001, 2024-09-19T05:45:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73934379100000003, 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5844 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359867+0000", 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359869+0000", 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.543e-06 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363862+0000", 2024-09-19T05:45:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0039926709999999997 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363876+0000", 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3951000000000001e-05 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363891+0000", 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4951e-05 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.975926+0000", 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61203488500000003 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.976042+0000", 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011658300000000001 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099026+0000", 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.122983839 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099101+0000", 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4606999999999999e-05 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099211+0000", 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010976099999999999 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5846 3.27 3:e6e86654:::100000001f8.000008d3:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.157631245000001, 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.71445892200000005, 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5846 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:45:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363909+0000", 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.363912+0000", 2024-09-19T05:45:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2819999999999999e-06 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367743+0000", 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038313359999999999 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367760+0000", 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6892e-05 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367774+0000", 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4248e-05 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.016385+0000", 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.64861128400000001 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.016504+0000", 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000118838 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078254+0000", 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.061749399000000003 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078303+0000", 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9301000000000001e-05 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.078368+0000", 2024-09-19T05:45:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5341999999999999e-05 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5863 3.27 3:e5e0177a:::100000001f8.000008e4:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.153740625000001, 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.74712701699999995, 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5863 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367800+0000", 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.367802+0000", 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6479999999999999e-06 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372984+0000", 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051814299999999999 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373015+0000", 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0611000000000002e-05 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373054+0000", 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9690000000000001e-05 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.031807+0000", 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65875312100000005 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.038739+0000", 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069312100000000001 2024-09-19T05:45:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114756+0000", 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.076017243999999998 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114836+0000", 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0644999999999998e-05 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114927+0000", 2024-09-19T05:45:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0418000000000001e-05 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5876 3.27 3:e58ad412:::100000001f8.000008f1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.148466207999999, 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.74196762800000005, 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5876 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373074+0000", 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.373077+0000", 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7520000000000001e-06 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378577+0000", 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0054999849999999998 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378607+0000", 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9850999999999999e-05 2024-09-19T05:45:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.378625+0000", 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8148e-05 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060530+0000", 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.68190477100000002 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.060633+0000", 2024-09-19T05:45:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010288700000000001 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114938+0000", 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054305327 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.114977+0000", 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8602000000000001e-05 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115042+0000", 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5304999999999999e-05 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5918 3.27 3:e7c6e76b:::100000001f8.0000091b:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.127677631000001, 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.72128804099999999, 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5918 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393863+0000", 2024-09-19T05:45:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393866+0000", 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2550000000000002e-06 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397678+0000", 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0038118190000000001 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397703+0000", 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4592000000000001e-05 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397720+0000", 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7348999999999999e-05 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070763+0000", 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.67304348999999997 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.070867+0000", 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103608 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115051+0000", 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044183605000000001 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115088+0000", 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7394999999999997e-05 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.115151+0000", 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2928000000000005e-05 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5925 3.1f 3:f886b5f6:::100000001f8.00000922:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.123783573000001, 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.77643015699999995, 2024-09-19T05:45:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5925 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397757+0000", 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.397761+0000", 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.963e-06 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421926+0000", 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024164986999999999 2024-09-19T05:45:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421953+0000", 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6874000000000001e-05 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421966+0000", 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3413e-05 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.077691+0000", 2024-09-19T05:45:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.65572456599999995 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.077798+0000", 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010718299999999999 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.173642+0000", 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.095844253000000004 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174071+0000", 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000428658 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174187+0000", 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011626 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5932 3.1f 3:f86d9493:::100000001f8.00000929:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.099545238000001, 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.85551307099999996, 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5932 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421995+0000", 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.421998+0000", 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7429999999999998e-06 2024-09-19T05:45:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432488+0000", 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010489777 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432514+0000", 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6315000000000001e-05 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.432526+0000", 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1926999999999999e-05 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.102287+0000", 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.66976068499999997 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.102472+0000", 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018525900000000001 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277214+0000", 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.17474197899999999 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277291+0000", 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7194999999999995e-05 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277508+0000", 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021719099999999999 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6011 3.27 3:e774c081:::100000001f8.00000978:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.079032050999999, 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78469899799999998, 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6011 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442508+0000", 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.442511+0000", 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7300000000000001e-06 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.511020+0000", 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068508736000000001 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517290+0000", 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0062700580000000002 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517329+0000", 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8649000000000002e-05 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099275+0000", 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.58194679299999996 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.099398+0000", 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012280099999999999 2024-09-19T05:45:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.226947+0000", 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12754907100000001 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.227099+0000", 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000151707 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.227207+0000", 2024-09-19T05:45:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108453 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6019 3.1f 3:f909e746:::100000001f8.00000980:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.004166389000002, 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.76027384200000003, 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6019 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517374+0000", 2024-09-19T05:45:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.517381+0000", 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4439999999999996e-06 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.519989+0000", 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026080410000000002 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520018+0000", 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9522e-05 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520033+0000", 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5097e-05 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.151257+0000", 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.63122408200000002 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.151415+0000", 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000158197 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277523+0000", 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.126107891 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277569+0000", 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6066999999999998e-05 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277648+0000", 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8500999999999999e-05 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6020 3.1f 3:f8dd987c:::100000001f8.00000981:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.001474103, 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.75784574900000001, 2024-09-19T05:45:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6020 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520066+0000", 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.520069+0000", 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.508e-06 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.540152+0000", 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020082744999999999 2024-09-19T05:45:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.582892+0000", 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042740527 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.582906+0000", 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4331999999999999e-05 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162437+0000", 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57953022099999996 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.162537+0000", 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9968000000000002e-05 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277661+0000", 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.11512412700000001 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277747+0000", 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6367999999999997e-05 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.277912+0000", 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016495299999999999 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6052 3.27 3:e7b93799:::100000001f8.000009a1:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.929902496, 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70507024600000001, 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6052 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591638+0000", 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.591641+0000", 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.695e-06 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595294+0000", 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00365373 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595319+0000", 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4411999999999999e-05 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595333+0000", 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4157e-05 2024-09-19T05:45:01.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174616+0000", 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.57928295299999999 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.174703+0000", 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7268000000000005e-05 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296561+0000", 2024-09-19T05:45:01.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.121857365 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296626+0000", 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5643999999999997e-05 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.296708+0000", 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2021999999999995e-05 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6053 3.27 3:e77fb6c3:::100000001f8.000009a2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.926178771, 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70595110000000005, 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6053 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595362+0000", 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.595364+0000", 2024-09-19T05:45:01.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.452e-06 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596835+0000", 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001470636 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596860+0000", 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4909000000000002e-05 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596874+0000", 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4321000000000001e-05 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.211679+0000", 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.61480495499999999 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.211794+0000", 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114678 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301141+0000", 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.089347139000000006 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301215+0000", 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4632e-05 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301313+0000", 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7378000000000002e-05 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:6061 3.27 3:e4baa440:::100000001f8.000009aa:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.924638195, 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.70453015299999999, 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:01.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 6061 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596902+0000", 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.596905+0000", 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6000000000000001e-06 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598385+0000", 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001479878 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598409+0000", 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3751999999999999e-05 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.598418+0000", 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.218690+0000", 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.62027168499999996 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.218818+0000", 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000127983 2024-09-19T05:45:01.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301324+0000", 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082505675000000001 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301366+0000", 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1947999999999999e-05 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:01.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.301432+0000", 2024-09-19T05:45:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6905999999999999e-05 2024-09-19T05:45:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:01.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:01.638 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:01.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-19T05:45:01.640 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:45:01.638+0000 7f9b2dcb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 51427) UID: 0 2024-09-19T05:45:01.640 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-19T05:45:01.740 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:45:01.738+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:45:01.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.786+0000 7f1252e8d640 1 Processor -- start 2024-09-19T05:45:01.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.786+0000 7f1252e8d640 1 -- start start 2024-09-19T05:45:01.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1252e8d640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f124c154d60 0x7f124c0c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1252e8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c06c040 0x7f124c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1252e8d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f124c0739e0 con 0x7f124c0c8ee0 2024-09-19T05:45:01.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1252e8d640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f124c15ed70 con 0x7f124c154d60 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1252e8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f124c15e160 con 0x7f124c06c040 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f125142a640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f124c0c8ee0 0x7f124c15ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44028/0 (socket says 172.21.15.2:44028) 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f125142a640 1 -- 172.21.15.2:0/1740965355 learned_addr learned my addr 172.21.15.2:0/1740965355 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f124bfff640 1 --2- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c06c040 0x7f124c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1250c29640 1 --2- 172.21.15.2:0/1740965355 >> v2:172.21.15.123:3300/0 conn(0x7f124c154d60 0x7f124c0c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f124b7fe640 1 -- 172.21.15.2:0/1740965355 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857770523 0 0) 0x7f124c0739e0 con 0x7f124c0c8ee0 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f124b7fe640 1 -- 172.21.15.2:0/1740965355 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f123c003520 con 0x7f124c0c8ee0 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1250c29640 1 -- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c06c040 msgr2=0x7f124c0c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1250c29640 1 --2- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c06c040 0x7f124c0c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1250c29640 1 -- 172.21.15.2:0/1740965355 >> v1:172.21.15.2:6789/0 conn(0x7f124c0c8ee0 legacy=0x7f124c15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f1250c29640 1 -- 172.21.15.2:0/1740965355 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c15e440 con 0x7f124c154d60 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.787+0000 7f124bfff640 1 --2- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c06c040 0x7f124c0c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.788+0000 7f1250c29640 1 --2- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c154d60 0x7f124c0c8060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f12400097a0 tx=0x7f124002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c4a99637a16203 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.788+0000 7f124b7fe640 1 -- 172.21.15.2:0/1740965355 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240037070 con 0x7f124c154d60 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.788+0000 7f124b7fe640 1 -- 172.21.15.2:0/1740965355 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1240032440 con 0x7f124c154d60 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.788+0000 7f124b7fe640 1 -- 172.21.15.2:0/1740965355 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240032810 con 0x7f124c154d60 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 -- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c154d60 msgr2=0x7f124c0c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 --2- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c154d60 0x7f124c0c8060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f12400097a0 tx=0x7f124002a180 comp rx=0 tx=0).stop 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 -- 172.21.15.2:0/1740965355 shutdown_connections 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 --2- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c06c040 0x7f124c0c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 --2- 172.21.15.2:0/1740965355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c154d60 0x7f124c0c8060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 -- 172.21.15.2:0/1740965355 >> 172.21.15.2:0/1740965355 conn(0x7f124c06f3d0 msgr2=0x7f124c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 -- 172.21.15.2:0/1740965355 shutdown_connections 2024-09-19T05:45:01.792 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.789+0000 7f1252e8d640 1 -- 172.21.15.2:0/1740965355 wait complete. 2024-09-19T05:45:01.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.790+0000 7f1252e8d640 1 Processor -- start 2024-09-19T05:45:01.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.790+0000 7f1252e8d640 1 -- start start 2024-09-19T05:45:01.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1252e8d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c06c040 0x7f124c1f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1252e8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0c8ee0 0x7f124c14e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1252e8d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f124c154d60 0x7f124c152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1252e8d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f124c0c8b80 con 0x7f124c154d60 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1252e8d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f124c15ea00 con 0x7f124c06c040 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1252e8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f124c0c8d00 con 0x7f124c0c8ee0 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f124c154d60 0x7f124c152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f124c154d60 0x7f124c152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46502/0 (socket says 172.21.15.2:46502) 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 -- 172.21.15.2:0/3711602164 learned_addr learned my addr 172.21.15.2:0/3711602164 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1250c29640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c06c040 0x7f124c1f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f124bfff640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0c8ee0 0x7f124c14e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 -- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0c8ee0 msgr2=0x7f124c14e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0c8ee0 0x7f124c14e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 -- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c06c040 msgr2=0x7f124c1f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c06c040 0x7f124c1f3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:01.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f125142a640 1 -- 172.21.15.2:0/3711602164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c150560 con 0x7f124c154d60 2024-09-19T05:45:01.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f1250c29640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c06c040 0x7f124c1f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:01.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.791+0000 7f124bfff640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0c8ee0 0x7f124c14e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:01.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.792+0000 7f125142a640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f124c154d60 0x7f124c152670 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f124400cbe0 tx=0x7f124400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.792+0000 7f12497fa640 1 -- 172.21.15.2:0/3711602164 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244017070 con 0x7f124c154d60 2024-09-19T05:45:01.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.792+0000 7f12497fa640 1 -- 172.21.15.2:0/3711602164 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124400ace0 con 0x7f124c154d60 2024-09-19T05:45:01.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.792+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c150840 con 0x7f124c154d60 2024-09-19T05:45:01.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.792+0000 7f12497fa640 1 -- 172.21.15.2:0/3711602164 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244012600 con 0x7f124c154d60 2024-09-19T05:45:01.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.792+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c14db90 con 0x7f124c154d60 2024-09-19T05:45:01.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.793+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c0735c0 con 0x7f124c154d60 2024-09-19T05:45:01.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.794+0000 7f12497fa640 1 -- 172.21.15.2:0/3711602164 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f124401a070 con 0x7f124c154d60 2024-09-19T05:45:01.797 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:01.797 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:45:01.797 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:45:01.798 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:01.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.794+0000 7f12497fa640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f123006c2e0 0x7f123006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:01.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.795+0000 7f12497fa640 1 -- 172.21.15.2:0/3711602164 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5799+0+0 (secure 0 0 0) 0x7f124408b100 con 0x7f124c154d60 2024-09-19T05:45:01.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.795+0000 7f1250c29640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f123006c2e0 0x7f123006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:01.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.795+0000 7f1250c29640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f123006c2e0 0x7f123006e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1240034000 tx=0x7f124002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:01.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.799+0000 7f12497fa640 1 -- 172.21.15.2:0/3711602164 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f12440591c0 con 0x7f124c154d60 2024-09-19T05:45:01.805 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-19T05:45:01.849 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-19T05:45:01.848+0000 7f5a492e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36703) UID: 0 2024-09-19T05:45:01.914 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:01.914 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:45:01.914 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:45:01.914 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:45:01.914 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:01.923 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-19T05:45:01.942 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:45:01.940+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:45:01.989 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:01.988+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f124c0c4610 con 0x7f124c154d60 2024-09-19T05:45:02.043 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:45:02.042+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:45:02.073 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:02.073 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:45:02.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:45:02.073 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:45:02.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.073 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2140 3.3c 3:3ebd280f:::100000001f8.00000459:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.477540808000001, 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0038021129999999, 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2140 2024-09-19T05:45:02.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.592189+0000", 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.641694+0000", 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049504800000000002 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646652+0000", 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0049580040000000002 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646670+0000", 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7944e-05 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.646681+0000", 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1555e-05 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429025+0000", 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.782343268 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.429156+0000", 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013093299999999999 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.435554+0000", 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063981860000000002 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.549237+0000", 2024-09-19T05:45:02.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.113683458 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595633+0000", 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.046396021000000003 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595950+0000", 2024-09-19T05:45:02.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000316514 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.595991+0000", 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1430000000000001e-05 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2232 2.14 2:2e2fa760:::200.00000001:head [write 2396859~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.080 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.266826446, 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9689986130000001, 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802903+0000", 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802904+0000", 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8100000000000003e-07 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802912+0000", 2024-09-19T05:45:02.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9489999999999996e-06 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802920+0000", 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5970000000000003e-06 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802925+0000", 2024-09-19T05:45:02.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5040000000000002e-06 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730249+0000", 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9273239709999999 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730352+0000", 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102648 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.730514+0000", 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016198499999999999 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755135+0000", 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024621601 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771776+0000", 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016640773000000001 2024-09-19T05:45:02.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771841+0000", 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5178000000000005e-05 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771902+0000", 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0825999999999997e-05 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.086 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2233 2.14 2:2e2fa760:::200.00000001:head [write 2398939~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.26679635, 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969058076, 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.802934+0000", 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807300+0000", 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043666679999999998 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807312+0000", 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2199e-05 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807327+0000", 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4260999999999999e-05 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807339+0000", 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737540+0000", 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9302006619999998 2024-09-19T05:45:02.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737660+0000", 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119881 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.737878+0000", 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000218614 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755190+0000", 2024-09-19T05:45:02.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017311756000000001 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771913+0000", 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016722954000000002 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771958+0000", 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5123000000000001e-05 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771992+0000", 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3657e-05 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2234 2.14 2:2e2fa760:::200.00000001:head [write 2401019~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.262342598, 2024-09-19T05:45:02.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9661180190000001, 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807387+0000", 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807388+0000", 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-07 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807399+0000", 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1338999999999999e-05 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807409+0000", 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7259999999999997e-06 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807416+0000", 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1180000000000003e-06 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743599+0000", 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9361828380000001 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743702+0000", 2024-09-19T05:45:02.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103303 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.743868+0000", 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000165432 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.771998+0000", 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028130420999999999 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773391+0000", 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013930699999999999 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773448+0000", 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6672999999999998e-05 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773505+0000", 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7445000000000003e-05 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2235 2.14 2:2e2fa760:::200.00000001:head [write 2403099~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.262303383999999, 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9662057229999998, 2024-09-19T05:45:02.099 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807427+0000", 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4000000000000002e-07 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807435+0000", 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.072e-06 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807478+0000", 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2460999999999999e-05 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807486+0000", 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5350000000000004e-06 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.745914+0000", 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9384274700000002 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.747669+0000", 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017549239999999999 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.748294+0000", 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00062547500000000001 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773338+0000", 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025043950999999998 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773551+0000", 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021271199999999999 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773599+0000", 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7876e-05 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773632+0000", 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3707000000000001e-05 2024-09-19T05:45:02.106 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2236 2.14 2:2e2fa760:::200.00000001:head [write 2405179~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.262233266999999, 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9899550850000001, 2024-09-19T05:45:02.107 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807497+0000", 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5499999999999998e-07 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807502+0000", 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4850000000000003e-06 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807509+0000", 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0859999999999996e-06 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807515+0000", 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9179999999999999e-06 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755230+0000", 2024-09-19T05:45:02.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9477156679999998 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755342+0000", 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111703 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.755522+0000", 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000179937 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.783617+0000", 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028095472 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797341+0000", 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013723506999999999 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797404+0000", 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3187000000000003e-05 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797452+0000", 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7567e-05 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2237 2.14 2:2e2fa760:::200.00000001:head [write 2407259~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.113 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.262208594000001, 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0092068749999998, 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807521+0000", 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807522+0000", 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4999999999999998e-07 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807530+0000", 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8930000000000005e-06 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807577+0000", 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7064999999999997e-05 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807606+0000", 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8963e-05 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761580+0000", 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.953974901 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761685+0000", 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010435800000000001 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.761827+0000", 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000142313 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792591+0000", 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030764073999999999 2024-09-19T05:45:02.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816632+0000", 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024040371000000001 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816688+0000", 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5951e-05 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.816728+0000", 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0636000000000002e-05 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2238 2.14 2:2e2fa760:::200.00000001:head [write 2409339~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.262110739000001, 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0122208439999998, 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.119 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.807619+0000", 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834275+0000", 2024-09-19T05:45:02.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026656197999999999 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834287+0000", 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1691e-05 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834301+0000", 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3974e-05 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834310+0000", 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8659999999999995e-06 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772040+0000", 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9377302950000002 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772148+0000", 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108136 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.772314+0000", 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016526400000000001 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.792597+0000", 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020283483000000001 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819691+0000", 2024-09-19T05:45:02.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027094025000000001 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819771+0000", 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0022000000000001e-05 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819840+0000", 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8889999999999999e-05 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2239 2.14 2:2e2fa760:::200.00000001:head [write 2411419~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.235400756000001, 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.985646596, 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:45:02.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.834329+0000", 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851429+0000", 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017099436999999999 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851444+0000", 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5449000000000001e-05 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851460+0000", 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.556e-05 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851470+0000", 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0001e-05 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773666+0000", 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9221966369999999 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773757+0000", 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1160000000000001e-05 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.773902+0000", 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014419500000000001 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795500+0000", 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021598346000000001 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819898+0000", 2024-09-19T05:45:02.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024398474 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819942+0000", 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3151000000000001e-05 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.819976+0000", 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4186000000000001e-05 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2240 2.14 2:2e2fa760:::200.00000001:head [write 2413499~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.218242033999999, 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9685921560000001, 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.851488+0000", 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862188+0000", 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010699774 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862213+0000", 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5536000000000001e-05 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862225+0000", 2024-09-19T05:45:02.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1328000000000001e-05 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862231+0000", 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.55e-06 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.780995+0000", 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.918763854 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781115+0000", 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120538 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.781261+0000", 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000145785 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.795505+0000", 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014244158 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820012+0000", 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024506607 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820051+0000", 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8597000000000003e-05 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820080+0000", 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9428999999999999e-05 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2245 2.14 2:2e2fa760:::200.00000001:head [write 2423899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.207366119, 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969609441, 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862364+0000", 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.08e-07 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862368+0000", 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7929999999999999e-06 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862375+0000", 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6680000000000004e-06 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862380+0000", 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.575e-06 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797491+0000", 2024-09-19T05:45:02.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9351107490000001 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797581+0000", 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9559000000000001e-05 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.797723+0000", 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014230199999999999 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.820590+0000", 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022867386 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827763+0000", 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0071727190000000001 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.828797+0000", 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010343570000000001 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831973+0000", 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003175925 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2246 2.14 2:2e2fa760:::200.00000001:head [write 2425979~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.207342205, 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9696936549999999, 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862388+0000", 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.96e-07 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862397+0000", 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862404+0000", 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7730000000000002e-06 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862409+0000", 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1320000000000002e-06 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804733+0000", 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.94232443 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804840+0000", 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106724 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.804998+0000", 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015801500000000001 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.826045+0000", 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021046664999999999 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.831983+0000", 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059387570000000002 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832038+0000", 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4475999999999998e-05 2024-09-19T05:45:02.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832081+0000", 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3591000000000003e-05 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2247 2.14 2:2e2fa760:::200.00000001:head [write 2428059~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.207315124000001, 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.969738048, 2024-09-19T05:45:02.151 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862415+0000", 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.72e-07 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862426+0000", 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1090999999999999e-05 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862433+0000", 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8070000000000002e-06 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862438+0000", 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1599999999999997e-06 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806006+0000", 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.943567885 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806105+0000", 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8782000000000004e-05 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.806289+0000", 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183678 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.827739+0000", 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021450258999999999 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832087+0000", 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0043483280000000003 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832124+0000", 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6816000000000001e-05 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832153+0000", 2024-09-19T05:45:02.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.887e-05 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2248 2.14 2:2e2fa760:::200.00000001:head [write 2430139~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.207285125999999, 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9698627229999999, 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.157 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.862445+0000", 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877451+0000", 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015005956000000001 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.877461+0000", 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0034000000000001e-05 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.896137+0000", 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.018675967000000002 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909217+0000", 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013080629999999999 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807553+0000", 2024-09-19T05:45:02.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8983351339999999 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807647+0000", 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4289000000000005e-05 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.807794+0000", 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000147467 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832160+0000", 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024365753 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832224+0000", 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3689000000000006e-05 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832274+0000", 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0649000000000002e-05 2024-09-19T05:45:02.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832308+0000", 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3154999999999997e-05 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2250 2.14 2:2e2fa760:::200.00000001:head [write 2434299~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.159844276000001, 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9595307040000001, 2024-09-19T05:45:02.163 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.909886+0000", 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939724+0000", 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029838587 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.939743+0000", 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9077e-05 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941640+0000", 2024-09-19T05:45:02.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018968310000000001 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:30.941651+0000", 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0379e-05 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814230+0000", 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.872579934 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814430+0000", 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199481 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.814571+0000", 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014074399999999999 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.832173+0000", 2024-09-19T05:45:02.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017602199999999998 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869325+0000", 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037152223999999998 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869374+0000", 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9149e-05 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.869416+0000", 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2098000000000002e-05 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.169 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2337 3.3c 3:3eebf982:::100000001f8.0000051e:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:45:02.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.057698135999999, 2024-09-19T05:45:02.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.014058033, 2024-09-19T05:45:02.170 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2337 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.012032+0000", 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.018343+0000", 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063114770000000002 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030374+0000", 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012031086999999999 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030396+0000", 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.156e-05 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.030411+0000", 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4875000000000001e-05 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897205+0000", 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8667940299999999 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897418+0000", 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021300400000000001 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.897573+0000", 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015481300000000001 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.943251+0000", 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.045678424000000002 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026021+0000", 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.082769669000000004 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026059+0000", 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8296000000000001e-05 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.026090+0000", 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0797999999999999e-05 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2267 2.14 2:2e2fa760:::200.00000001:head [write 2469659~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 31.001055868000002, 2024-09-19T05:45:02.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9703313549999999, 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.068674+0000", 2024-09-19T05:45:02.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076283+0000", 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076092460000000001 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076295+0000", 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1857e-05 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076309+0000", 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3661e-05 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076319+0000", 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0589e-05 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963409+0000", 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.887089915 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963490+0000", 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1137000000000006e-05 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.963615+0000", 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012412999999999999 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012633+0000", 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049018171999999999 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038886+0000", 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026253216999999999 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.038950+0000", 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3876999999999998e-05 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039005+0000", 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5553999999999997e-05 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2268 2.14 2:2e2fa760:::200.00000001:head [write 2471739~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.993380370000001, 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962740785, 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:45:02.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076350+0000", 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999997e-07 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076361+0000", 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.049e-05 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076368+0000", 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4240000000000004e-06 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076374+0000", 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0179999999999998e-06 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975236+0000", 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.898862345 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975332+0000", 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5217999999999995e-05 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.975458+0000", 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125986 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012673+0000", 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.037215224999999998 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039014+0000", 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026341256 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039056+0000", 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1944999999999999e-05 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039090+0000", 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4301999999999998e-05 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.193 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2269 2.14 2:2e2fa760:::200.00000001:head [write 2473819~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.993346811999999, 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9627868230000001, 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076383+0000", 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076384+0000", 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3099999999999998e-07 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076392+0000", 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.174e-06 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076398+0000", 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4309999999999999e-06 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076403+0000", 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6399999999999996e-06 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976688+0000", 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9002852180000001 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976785+0000", 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6963e-05 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.976951+0000", 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016594799999999999 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012708+0000", 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035757376 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039097+0000", 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026388551 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039137+0000", 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0025999999999999e-05 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039170+0000", 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3065000000000003e-05 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(mds.0.4:2270 2.14 2:2e2fa760:::200.00000001:head [write 2475899~2080 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e70)", 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.993320847, 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.962839365, 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.076409+0000", 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102351+0000", 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025941703999999999 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102359+0000", 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9699999999999999e-06 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102372+0000", 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3749e-05 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.102382+0000", 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0229999999999998e-06 2024-09-19T05:45:02.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978315+0000", 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.875933294 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978422+0000", 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107007 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.978575+0000", 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015296099999999999 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.012746+0000", 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034170849000000003 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039176+0000", 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026430282999999999 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039215+0000", 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9368e-05 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.039248+0000", 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3157000000000001e-05 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.205 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:02.207 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-19T05:45:02.208 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:45:02.206+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:45:02.308 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-09-19T05:45:02.307+0000 7f9b2dcb3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 51427) UID: 0 2024-09-19T05:45:02.358 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:45:02.358 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-19T05:45:02.358 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:02.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-19T05:45:02.409 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:45:02.407+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:45:02.474 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:02.474 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-19T05:45:02.474 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 180, 2024-09-19T05:45:02.474 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-19T05:45:02.474 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:02.483 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-19T05:45:02.521 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:45:02.521 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:45:02.521 INFO:tasks.workunit.client.0.smithi002.stdout: 2024-09-19T05:45:02.521 INFO:tasks.workunit.client.0.smithi002.stdout:iozone test complete. 2024-09-19T05:45:02.522 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:45:02.520+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:45:02.523 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-19T05:45:02.523 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-19T05:45:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.581+0000 7f12497fa640 1 -- 172.21.15.2:0/3711602164 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v85) ==== 87+0+0 (secure 0 0 0) 0x7f124405d4c0 con 0x7f124c154d60 2024-09-19T05:45:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-19T05:45:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f123006c2e0 msgr2=0x7f123006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f123006c2e0 0x7f123006e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1240034000 tx=0x7f124002e000 comp rx=0 tx=0).stop 2024-09-19T05:45:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f124c154d60 msgr2=0x7f124c152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f124c154d60 0x7f124c152670 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f124400cbe0 tx=0x7f124400a430 comp rx=0 tx=0).stop 2024-09-19T05:45:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 shutdown_connections 2024-09-19T05:45:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f123006c2e0 0x7f123006e7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f124c154d60 0x7f124c152670 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f124c0c8ee0 0x7f124c14e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 --2- 172.21.15.2:0/3711602164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c06c040 0x7f124c1f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 >> 172.21.15.2:0/3711602164 conn(0x7f124c06f3d0 msgr2=0x7f124c15cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:02.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 shutdown_connections 2024-09-19T05:45:02.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:02.584+0000 7f1252e8d640 1 -- 172.21.15.2:0/3711602164 wait complete. 2024-09-19T05:45:02.602 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:02.602 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2666 3.9 3:92ca8d9b:::100000001f8.00000667:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.699332551000001, 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3293169570000001, 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2666 2024-09-19T05:45:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:45:02.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894725+0000", 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.894727+0000", 2024-09-19T05:45:02.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8439999999999998e-06 2024-09-19T05:45:02.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897799+0000", 2024-09-19T05:45:02.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030718199999999998 2024-09-19T05:45:02.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897817+0000", 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7847999999999999e-05 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897831+0000", 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3845999999999999e-05 2024-09-19T05:45:02.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897938+0000", 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010742399999999999 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898111+0000", 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017279099999999999 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.898360+0000", 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024885800000000002 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956644+0000", 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058284217999999999 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223960+0000", 2024-09-19T05:45:02.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.267316119 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224004+0000", 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4039000000000003e-05 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224042+0000", 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7150000000000002e-05 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2667 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:45:02.617 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.696192703000001, 2024-09-19T05:45:02.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3258909560000001, 2024-09-19T05:45:02.617 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2667 2024-09-19T05:45:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897864+0000", 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.897867+0000", 2024-09-19T05:45:02.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9299999999999999e-06 2024-09-19T05:45:02.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902567+0000", 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004699356 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902588+0000", 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1631e-05 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.902602+0000", 2024-09-19T05:45:02.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3478e-05 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.906894+0000", 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042925760000000002 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907078+0000", 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000183067 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.907289+0000", 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021123499999999999 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956671+0000", 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049382078000000003 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223652+0000", 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.266981122 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223704+0000", 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2244999999999999e-05 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223755+0000", 2024-09-19T05:45:02.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1238e-05 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2673 3.2d 3:b45a3f48:::100000001f8.0000066e:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.683195275999999, 2024-09-19T05:45:02.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.373980822, 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2673 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910862+0000", 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.910864+0000", 2024-09-19T05:45:02.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.525e-06 2024-09-19T05:45:02.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915100+0000", 2024-09-19T05:45:02.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042352259999999999 2024-09-19T05:45:02.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915123+0000", 2024-09-19T05:45:02.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3334999999999999e-05 2024-09-19T05:45:02.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915137+0000", 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4178999999999999e-05 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915220+0000", 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2693000000000003e-05 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915395+0000", 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017551099999999999 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.915574+0000", 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000178357 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951064+0000", 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.035490116000000002 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284764+0000", 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.333700259 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284808+0000", 2024-09-19T05:45:02.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3813999999999998e-05 2024-09-19T05:45:02.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.284843+0000", 2024-09-19T05:45:02.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4807000000000001e-05 2024-09-19T05:45:02.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2700 3.30 3:0e790ae4:::100000001f8.00000689:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:45:02.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.650493174000001, 2024-09-19T05:45:02.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.25937777, 2024-09-19T05:45:02.636 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.636 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2700 2024-09-19T05:45:02.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943564+0000", 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.943567+0000", 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.083e-06 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946402+0000", 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028349439999999998 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946419+0000", 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7105999999999999e-05 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.946429+0000", 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8709999999999992e-06 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.947951+0000", 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015219960000000001 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948153+0000", 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020153400000000001 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.948415+0000", 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00026250200000000002 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.064312+0000", 2024-09-19T05:45:02.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.115897369 2024-09-19T05:45:02.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202811+0000", 2024-09-19T05:45:02.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.138498891 2024-09-19T05:45:02.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202872+0000", 2024-09-19T05:45:02.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0529000000000003e-05 2024-09-19T05:45:02.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.202942+0000", 2024-09-19T05:45:02.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9944999999999995e-05 2024-09-19T05:45:02.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.645 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.646 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2711 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.646 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:45:02.646 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.642975565, 2024-09-19T05:45:02.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.27282242, 2024-09-19T05:45:02.646 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2711 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951082+0000", 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.951084+0000", 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.553e-06 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956906+0000", 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058222400000000002 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956926+0000", 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9619e-05 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.956937+0000", 2024-09-19T05:45:02.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0594999999999999e-05 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958546+0000", 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001609369 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.958792+0000", 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024587600000000002 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:31.959831+0000", 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010394079999999999 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293509+0000", 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33367725100000001 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223817+0000", 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93030886999999995 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223864+0000", 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6465000000000003e-05 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.223904+0000", 2024-09-19T05:45:02.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0173999999999999e-05 2024-09-19T05:45:02.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2732 3.9 3:92ca8380:::100000001f8.000006a9:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.586105961000001, 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2619013319999999, 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.655 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2732 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007951+0000", 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.007954+0000", 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018132+0000", 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010177734000000001 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018148+0000", 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6555999999999999e-05 2024-09-19T05:45:02.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.018187+0000", 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8760000000000002e-05 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020347+0000", 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021597589999999998 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020514+0000", 2024-09-19T05:45:02.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016697899999999999 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.020695+0000", 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018172199999999999 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.375158+0000", 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.35446218200000001 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269697+0000", 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89453943599999997 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269768+0000", 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0619999999999998e-05 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.269853+0000", 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4876000000000005e-05 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2746 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.558568748999999, 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1890195509999999, 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2746 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.661 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035488+0000", 2024-09-19T05:45:02.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.035491+0000", 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4499999999999998e-06 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043360+0000", 2024-09-19T05:45:02.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0078693240000000005 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.043383+0000", 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2853e-05 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.045231+0000", 2024-09-19T05:45:02.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001847476 2024-09-19T05:45:02.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.122884+0000", 2024-09-19T05:45:02.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.077653244999999996 2024-09-19T05:45:02.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123275+0000", 2024-09-19T05:45:02.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039126200000000002 2024-09-19T05:45:02.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.123895+0000", 2024-09-19T05:45:02.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061987200000000004 2024-09-19T05:45:02.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.293518+0000", 2024-09-19T05:45:02.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.169623147 2024-09-19T05:45:02.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224430+0000", 2024-09-19T05:45:02.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.93091173699999996 2024-09-19T05:45:02.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224474+0000", 2024-09-19T05:45:02.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3816000000000002e-05 2024-09-19T05:45:02.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.224508+0000", 2024-09-19T05:45:02.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4369000000000002e-05 2024-09-19T05:45:02.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.671 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.672 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2753 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.672 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:45:02.672 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.532545912, 2024-09-19T05:45:02.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.188208317, 2024-09-19T05:45:02.672 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2753 2024-09-19T05:45:02.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:45:02.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061511+0000", 2024-09-19T05:45:02.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.061513+0000", 2024-09-19T05:45:02.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.871e-06 2024-09-19T05:45:02.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078405+0000", 2024-09-19T05:45:02.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016891461 2024-09-19T05:45:02.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078433+0000", 2024-09-19T05:45:02.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8436e-05 2024-09-19T05:45:02.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.078447+0000", 2024-09-19T05:45:02.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3607e-05 2024-09-19T05:45:02.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137285+0000", 2024-09-19T05:45:02.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058837911 2024-09-19T05:45:02.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137457+0000", 2024-09-19T05:45:02.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017199399999999999 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.137637+0000", 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000180173 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.460405+0000", 2024-09-19T05:45:02.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.322768426 2024-09-19T05:45:02.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249610+0000", 2024-09-19T05:45:02.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78920498100000003 2024-09-19T05:45:02.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249664+0000", 2024-09-19T05:45:02.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3668000000000002e-05 2024-09-19T05:45:02.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.249720+0000", 2024-09-19T05:45:02.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5788999999999997e-05 2024-09-19T05:45:02.685 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:2808 3.11 3:8bd5b656:::100000001f8.000006f5:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e70)", 2024-09-19T05:45:02.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:45:02.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.353929649000001, 2024-09-19T05:45:02.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.160580451, 2024-09-19T05:45:02.687 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2808 2024-09-19T05:45:02.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:45:02.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240128+0000", 2024-09-19T05:45:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.240131+0000", 2024-09-19T05:45:02.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-19T05:45:02.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244285+0000", 2024-09-19T05:45:02.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041546919999999998 2024-09-19T05:45:02.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244303+0000", 2024-09-19T05:45:02.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7646999999999999e-05 2024-09-19T05:45:02.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.244316+0000", 2024-09-19T05:45:02.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2931999999999999e-05 2024-09-19T05:45:02.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373296+0000", 2024-09-19T05:45:02.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12897989000000001 2024-09-19T05:45:02.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373463+0000", 2024-09-19T05:45:02.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000166847 2024-09-19T05:45:02.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.373621+0000", 2024-09-19T05:45:02.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015846400000000001 2024-09-19T05:45:02.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:32.585995+0000", 2024-09-19T05:45:02.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.212374013 2024-09-19T05:45:02.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400581+0000", 2024-09-19T05:45:02.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.81458587900000001 2024-09-19T05:45:02.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400646+0000", 2024-09-19T05:45:02.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5036999999999994e-05 2024-09-19T05:45:02.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:33.400708+0000", 2024-09-19T05:45:02.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1937e-05 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5105 3.11 3:89c7a7f0:::100000001f8.000005ee:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:45:02.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.873638697000001, 2024-09-19T05:45:02.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2072587750000001, 2024-09-19T05:45:02.700 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5105 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720418+0000", 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.720422+0000", 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.241e-06 2024-09-19T05:45:02.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723834+0000", 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003412389 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723847+0000", 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2404e-05 2024-09-19T05:45:02.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723856+0000", 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6609999999999995e-06 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.723919+0000", 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2928999999999993e-05 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.724040+0000", 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120505 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.724239+0000", 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019924600000000001 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.782736+0000", 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.058496758000000003 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927592+0000", 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1448568539999999 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927639+0000", 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6495000000000001e-05 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927677+0000", 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8293000000000001e-05 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5127 3.11 3:8b184820:::100000001f8.00000604:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.829636257000001, 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.163370802, 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5127 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764421+0000", 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.764428+0000", 2024-09-19T05:45:02.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.778132+0000", 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013703958 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.790747+0000", 2024-09-19T05:45:02.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.012615167 2024-09-19T05:45:02.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.790767+0000", 2024-09-19T05:45:02.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9216999999999999e-05 2024-09-19T05:45:02.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.793911+0000", 2024-09-19T05:45:02.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031443019999999999 2024-09-19T05:45:02.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.794025+0000", 2024-09-19T05:45:02.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011406200000000001 2024-09-19T05:45:02.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.794225+0000", 2024-09-19T05:45:02.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001997 2024-09-19T05:45:02.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.836693+0000", 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042467799000000001 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927716+0000", 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0910237540000001 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927758+0000", 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1959999999999998e-05 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.927792+0000", 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3470000000000003e-05 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5160 3.14 3:2aefce75:::100000001f8.00000625:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.718541514999998, 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3210729219999999, 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5160 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875516+0000", 2024-09-19T05:45:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.875518+0000", 2024-09-19T05:45:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.723e-06 2024-09-19T05:45:02.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.880298+0000", 2024-09-19T05:45:02.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0047798010000000002 2024-09-19T05:45:02.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.888744+0000", 2024-09-19T05:45:02.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084454649999999992 2024-09-19T05:45:02.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.888764+0000", 2024-09-19T05:45:02.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0517e-05 2024-09-19T05:45:02.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889116+0000", 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000351391 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889240+0000", 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012396099999999999 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.889440+0000", 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00020088800000000001 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:42.939338+0000", 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.049897237999999997 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196475+0000", 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.257137682 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196526+0000", 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0680999999999998e-05 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.196589+0000", 2024-09-19T05:45:02.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2575000000000003e-05 2024-09-19T05:45:02.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5205 3.2d 3:b48c1fbb:::100000001f8.00000652:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:45:02.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.543653264, 2024-09-19T05:45:02.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1819966100000001, 2024-09-19T05:45:02.728 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5205 2024-09-19T05:45:02.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050404+0000", 2024-09-19T05:45:02.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.050407+0000", 2024-09-19T05:45:02.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5149999999999998e-06 2024-09-19T05:45:02.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053361+0000", 2024-09-19T05:45:02.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029540410000000001 2024-09-19T05:45:02.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053381+0000", 2024-09-19T05:45:02.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9471999999999999e-05 2024-09-19T05:45:02.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.053393+0000", 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2301e-05 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372360+0000", 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.318966784 2024-09-19T05:45:02.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372504+0000", 2024-09-19T05:45:02.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014391600000000001 2024-09-19T05:45:02.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.372718+0000", 2024-09-19T05:45:02.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021371699999999999 2024-09-19T05:45:02.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.499375+0000", 2024-09-19T05:45:02.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.12665742599999999 2024-09-19T05:45:02.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232287+0000", 2024-09-19T05:45:02.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.73291224399999999 2024-09-19T05:45:02.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232347+0000", 2024-09-19T05:45:02.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9920000000000002e-05 2024-09-19T05:45:02.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.232401+0000", 2024-09-19T05:45:02.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3273999999999998e-05 2024-09-19T05:45:02.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.740 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5227 3.11 3:88f3566a:::100000001f8.00000668:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.741 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:45:02.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.509142947000001, 2024-09-19T05:45:02.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.532865806, 2024-09-19T05:45:02.741 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5227 2024-09-19T05:45:02.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:45:02.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084914+0000", 2024-09-19T05:45:02.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.084919+0000", 2024-09-19T05:45:02.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7650000000000001e-06 2024-09-19T05:45:02.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091910+0000", 2024-09-19T05:45:02.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069906250000000003 2024-09-19T05:45:02.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091939+0000", 2024-09-19T05:45:02.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9274000000000001e-05 2024-09-19T05:45:02.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.091955+0000", 2024-09-19T05:45:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6280999999999999e-05 2024-09-19T05:45:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165174+0000", 2024-09-19T05:45:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.073219124999999996 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165300+0000", 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012565699999999999 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.165490+0000", 2024-09-19T05:45:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000190519 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.233620+0000", 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.068129713999999994 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617666+0000", 2024-09-19T05:45:02.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.384046068 2024-09-19T05:45:02.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617724+0000", 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7410000000000001e-05 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617780+0000", 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6368e-05 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.759 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5271 3.11 3:8addb635:::100000001f8.00000694:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.349130515999999, 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.372975533, 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5271 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:45:02.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244927+0000", 2024-09-19T05:45:02.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.244929+0000", 2024-09-19T05:45:02.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6460000000000002e-06 2024-09-19T05:45:02.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.251094+0000", 2024-09-19T05:45:02.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0061647539999999997 2024-09-19T05:45:02.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258025+0000", 2024-09-19T05:45:02.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00693065 2024-09-19T05:45:02.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258041+0000", 2024-09-19T05:45:02.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6708999999999999e-05 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258150+0000", 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010840000000000001 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.258311+0000", 2024-09-19T05:45:02.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000160688 2024-09-19T05:45:02.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.260274+0000", 2024-09-19T05:45:02.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001963027 2024-09-19T05:45:02.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.359521+0000", 2024-09-19T05:45:02.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.099247940000000007 2024-09-19T05:45:02.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617826+0000", 2024-09-19T05:45:02.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.258304525 2024-09-19T05:45:02.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617868+0000", 2024-09-19T05:45:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2454999999999999e-05 2024-09-19T05:45:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617902+0000", 2024-09-19T05:45:02.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3738999999999997e-05 2024-09-19T05:45:02.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.771 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.771 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5306 3.11 3:88876655:::100000001f8.000006b7:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.772 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:45:02.772 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.200514900000002, 2024-09-19T05:45:02.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.224487935, 2024-09-19T05:45:02.772 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.772 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5306 2024-09-19T05:45:02.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:45:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393542+0000", 2024-09-19T05:45:02.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.393546+0000", 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6389999999999998e-06 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.398380+0000", 2024-09-19T05:45:02.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0048341859999999999 2024-09-19T05:45:02.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.433026+0000", 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.034645428999999998 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.433052+0000", 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6945e-05 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.435900+0000", 2024-09-19T05:45:02.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028475969999999999 2024-09-19T05:45:02.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.436007+0000", 2024-09-19T05:45:02.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010702 2024-09-19T05:45:02.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.436183+0000", 2024-09-19T05:45:02.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00017633300000000001 2024-09-19T05:45:02.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531619+0000", 2024-09-19T05:45:02.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.095435107000000005 2024-09-19T05:45:02.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617951+0000", 2024-09-19T05:45:02.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0863320190000001 2024-09-19T05:45:02.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.617995+0000", 2024-09-19T05:45:02.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4474999999999997e-05 2024-09-19T05:45:02.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.618030+0000", 2024-09-19T05:45:02.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5185000000000003e-05 2024-09-19T05:45:02.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.783 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5313 3.11 3:8b36eb17:::100000001f8.000006be:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.156031202000001, 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2520088149999999, 2024-09-19T05:45:02.784 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5313 2024-09-19T05:45:02.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438026+0000", 2024-09-19T05:45:02.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.438028+0000", 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1610000000000001e-06 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440687+0000", 2024-09-19T05:45:02.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026591789999999998 2024-09-19T05:45:02.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440706+0000", 2024-09-19T05:45:02.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8236999999999998e-05 2024-09-19T05:45:02.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.440717+0000", 2024-09-19T05:45:02.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.171e-05 2024-09-19T05:45:02.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444350+0000", 2024-09-19T05:45:02.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036329819999999999 2024-09-19T05:45:02.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444438+0000", 2024-09-19T05:45:02.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7292000000000004e-05 2024-09-19T05:45:02.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.444578+0000", 2024-09-19T05:45:02.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014013099999999999 2024-09-19T05:45:02.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.684423+0000", 2024-09-19T05:45:02.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.239845633 2024-09-19T05:45:02.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.689914+0000", 2024-09-19T05:45:02.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.005491175 2024-09-19T05:45:02.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.689975+0000", 2024-09-19T05:45:02.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0943999999999998e-05 2024-09-19T05:45:02.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690035+0000", 2024-09-19T05:45:02.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9370999999999998e-05 2024-09-19T05:45:02.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.796 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.796 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5329 3.11 3:8be9d197:::100000001f8.000006ce:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.796 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.147342080000001, 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.24346679, 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5329 2024-09-19T05:45:02.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446715+0000", 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.446718+0000", 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2169999999999999e-06 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453551+0000", 2024-09-19T05:45:02.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0068323860000000002 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453567+0000", 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6538e-05 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.453579+0000", 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.218e-05 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.475998+0000", 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022418223000000001 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.477939+0000", 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001941639 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.478106+0000", 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016716699999999999 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.780733+0000", 2024-09-19T05:45:02.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.302626495 2024-09-19T05:45:02.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690099+0000", 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90936588600000001 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690144+0000", 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5019999999999999e-05 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.690182+0000", 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8039e-05 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5333 3.9 3:9102af4b:::100000001f8.000006d2:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.136068141999999, 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1967455760000001, 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5333 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457989+0000", 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.457991+0000", 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.08e-06 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462463+0000", 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044715370000000003 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462495+0000", 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2617999999999998e-05 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.462510+0000", 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4452e-05 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531565+0000", 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.069055635000000004 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531664+0000", 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8634999999999993e-05 2024-09-19T05:45:02.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.531822+0000", 2024-09-19T05:45:02.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015805600000000001 2024-09-19T05:45:02.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.867710+0000", 2024-09-19T05:45:02.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.33588781299999998 2024-09-19T05:45:02.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654623+0000", 2024-09-19T05:45:02.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78691265499999996 2024-09-19T05:45:02.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654683+0000", 2024-09-19T05:45:02.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0380000000000001e-05 2024-09-19T05:45:02.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654735+0000", 2024-09-19T05:45:02.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1715000000000002e-05 2024-09-19T05:45:02.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.815 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.816 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4534.0:5356 3.9 3:9157a25d:::100000001f8.000006e9:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:02.816 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:45:02.816 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.112248739000002, 2024-09-19T05:45:02.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.173045817, 2024-09-19T05:45:02.816 INFO:teuthology.orchestra.run.smithi002.stdout: "continuous": false, 2024-09-19T05:45:02.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-19T05:45:02.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:02.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-19T05:45:02.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4534", 2024-09-19T05:45:02.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:02.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 5356 2024-09-19T05:45:02.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.818 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-19T05:45:02.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-19T05:45:02.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:45:02.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-19T05:45:02.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481808+0000", 2024-09-19T05:45:02.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-19T05:45:02.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-19T05:45:02.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.481811+0000", 2024-09-19T05:45:02.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7159999999999999e-06 2024-09-19T05:45:02.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-19T05:45:02.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493724+0000", 2024-09-19T05:45:02.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011912668 2024-09-19T05:45:02.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-19T05:45:02.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493759+0000", 2024-09-19T05:45:02.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4968999999999997e-05 2024-09-19T05:45:02.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-19T05:45:02.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.493772+0000", 2024-09-19T05:45:02.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2845e-05 2024-09-19T05:45:02.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-19T05:45:02.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645428+0000", 2024-09-19T05:45:02.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.15165590900000001 2024-09-19T05:45:02.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645559+0000", 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013117000000000001 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 4", 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.645810+0000", 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000251432 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:43.867748+0000", 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.22193777100000001 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654779+0000", 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.78703058599999998 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-19T05:45:02.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654821+0000", 2024-09-19T05:45:02.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2540999999999998e-05 2024-09-19T05:45:02.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-19T05:45:02.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-19T05:45:02.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-19T05:45:02.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-19T05:44:44.654854+0000", 2024-09-19T05:45:02.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3210000000000002e-05 2024-09-19T05:45:02.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.829 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.829 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-19T05:45:02.830 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-19T05:45:02.830 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:02.832 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-19T05:45:02.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T05:45:02.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-19T05:45:02.837 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-09-19T05:45:02.835+0000 7f11b5948640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37603) UID: 0 2024-09-19T05:45:02.936 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-19T05:45:02.935+0000 7f5bfe3af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36693) UID: 0 2024-09-19T05:45:02.954 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:45:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:45:02.954 INFO:teuthology.orchestra.run.smithi123.stdout: "num_ops": 0 2024-09-19T05:45:02.955 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:45:02.964 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-19T05:45:03.037 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-09-19T05:45:03.035+0000 7fa5d02e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37606) UID: 0 2024-09-19T05:45:03.068 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:45:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [], 2024-09-19T05:45:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "complaint_time": 180, 2024-09-19T05:45:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: "num_blocked_ops": 0 2024-09-19T05:45:03.069 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:45:03.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-19T05:45:03.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-19T05:45:03.139 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-19T05:45:03.138+0000 7f7e39ec4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36694) UID: 0 2024-09-19T05:45:03.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.174+0000 7ff7a2a8c640 1 Processor -- start 2024-09-19T05:45:03.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.174+0000 7ff7a2a8c640 1 -- start start 2024-09-19T05:45:03.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a2a8c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c154dc0 0x7ff79c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a2a8c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff79c070880 0x7ff79c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a2a8c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79c0680b0 con 0x7ff79c06d030 2024-09-19T05:45:03.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a2a8c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ff79c15af80 con 0x7ff79c070880 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a2a8c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c067af0 con 0x7ff79c154dc0 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff79bfff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff79c070880 0x7ff79c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a1029640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff79c06d030 0x7ff79c15e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44032/0 (socket says 172.21.15.2:44032) 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a1029640 1 -- 172.21.15.2:0/137082972 learned_addr learned my addr 172.21.15.2:0/137082972 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff7a0828640 1 --2- 172.21.15.2:0/137082972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c154dc0 0x7ff79c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff79bfff640 1 -- 172.21.15.2:0/137082972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c154dc0 msgr2=0x7ff79c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff79bfff640 1 --2- 172.21.15.2:0/137082972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c154dc0 0x7ff79c15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff79bfff640 1 -- 172.21.15.2:0/137082972 >> v1:172.21.15.2:6789/0 conn(0x7ff79c06d030 legacy=0x7ff79c15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff79bfff640 1 -- 172.21.15.2:0/137082972 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c067dd0 con 0x7ff79c070880 2024-09-19T05:45:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.175+0000 7ff79b7fe640 1 -- 172.21.15.2:0/137082972 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066115035 0 0) 0x7ff79c0680b0 con 0x7ff79c06d030 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff79bfff640 1 --2- 172.21.15.2:0/137082972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c070880 0x7ff79c0cb8a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff7880097a0 tx=0x7ff78802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd3961030f87c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff79b7fe640 1 -- 172.21.15.2:0/137082972 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff79c070880 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff79b7fe640 1 -- 172.21.15.2:0/137082972 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788032440 con 0x7ff79c070880 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff79b7fe640 1 -- 172.21.15.2:0/137082972 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788032810 con 0x7ff79c070880 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/137082972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c070880 msgr2=0x7ff79c0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/137082972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c070880 0x7ff79c0cb8a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff7880097a0 tx=0x7ff78802a210 comp rx=0 tx=0).stop 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/137082972 shutdown_connections 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/137082972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c070880 0x7ff79c0cb8a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/137082972 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c154dc0 0x7ff79c15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/137082972 >> 172.21.15.2:0/137082972 conn(0x7ff79c06fcd0 msgr2=0x7ff79c0700e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:03.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/137082972 shutdown_connections 2024-09-19T05:45:03.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.177+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/137082972 wait complete. 2024-09-19T05:45:03.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 Processor -- start 2024-09-19T05:45:03.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 -- start start 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c06d030 0x7ff79c0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c070880 0x7ff79c1f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff79c154dc0 0x7ff79c1f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c15b950 con 0x7ff79c06d030 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c1551a0 con 0x7ff79c154dc0 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.179+0000 7ff7a2a8c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c1595a0 con 0x7ff79c070880 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c06d030 0x7ff79c0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c06d030 0x7ff79c0c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46516/0 (socket says 172.21.15.2:46516) 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 -- 172.21.15.2:0/960969105 learned_addr learned my addr 172.21.15.2:0/960969105 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff79bfff640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c070880 0x7ff79c1f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a1029640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff79c154dc0 0x7ff79c1f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c070880 msgr2=0x7ff79c1f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff79c070880 0x7ff79c1f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff79c154dc0 msgr2=0x7ff79c1f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff79c154dc0 0x7ff79c1f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c15a600 con 0x7ff79c06d030 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c06d030 0x7ff79c0c7260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff79000e870 tx=0x7ff79000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790017070 con 0x7ff79c06d030 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff79000c940 con 0x7ff79c06d030 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79c0c4b40 con 0x7ff79c06d030 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79000cd30 con 0x7ff79c06d030 2024-09-19T05:45:03.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.180+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79c0c5100 con 0x7ff79c06d030 2024-09-19T05:45:03.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.181+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff79c0c9110 con 0x7ff79c06d030 2024-09-19T05:45:03.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.182+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff79001a070 con 0x7ff79c06d030 2024-09-19T05:45:03.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.182+0000 7ff7997fa640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff78006c2e0 0x7ff78006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5799+0+0 (secure 0 0 0) 0x7ff79008b5a0 con 0x7ff79c06d030 2024-09-19T05:45:03.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff79bfff640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff78006c2e0 0x7ff78006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v85) ==== 54+0+16 (secure 0 0 0) 0x7ff790091170 con 0x7ff79c06d030 2024-09-19T05:45:03.187 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:03.188 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:45:03.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff79c0c39c0 0x7ff79c1f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff79c0cc000 con 0x7ff79c0c39c0 2024-09-19T05:45:03.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff7a1029640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff79c0c39c0 0x7ff79c1f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff79bfff640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff78006c2e0 0x7ff78006e7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.183+0000 7ff7a1029640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff79c0c39c0 0x7ff79c1f38c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.189 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.184+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c0cc000 con 0x7ff79c0c39c0 2024-09-19T05:45:03.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.198+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c075170 con 0x7ff79c0c39c0 2024-09-19T05:45:03.201 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-19T05:45:03.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.200+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c075170 con 0x7ff79c0c39c0 2024-09-19T05:45:03.202 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-19T05:45:03.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.200+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff79c1f4310 0x7ff79c144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.200+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff79c1f46f0 con 0x7ff79c1f4310 2024-09-19T05:45:03.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.200+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff79c1f4310 0x7ff79c144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.201+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff79c1f4310 0x7ff79c144e70 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.203 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.201+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c1f46f0 con 0x7ff79c1f4310 2024-09-19T05:45:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.216+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c074e90 con 0x7ff79c1f4310 2024-09-19T05:45:03.221 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.218+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c074e90 con 0x7ff79c1f4310 2024-09-19T05:45:03.222 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-19T05:45:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-19T05:45:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.218+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff79c1458c0 0x7ff79c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.222 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.218+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff79c145ca0 con 0x7ff79c1458c0 2024-09-19T05:45:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.218+0000 7ff79bfff640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff79c1458c0 0x7ff79c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.219+0000 7ff79bfff640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff79c1458c0 0x7ff79c062630 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.223 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.219+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c145ca0 con 0x7ff79c1458c0 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "size": 20, 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 600, 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "ops": [ 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5170 3.2 3:40ee304f:::100000001f8.0000062f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:45:03.224 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.370424108000002, 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.07064564, 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5170 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.225 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848606+0000", 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.848608+0000", 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6699999999999998e-06 2024-09-19T05:45:03.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861615+0000", 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013006636 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861637+0000", 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2447e-05 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861651+0000", 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3431000000000001e-05 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861732+0000", 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1124000000000005e-05 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861822+0000", 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9981000000000005e-05 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.228 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.861969+0000", 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00014714700000000001 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.868505+0000", 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0065363340000000004 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919134+0000", 2024-09-19T05:45:03.229 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.050628731 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919203+0000", 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.9207000000000003e-05 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.919251+0000", 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.7932000000000001e-05 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5191 3.2 3:41cf9845:::100000001f8.00000644:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.306905155999999, 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1864950409999999, 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5191 2024-09-19T05:45:03.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912125+0000", 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.232 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.912127+0000", 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9400000000000001e-06 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925516+0000", 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013389509000000001 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925536+0000", 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9579e-05 2024-09-19T05:45:03.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925544+0000", 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.3450000000000006e-06 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925643+0000", 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9018000000000006e-05 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925711+0000", 2024-09-19T05:45:03.234 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8351999999999999e-05 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.925816+0000", 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000104832 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:42.931570+0000", 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0057541279999999998 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.235 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098516+0000", 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1669453750000001 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098577+0000", 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.1736999999999995e-05 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098620+0000", 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2225999999999999e-05 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.236 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5264 3.2b 3:d5fb1e74:::100000001f8.0000068d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.218916534000002, 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.098400515, 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.237 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5264 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000113+0000", 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.000115+0000", 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0650000000000001e-06 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.239 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013720+0000", 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013604842000000001 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013737+0000", 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7173999999999999e-05 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013745+0000", 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4850000000000003e-06 2024-09-19T05:45:03.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013799+0000", 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4543000000000003e-05 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013869+0000", 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.0032999999999998e-05 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.241 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013997+0000", 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012786399999999999 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.018990+0000", 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049925860000000002 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098402+0000", 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0794119390000001 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.242 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098460+0000", 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8700999999999999e-05 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.098514+0000", 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.3282999999999998e-05 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.243 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5286 3.2 3:41c49e27:::100000001f8.000006a3:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.205264805999999, 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.368367997, 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5286 2024-09-19T05:45:03.244 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013765+0000", 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.013767+0000", 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0320000000000002e-06 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020719+0000", 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0069518940000000001 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020732+0000", 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3560999999999999e-05 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.246 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020743+0000", 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0156000000000001e-05 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020811+0000", 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8093e-05 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020888+0000", 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.7484000000000006e-05 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.247 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.021010+0000", 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012149 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032253+0000", 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.011243367000000001 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382067+0000", 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3498136039999999 2024-09-19T05:45:03.248 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382105+0000", 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8525000000000001e-05 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382133+0000", 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.7790999999999999e-05 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5291 3.2 3:433c69f2:::100000001f8.000006a8:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.198259949000001, 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.361485925, 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5291 2024-09-19T05:45:03.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020770+0000", 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.251 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.020771+0000", 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5850000000000001e-06 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027794+0000", 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070227889999999998 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027809+0000", 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.4959e-05 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027817+0000", 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1259999999999998e-06 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027886+0000", 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.8356999999999996e-05 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.027986+0000", 2024-09-19T05:45:03.253 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010082900000000001 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.028127+0000", 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000140829 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.032805+0000", 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0046781679999999999 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.254 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382178+0000", 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3493729270000001 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382219+0000", 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.0924999999999999e-05 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382256+0000", 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6430999999999997e-05 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.255 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5360 3.2b 3:d6b2cae8:::100000001f8.000006ed:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.061863552999998, 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1641596679999999, 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.256 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5360 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157166+0000", 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.157169+0000", 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6089999999999999e-06 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173455+0000", 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016286248 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.258 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173477+0000", 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1716999999999999e-05 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173488+0000", 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1620000000000001e-05 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173563+0000", 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4161999999999999e-05 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.259 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173653+0000", 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.9974000000000003e-05 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173776+0000", 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012363000000000001 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.260 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.179905+0000", 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0061287110000000002 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321250+0000", 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1413448260000001 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321287+0000", 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6937999999999998e-05 2024-09-19T05:45:03.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.321326+0000", 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.9233000000000002e-05 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5363 3.2 3:41c08734:::100000001f8.000006f0:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.045512998, 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2088135710000001, 2024-09-19T05:45:03.262 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5363 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173517+0000", 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.173519+0000", 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3089999999999998e-06 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.264 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186577+0000", 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013058449999999999 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186607+0000", 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9872000000000001e-05 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186618+0000", 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0772e-05 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.265 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186771+0000", 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000153024 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186856+0000", 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.4907999999999994e-05 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.186976+0000", 2024-09-19T05:45:03.266 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00011951 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.193253+0000", 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0062774969999999999 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382286+0000", 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1890333310000001 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.267 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382310+0000", 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4083e-05 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.382330+0000", 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.9814999999999999e-05 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.268 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5383 3.2 3:42776a0c:::100000001f8.00000704:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 20.005837596999999, 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.270768092, 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.269 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5383 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213192+0000", 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.270 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.213194+0000", 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1160000000000002e-06 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230888+0000", 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.017693296000000001 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230908+0000", 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0624000000000001e-05 2024-09-19T05:45:03.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230918+0000", 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.5119999999999997e-06 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231017+0000", 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9586000000000005e-05 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.272 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231189+0000", 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000172053 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.231464+0000", 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000274916 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.238533+0000", 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.007068469 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.273 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.483856+0000", 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2453235620000001 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.483912+0000", 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.5911000000000001e-05 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.483960+0000", 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.8047000000000003e-05 2024-09-19T05:45:03.274 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5390 3.2 3:4215b5dd:::100000001f8.0000070b:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.988056330999999, 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.253141504, 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.275 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5390 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230973+0000", 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.230976+0000", 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.419e-06 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244642+0000", 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.013666417 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244663+0000", 2024-09-19T05:45:03.277 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0454000000000001e-05 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244690+0000", 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.6962e-05 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244765+0000", 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4884999999999999e-05 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.278 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.244873+0000", 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000108904 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.245011+0000", 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013788200000000001 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.251134+0000", 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.006122677 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484025+0000", 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2328906040000001 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484074+0000", 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.9221000000000002e-05 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484115+0000", 2024-09-19T05:45:03.280 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1078999999999998e-05 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5403 3.2b 3:d515b219:::100000001f8.00000718:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.883922646999999, 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.99261407000000001, 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.281 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5403 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.282 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335107+0000", 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.335110+0000", 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.27e-06 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351404+0000", 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016293426 2024-09-19T05:45:03.283 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351429+0000", 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.5131999999999999e-05 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351442+0000", 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3281e-05 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351543+0000", 2024-09-19T05:45:03.284 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000100887 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351640+0000", 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.6569000000000003e-05 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351796+0000", 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015682900000000001 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.285 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.358222+0000", 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0064259030000000002 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327636+0000", 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.96941361299999995 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327678+0000", 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.1897000000000002e-05 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.286 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.327721+0000", 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.3263000000000002e-05 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5405 3.2 3:41109be1:::100000001f8.0000071a:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:45:03.287 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.867554309999999, 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.1327694880000001, 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5405 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:45:03.288 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351475+0000", 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.351477+0000", 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.846e-06 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.289 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367351+0000", 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015873627000000001 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367458+0000", 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106702 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367496+0000", 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.8389000000000001e-05 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.290 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367630+0000", 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013432400000000001 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367737+0000", 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010691700000000001 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.291 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.367859+0000", 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012197999999999999 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.373387+0000", 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0055282539999999998 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484166+0000", 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.110778155 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.292 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484208+0000", 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2738000000000003e-05 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484245+0000", 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.6556e-05 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.293 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5478 3.2b 3:d663163e:::100000001f8.00000763:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.725144588999999, 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.059029182, 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5478 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.294 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493885+0000", 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.493886+0000", 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2529999999999999e-06 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.295 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506674+0000", 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.012787552000000001 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506696+0000", 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.2259000000000002e-05 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506712+0000", 2024-09-19T05:45:03.296 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5965e-05 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506793+0000", 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1075000000000006e-05 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506944+0000", 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000151065 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.297 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.507151+0000", 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00020636600000000001 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.515901+0000", 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0087503040000000004 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552800+0000", 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.0368988699999999 2024-09-19T05:45:03.298 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552858+0000", 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.7877000000000002e-05 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552914+0000", 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6595999999999998e-05 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.299 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5491 3.2 3:418769c2:::100000001f8.00000770:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.712286209999998, 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.97763760899999996, 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5491 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.300 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506744+0000", 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.506746+0000", 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.8059999999999999e-06 2024-09-19T05:45:03.301 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.520997+0000", 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.014250966 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521021+0000", 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.368e-05 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.302 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521038+0000", 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.6708e-05 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521168+0000", 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013036300000000001 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521266+0000", 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.7849000000000005e-05 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.303 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.521457+0000", 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000191385 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.528522+0000", 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070645830000000001 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484294+0000", 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.95577166700000005 2024-09-19T05:45:03.304 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484344+0000", 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0512e-05 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484381+0000", 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.7089999999999999e-05 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.305 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5506 3.2b 3:d506a813:::100000001f8.0000077f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.679963989000001, 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.013993707, 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5506 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.306 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539066+0000", 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.539068+0000", 2024-09-19T05:45:03.307 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0710000000000002e-06 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555200+0000", 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.016132480000000001 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555230+0000", 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9728999999999999e-05 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.308 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555243+0000", 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2697999999999999e-05 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555334+0000", 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.1113999999999995e-05 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.555436+0000", 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00010182 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.309 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.556369+0000", 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00093375000000000001 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.562703+0000", 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063334139999999999 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.552979+0000", 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.99027636699999999 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.310 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553029+0000", 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0214999999999999e-05 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553059+0000", 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.0049e-05 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.311 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5530 3.2 3:43043671:::100000001f8.00000797:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.608012021, 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.87348732500000004, 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5530 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.312 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611018+0000", 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.313 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.611019+0000", 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2619999999999999e-06 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618028+0000", 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0070088199999999998 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618048+0000", 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.0616e-05 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.314 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618061+0000", 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.2678e-05 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618127+0000", 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.6276e-05 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618227+0000", 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9356000000000002e-05 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.315 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.618379+0000", 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000152151 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.623797+0000", 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0054177050000000001 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484423+0000", 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.860626522 2024-09-19T05:45:03.316 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484469+0000", 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6242999999999998e-05 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.484505+0000", 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.5695999999999998e-05 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.317 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5671 3.2b 3:d7657d1c:::100000001f8.00000824:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.51227214, 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.84642695999999995, 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5671 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.318 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706758+0000", 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.706760+0000", 2024-09-19T05:45:03.319 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.243e-06 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715117+0000", 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0083569019999999994 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715133+0000", 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.5818e-05 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.320 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715140+0000", 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.1849999999999998e-06 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715202+0000", 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2315000000000002e-05 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715304+0000", 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000101786 2024-09-19T05:45:03.321 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.715421+0000", 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000117333 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.720417+0000", 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0049962840000000001 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553108+0000", 2024-09-19T05:45:03.322 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.83269075100000001 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553150+0000", 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.2104000000000002e-05 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.553185+0000", 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 3.4239000000000002e-05 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.323 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5682 3.2 3:43bb8046:::100000001f8.0000082f:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.497020321000001, 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.90941017300000004, 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5682 2024-09-19T05:45:03.324 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722009+0000", 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.325 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.722012+0000", 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.1849999999999998e-06 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729368+0000", 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0073561759999999999 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729391+0000", 2024-09-19T05:45:03.326 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.3309e-05 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729404+0000", 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3016e-05 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729479+0000", 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.4914999999999997e-05 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729591+0000", 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000111607 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.729717+0000", 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00012588200000000001 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.735604+0000", 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0058875100000000003 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.328 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.631310+0000", 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.89570617100000005 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.631368+0000", 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.8213e-05 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.631420+0000", 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.1189000000000001e-05 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.329 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5723 3.2 3:4214d263:::100000001f8.00000858:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.451988828000001, 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.91036275700000002, 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.330 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5723 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767041+0000", 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.331 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.767043+0000", 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.041e-06 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782398+0000", 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.015355056000000001 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782422+0000", 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4190999999999999e-05 2024-09-19T05:45:03.332 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782441+0000", 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.8691000000000001e-05 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782517+0000", 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.6186000000000005e-05 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.333 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782591+0000", 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 7.3604999999999997e-05 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.782725+0000", 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00013430699999999999 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.786761+0000", 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004035673 2024-09-19T05:45:03.334 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677290+0000", 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.89052959600000003 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677353+0000", 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 6.2731000000000006e-05 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677404+0000", 2024-09-19T05:45:03.335 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.0680000000000003e-05 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5759 3.2 3:42bacaf4:::100000001f8.0000087c:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.397153657, 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.85564746599999997, 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.336 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5759 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821876+0000", 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.337 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.821879+0000", 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829742+0000", 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0078630269999999999 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.338 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829755+0000", 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.3308e-05 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829763+0000", 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.1839999999999999e-06 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829849+0000", 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 8.6017e-05 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.339 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.829955+0000", 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000106059 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.830113+0000", 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00015798799999999999 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.834572+0000", 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.004458755 2024-09-19T05:45:03.340 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677448+0000", 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.84287563099999996 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677477+0000", 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.9964999999999999e-05 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.341 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677524+0000", 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 4.6075999999999997e-05 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "description": "osd_op(client.4534.0:5776 3.2 3:42907a88:::100000001f8.0000088d:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected e74)", 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "initiated_at": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "age": 19.360420241, 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.81907758500000005, 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "continuous": false, 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "type_data": { 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-19T05:45:03.342 INFO:teuthology.orchestra.run.smithi123.stdout: "client_info": { 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "client": "client.4534", 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "client_addr": "v1:172.21.15.2:0/3907881870", 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "tid": 5776 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "events": [ 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "initiated", 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.343 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "header_read", 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858609+0000", 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "throttled", 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.858612+0000", 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 2.4559999999999999e-06 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "all_read", 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866396+0000", 2024-09-19T05:45:03.344 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0077837369999999998 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "dispatched", 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866413+0000", 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 1.7481000000000001e-05 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "queued_for_pg", 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866423+0000", 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 9.9799999999999993e-06 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.345 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "reached_pg", 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866567+0000", 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000143683 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "started", 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866677+0000", 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.000110042 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "waiting for subops from 2", 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.866889+0000", 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.00021198500000000001 2024-09-19T05:45:03.346 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "op_commit", 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:43.873285+0000", 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.0063963800000000001 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "sub_op_commit_rec", 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677576+0000", 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 0.80429050499999999 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.347 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "commit_sent", 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677632+0000", 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.6610999999999997e-05 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: { 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "event": "done", 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "time": "2024-09-19T05:44:44.677687+0000", 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: "duration": 5.4725000000000002e-05 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-19T05:45:03.348 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-19T05:45:03.350 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-19T05:45:03.350 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-19T05:45:03.360 INFO:tasks.thrashosds:joining thrashosds 2024-09-19T05:45:03.360 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-19T05:45:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.234+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c0c54b0 con 0x7ff79c1458c0 2024-09-19T05:45:03.361 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.236+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c0c54b0 con 0x7ff79c1458c0 2024-09-19T05:45:03.361 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-19T05:45:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-19T05:45:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.236+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff79c0ca790 0x7ff79c0cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.236+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff79c063440 con 0x7ff79c0ca790 2024-09-19T05:45:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.236+0000 7ff7a1029640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff79c0ca790 0x7ff79c0cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.362 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.236+0000 7ff7a1029640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff79c0ca790 0x7ff79c0cabb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.237+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c063440 con 0x7ff79c0ca790 2024-09-19T05:45:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.251+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c0cb0f0 con 0x7ff79c0ca790 2024-09-19T05:45:03.363 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.252+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c0cb0f0 con 0x7ff79c0ca790 2024-09-19T05:45:03.363 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-19T05:45:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-19T05:45:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.252+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff79c02c640 0x7ff79c02caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.364 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.252+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff79c02cfe0 con 0x7ff79c02c640 2024-09-19T05:45:03.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.253+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff79c02c640 0x7ff79c02caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.253+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff79c02c640 0x7ff79c02caa0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.254+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c02cfe0 con 0x7ff79c02c640 2024-09-19T05:45:03.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.267+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c02cfe0 con 0x7ff79c02c640 2024-09-19T05:45:03.365 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.269+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c02cfe0 con 0x7ff79c02c640 2024-09-19T05:45:03.365 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-19T05:45:03.366 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-19T05:45:03.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.269+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7ff79c02d660 0x7ff79c206e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.269+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff79c02dae0 con 0x7ff79c02d660 2024-09-19T05:45:03.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.269+0000 7ff79bfff640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7ff79c02d660 0x7ff79c206e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.269+0000 7ff79bfff640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7ff79c02d660 0x7ff79c206e60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.270+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.5 v2:172.21.15.123:6800/2989037926 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c02dae0 con 0x7ff79c02d660 2024-09-19T05:45:03.366 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.284+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c02dae0 con 0x7ff79c02d660 2024-09-19T05:45:03.367 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.286+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.5 v2:172.21.15.123:6800/2989037926 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c02dae0 con 0x7ff79c02d660 2024-09-19T05:45:03.367 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.286+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff79c2078b0 0x7ff79c209ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.286+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff79c20ca70 con 0x7ff79c2078b0 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.286+0000 7ff7a1029640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff79c2078b0 0x7ff79c209ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.287+0000 7ff7a1029640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff79c2078b0 0x7ff79c209ca0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.287+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c20ca70 con 0x7ff79c2078b0 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.301+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c20a1e0 con 0x7ff79c2078b0 2024-09-19T05:45:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.303+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c20a1e0 con 0x7ff79c2078b0 2024-09-19T05:45:03.369 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-19T05:45:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.303+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff79c20a900 0x7ff79c20ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.303+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff79c20fb90 con 0x7ff79c20a900 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.304+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff79c20a900 0x7ff79c20ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.304+0000 7ff7a0828640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff79c20a900 0x7ff79c20ccf0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.305+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff79c20fb90 con 0x7ff79c20a900 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.318+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7ff79c0683f0 con 0x7ff79c20a900 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.320+0000 7ff7997fa640 1 -- 172.21.15.2:0/960969105 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff79c0683f0 con 0x7ff79c20a900 2024-09-19T05:45:03.370 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-19T05:45:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-19T05:45:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff79c0c39c0 msgr2=0x7ff79c1f38c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ff79c0c39c0 0x7ff79c1f38c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff79c1f4310 msgr2=0x7ff79c144e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7ff79c1f4310 0x7ff79c144e70 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.373 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff79c1458c0 msgr2=0x7ff79c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7ff79c1458c0 0x7ff79c062630 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff79c0ca790 msgr2=0x7ff79c0cabb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a1029640 1 -- 172.21.15.2:0/960969105 reap_dead start 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7ff79c0ca790 0x7ff79c0cabb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff79c02c640 msgr2=0x7ff79c02caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7ff79c02c640 0x7ff79c02caa0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7ff79c02d660 msgr2=0x7ff79c206e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7ff79c02d660 0x7ff79c206e60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff79c2078b0 msgr2=0x7ff79c209ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7ff79c2078b0 0x7ff79c209ca0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff79c20a900 msgr2=0x7ff79c20ccf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7ff79c20a900 0x7ff79c20ccf0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.374 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a1029640 1 -- 172.21.15.2:0/960969105 reap_dead start 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff78006c2e0 msgr2=0x7ff78006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff78006c2e0 0x7ff78006e7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802e000 comp rx=0 tx=0).stop 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c06d030 msgr2=0x7ff79c0c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c06d030 0x7ff79c0c7260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff79000e870 tx=0x7ff79000ed30 comp rx=0 tx=0).stop 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 shutdown_connections 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff78006c2e0 0x7ff78006e7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 --2- 172.21.15.2:0/960969105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff79c06d030 0x7ff79c0c7260 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 >> 172.21.15.2:0/960969105 conn(0x7ff79c06fcd0 msgr2=0x7ff79c14e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 shutdown_connections 2024-09-19T05:45:03.375 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:03.321+0000 7ff7a2a8c640 1 -- 172.21.15.2:0/960969105 wait complete. 2024-09-19T05:45:03.376 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-19T05:45:03.349+0000 7fa147a40640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36708) UID: 0 2024-09-19T05:45:03.466 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:03.467 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:03.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-19T05:45:03.577 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:03.577 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:03.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-19T05:45:03.690 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:03.690 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:03.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-19T05:45:03.802 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:03.802 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:03.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-19T05:45:03.871 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:45:03.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:45:03.916 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:03.916 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:03.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-19T05:45:04.068 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:04.068 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:04.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-19T05:45:04.180 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:04.180 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:04.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-19T05:45:04.291 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:04.291 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:04.300 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-19T05:45:04.403 INFO:teuthology.orchestra.run.smithi123.stdout:ERROR: (22) Invalid argument 2024-09-19T05:45:04.403 INFO:teuthology.orchestra.run.smithi123.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-19T05:45:04.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-19T05:45:04.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.948+0000 7f070e8a5640 1 Processor -- start 2024-09-19T05:45:04.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.948+0000 7f070e8a5640 1 -- start start 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f070e8a5640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0708154c20 0x7f07080cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f070e8a5640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f07080caf40 0x7f070815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f070e8a5640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07080ccbc0 con 0x7f0708155000 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f070e8a5640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f07080cc600 con 0x7f07080caf40 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f070e8a5640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f07080cc8e0 con 0x7f0708154c20 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f070ce42640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f07080caf40 0x7f070815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f07077fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f0708155000 0x7f0708073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44038/0 (socket says 172.21.15.2:44038) 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f07077fe640 1 -- 172.21.15.2:0/1403084140 learned_addr learned my addr 172.21.15.2:0/1403084140 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:04.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.949+0000 7f0707fff640 1 --2- 172.21.15.2:0/1403084140 >> v2:172.21.15.123:3300/0 conn(0x7f0708154c20 0x7f07080cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.950+0000 7f070ce42640 1 -- 172.21.15.2:0/1403084140 >> v2:172.21.15.123:3300/0 conn(0x7f0708154c20 msgr2=0x7f07080cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:04.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.950+0000 7f070ce42640 1 --2- 172.21.15.2:0/1403084140 >> v2:172.21.15.123:3300/0 conn(0x7f0708154c20 0x7f07080cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.950+0000 7f070ce42640 1 -- 172.21.15.2:0/1403084140 >> v1:172.21.15.2:6789/0 conn(0x7f0708155000 legacy=0x7f0708073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.950+0000 7f070ce42640 1 -- 172.21.15.2:0/1403084140 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07080cbc60 con 0x7f07080caf40 2024-09-19T05:45:04.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.950+0000 7f0706ffd640 1 -- 172.21.15.2:0/1403084140 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075281821 0 0) 0x7f07080ccbc0 con 0x7f0708155000 2024-09-19T05:45:04.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.950+0000 7f070ce42640 1 --2- 172.21.15.2:0/1403084140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07080caf40 0x7f070815aae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0708073ed0 tx=0x7f070002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb628aab9775edb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:04.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f0706ffd640 1 -- 172.21.15.2:0/1403084140 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700007740 con 0x7f07080caf40 2024-09-19T05:45:04.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f0706ffd640 1 -- 172.21.15.2:0/1403084140 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07000078e0 con 0x7f07080caf40 2024-09-19T05:45:04.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f0706ffd640 1 -- 172.21.15.2:0/1403084140 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700007cb0 con 0x7f07080caf40 2024-09-19T05:45:04.953 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 -- 172.21.15.2:0/1403084140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07080caf40 msgr2=0x7f070815aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 --2- 172.21.15.2:0/1403084140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07080caf40 0x7f070815aae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0708073ed0 tx=0x7f070002a680 comp rx=0 tx=0).stop 2024-09-19T05:45:04.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 -- 172.21.15.2:0/1403084140 shutdown_connections 2024-09-19T05:45:04.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 --2- 172.21.15.2:0/1403084140 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f07080caf40 0x7f070815aae0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 --2- 172.21.15.2:0/1403084140 >> v2:172.21.15.123:3300/0 conn(0x7f0708154c20 0x7f07080cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 -- 172.21.15.2:0/1403084140 >> 172.21.15.2:0/1403084140 conn(0x7f070806d0e0 msgr2=0x7f070806d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:04.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 -- 172.21.15.2:0/1403084140 shutdown_connections 2024-09-19T05:45:04.954 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.951+0000 7f070e8a5640 1 -- 172.21.15.2:0/1403084140 wait complete. 2024-09-19T05:45:04.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.953+0000 7f070e8a5640 1 Processor -- start 2024-09-19T05:45:04.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.953+0000 7f070e8a5640 1 -- start start 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f070e8a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07080caf40 0x7f0708155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f070e8a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0708154c20 0x7f07081f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f070e8a5640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0708155000 0x7f07081f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f070e8a5640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0708073a50 con 0x7f0708154c20 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f070e8a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07080cb430 con 0x7f07080caf40 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f070e8a5640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0708073bd0 con 0x7f0708155000 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0708154c20 0x7f07081f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0708154c20 0x7f07081f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46534/0 (socket says 172.21.15.2:46534) 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 -- 172.21.15.2:0/820776525 learned_addr learned my addr 172.21.15.2:0/820776525 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f0707fff640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07080caf40 0x7f0708155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f070ce42640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0708155000 0x7f07081f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 -- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0708155000 msgr2=0x7f07081f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0708155000 0x7f07081f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.957 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 -- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07080caf40 msgr2=0x7f0708155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07080caf40 0x7f0708155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 -- 172.21.15.2:0/820776525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07081e2b00 con 0x7f0708154c20 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f0707fff640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07080caf40 0x7f0708155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f07077fe640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0708154c20 0x7f07081f2040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c4c0 tx=0x7f06fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.954+0000 7f0704ff9640 1 -- 172.21.15.2:0/820776525 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0074e0 con 0x7f0708154c20 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.955+0000 7f0704ff9640 1 -- 172.21.15.2:0/820776525 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc0047e0 con 0x7f0708154c20 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.955+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07081e2de0 con 0x7f0708154c20 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.955+0000 7f0704ff9640 1 -- 172.21.15.2:0/820776525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc004bd0 con 0x7f0708154c20 2024-09-19T05:45:04.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.955+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070814ea00 con 0x7f0708154c20 2024-09-19T05:45:04.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.956+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07081f1e70 con 0x7f0708154c20 2024-09-19T05:45:04.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.956+0000 7f0704ff9640 1 -- 172.21.15.2:0/820776525 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f06fc011070 con 0x7f0708154c20 2024-09-19T05:45:04.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.957+0000 7f0704ff9640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06f406c2b0 0x7f06f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:04.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.957+0000 7f0704ff9640 1 -- 172.21.15.2:0/820776525 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5799+0+0 (secure 0 0 0) 0x7f06fc08ae50 con 0x7f0708154c20 2024-09-19T05:45:04.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.957+0000 7f0707fff640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06f406c2b0 0x7f06f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:04.959 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.957+0000 7f0707fff640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06f406c2b0 0x7f06f406e770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009f30 tx=0x7f06f8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:04.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:04.961+0000 7f0704ff9640 1 -- 172.21.15.2:0/820776525 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f06fc058fa0 con 0x7f0708154c20 2024-09-19T05:45:05.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.149+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f07080c5770 con 0x7f0708154c20 2024-09-19T05:45:05.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-19T05:45:05.491 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.490+0000 7fe946da2640 1 Processor -- start 2024-09-19T05:45:05.492 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe946da2640 1 -- start start 2024-09-19T05:45:05.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe946da2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe940154d60 0x7fe94015b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe946da2640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe94015de70 0x7fe94006f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe946da2640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe94015b870 con 0x7fe9400c8cc0 2024-09-19T05:45:05.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe946da2640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe94015e320 con 0x7fe94015de70 2024-09-19T05:45:05.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe946da2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe94015e600 con 0x7fe940154d60 2024-09-19T05:45:05.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe9465a1640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe9400c8cc0 0x7fe9400ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44044/0 (socket says 172.21.15.2:44044) 2024-09-19T05:45:05.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.491+0000 7fe9465a1640 1 -- 172.21.15.2:0/872889032 learned_addr learned my addr 172.21.15.2:0/872889032 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe945da0640 1 --2- 172.21.15.2:0/872889032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe940154d60 0x7fe94015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe94559f640 1 --2- 172.21.15.2:0/872889032 >> v2:172.21.15.123:3300/0 conn(0x7fe94015de70 0x7fe94006f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe944d9e640 1 -- 172.21.15.2:0/872889032 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2177311312 0 0) 0x7fe94015b870 con 0x7fe9400c8cc0 2024-09-19T05:45:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe944d9e640 1 -- 172.21.15.2:0/872889032 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe928003520 con 0x7fe9400c8cc0 2024-09-19T05:45:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe945da0640 1 -- 172.21.15.2:0/872889032 >> v2:172.21.15.123:3300/0 conn(0x7fe94015de70 msgr2=0x7fe94006f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe945da0640 1 --2- 172.21.15.2:0/872889032 >> v2:172.21.15.123:3300/0 conn(0x7fe94015de70 0x7fe94006f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe945da0640 1 -- 172.21.15.2:0/872889032 >> v1:172.21.15.2:6789/0 conn(0x7fe9400c8cc0 legacy=0x7fe9400ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe945da0640 1 -- 172.21.15.2:0/872889032 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94015e8e0 con 0x7fe940154d60 2024-09-19T05:45:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.492+0000 7fe94559f640 1 --2- 172.21.15.2:0/872889032 >> v2:172.21.15.123:3300/0 conn(0x7fe94015de70 0x7fe94006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.493+0000 7fe945da0640 1 --2- 172.21.15.2:0/872889032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe940154d60 0x7fe94015b330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe930009d90 tx=0x7fe93002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4349dcdd3f603a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.493+0000 7fe944d9e640 1 -- 172.21.15.2:0/872889032 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930007740 con 0x7fe940154d60 2024-09-19T05:45:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.493+0000 7fe944d9e640 1 -- 172.21.15.2:0/872889032 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9300078e0 con 0x7fe940154d60 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.493+0000 7fe944d9e640 1 -- 172.21.15.2:0/872889032 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930007c90 con 0x7fe940154d60 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 -- 172.21.15.2:0/872889032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe940154d60 msgr2=0x7fe94015b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 --2- 172.21.15.2:0/872889032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe940154d60 0x7fe94015b330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe930009d90 tx=0x7fe93002a6f0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 -- 172.21.15.2:0/872889032 shutdown_connections 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 --2- 172.21.15.2:0/872889032 >> v2:172.21.15.123:3300/0 conn(0x7fe94015de70 0x7fe94006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 --2- 172.21.15.2:0/872889032 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe940154d60 0x7fe94015b330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 -- 172.21.15.2:0/872889032 >> 172.21.15.2:0/872889032 conn(0x7fe940067950 msgr2=0x7fe940067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:05.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 -- 172.21.15.2:0/872889032 shutdown_connections 2024-09-19T05:45:05.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.494+0000 7fe946da2640 1 -- 172.21.15.2:0/872889032 wait complete. 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.495+0000 7fe946da2640 1 Processor -- start 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.495+0000 7fe946da2640 1 -- start start 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe946da2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400c8cc0 0x7fe9401f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe946da2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe940154d60 0x7fe9401f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe946da2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94015de70 0x7fe9401f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe946da2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9400caa10 con 0x7fe9400c8cc0 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe946da2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9400ca890 con 0x7fe940154d60 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe946da2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe94006f9d0 con 0x7fe94015de70 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400c8cc0 0x7fe9401f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400c8cc0 0x7fe9401f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46540/0 (socket says 172.21.15.2:46540) 2024-09-19T05:45:05.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 -- 172.21.15.2:0/3765956236 learned_addr learned my addr 172.21.15.2:0/3765956236 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:05.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94015de70 0x7fe9401f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe94559f640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe940154d60 0x7fe9401f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94015de70 msgr2=0x7fe9401f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94015de70 0x7fe9401f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe940154d60 msgr2=0x7fe9401f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe940154d60 0x7fe9401f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9400cea50 con 0x7fe9400c8cc0 2024-09-19T05:45:05.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe94559f640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe940154d60 0x7fe9401f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:05.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe945da0640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400c8cc0 0x7fe9401f1200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe930009eb0 tx=0x7fe930004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930041030 con 0x7fe9400c8cc0 2024-09-19T05:45:05.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe930031a40 con 0x7fe9400c8cc0 2024-09-19T05:45:05.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930009450 con 0x7fe9400c8cc0 2024-09-19T05:45:05.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.496+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9400ced30 con 0x7fe9400c8cc0 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.497+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9401f0a00 con 0x7fe9400c8cc0 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.497+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe94006eac0 con 0x7fe9400c8cc0 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.498+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe930044030 con 0x7fe9400c8cc0 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.498+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe92c06c2b0 0x7fe92c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5799+0+0 (secure 0 0 0) 0x7fe9300ac710 con 0x7fe9400c8cc0 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1281 (secure 0 0 0) 0x7fe94006eac0 con 0x7fe9400c8cc0 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe94559f640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe92c06c2b0 0x7fe92c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940153110 0x7fe940151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401f0bd0 con 0x7fe940153110 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940153110 0x7fe940151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7fe9401f1050 con 0x7fe940153110 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe94559f640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe92c06c2b0 0x7fe92c06e770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9400c9dc0 tx=0x7fe934008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.499+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940153110 0x7fe940151ac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe938000f60 tx=0x7fe938001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.500+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938017020 con 0x7fe940153110 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.500+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe9380136c0 con 0x7fe940153110 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.500+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940153110 msgr2=0x7fe940151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.500+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940153110 0x7fe940151ac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe938000f60 tx=0x7fe938001990 comp rx=0 tx=0).stop 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.500+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940153110 msgr2=0x7fe940151ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:45:05.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.500+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe940153110 0x7fe940151ac0 secure :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe938000f60 tx=0x7fe938001990 comp rx=0 tx=0).stop 2024-09-19T05:45:05.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.506+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9401534f0 0x7fe940152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.506+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe9401f4ad0 con 0x7fe9401534f0 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.506+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9401534f0 0x7fe940152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.506+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7fe940152540 con 0x7fe9401534f0 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.506+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9401534f0 0x7fe940152000 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9380078c0 tx=0x7fe938001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.507+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938017030 con 0x7fe9401534f0 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fe9380062d0 con 0x7fe9401534f0 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9401534f0 msgr2=0x7fe940152000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9401534f0 0x7fe940152000 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe9380078c0 tx=0x7fe938001150 comp rx=0 tx=0).stop 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9401534f0 msgr2=0x7fe940152000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:45:05.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9401534f0 0x7fe940152000 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.512 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a: {} 2024-09-19T05:45:05.513 INFO:teuthology.orchestra.run.smithi002.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T05:45:05.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9400cd110 0x7fe9400cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9400ca2a0 con 0x7fe9400cd110 2024-09-19T05:45:05.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.508+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9400cd110 0x7fe9400cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.509+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7fe9400c91d0 con 0x7fe9400cd110 2024-09-19T05:45:05.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.509+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9400cd110 0x7fe9400cd520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe938008fd0 tx=0x7fe938007cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.509+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93802a070 con 0x7fe9400cd110 2024-09-19T05:45:05.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.510+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.1 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe938026700 con 0x7fe9400cd110 2024-09-19T05:45:05.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.510+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9400cd110 msgr2=0x7fe9400cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.510+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9400cd110 0x7fe9400cd520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe938008fd0 tx=0x7fe938007cb0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.510+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9400cd110 msgr2=0x7fe9400cd520 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:45:05.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.510+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 reap_dead start 2024-09-19T05:45:05.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.510+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9400cd110 0x7fe9400cd520 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.516+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9401f1a80 0x7fe9401f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.516+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe940067f80 con 0x7fe9401f1a80 2024-09-19T05:45:05.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.516+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9401f1a80 0x7fe9401f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.516+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7fe9401f0de0 con 0x7fe9401f1a80 2024-09-19T05:45:05.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.516+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9401f1a80 0x7fe9401f4590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9380019d0 tx=0x7fe938026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.517+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93803a770 con 0x7fe9401f1a80 2024-09-19T05:45:05.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.518+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.1 v2:172.21.15.123:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fe938031470 con 0x7fe9401f1a80 2024-09-19T05:45:05.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.518+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9401f1a80 msgr2=0x7fe9401f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.518+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9401f1a80 0x7fe9401f4590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9380019d0 tx=0x7fe938026000 comp rx=0 tx=0).stop 2024-09-19T05:45:05.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.518+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9401f1a80 msgr2=0x7fe9401f4590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:45:05.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.518+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9401f1a80 0x7fe9401f4590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.520 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b: {} 2024-09-19T05:45:05.521 INFO:teuthology.orchestra.run.smithi002.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T05:45:05.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.518+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400cda60 0x7fe9400c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.518+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9400008d0 con 0x7fe9400cda60 2024-09-19T05:45:05.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.519+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400cda60 0x7fe9400c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.519+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7fe94006f410 con 0x7fe9400cda60 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.519+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400cda60 0x7fe9400c39a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe938006a60 tx=0x7fe938007cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.519+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938050070 con 0x7fe9400cda60 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.520+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.2 v2:172.21.15.123:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe93803a980 con 0x7fe9400cda60 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.520+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400cda60 msgr2=0x7fe9400c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.520+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400cda60 0x7fe9400c39a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe938006a60 tx=0x7fe938007cb0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.520+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400cda60 msgr2=0x7fe9400c39a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:45:05.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.520+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400cda60 0x7fe9400c39a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.526+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400c3ee0 0x7fe9400c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.526+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe940075510 con 0x7fe9400c3ee0 2024-09-19T05:45:05.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.526+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400c3ee0 0x7fe9400c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.526+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7fe9400cde40 con 0x7fe9400c3ee0 2024-09-19T05:45:05.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.526+0000 7fe9465a1640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400c3ee0 0x7fe9400c42c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe938013690 tx=0x7fe938044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.527+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938031a30 con 0x7fe9400c3ee0 2024-09-19T05:45:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.528+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 <== mon.2 v2:172.21.15.123:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7fe938031830 con 0x7fe9400c3ee0 2024-09-19T05:45:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.528+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400c3ee0 msgr2=0x7fe9400c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.528+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400c3ee0 0x7fe9400c42c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe938013690 tx=0x7fe938044000 comp rx=0 tx=0).stop 2024-09-19T05:45:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.528+0000 7fe93e7fc640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400c3ee0 msgr2=0x7fe9400c42c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-19T05:45:05.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.528+0000 7fe93e7fc640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9400c3ee0 0x7fe9400c42c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.531 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c: {} 2024-09-19T05:45:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-19T05:45:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe92c06c2b0 msgr2=0x7fe92c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe92c06c2b0 0x7fe92c06e770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9400c9dc0 tx=0x7fe934008040 comp rx=0 tx=0).stop 2024-09-19T05:45:05.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400c8cc0 msgr2=0x7fe9401f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 --2- 172.21.15.2:0/3765956236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9400c8cc0 0x7fe9401f1200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe930009eb0 tx=0x7fe930004a40 comp rx=0 tx=0).stop 2024-09-19T05:45:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe9465a1640 1 -- 172.21.15.2:0/3765956236 reap_dead start 2024-09-19T05:45:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 shutdown_connections 2024-09-19T05:45:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 >> 172.21.15.2:0/3765956236 conn(0x7fe940067950 msgr2=0x7fe9401596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 shutdown_connections 2024-09-19T05:45:05.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.529+0000 7fe946da2640 1 -- 172.21.15.2:0/3765956236 wait complete. 2024-09-19T05:45:05.541 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-19T05:45:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.580+0000 7f0704ff9640 1 -- 172.21.15.2:0/820776525 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v86) ==== 91+0+0 (secure 0 0 0) 0x7f06fc05d2a0 con 0x7f0708154c20 2024-09-19T05:45:05.582 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-19T05:45:05.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06f406c2b0 msgr2=0x7f06f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06f406c2b0 0x7f06f406e770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009f30 tx=0x7f06f8009190 comp rx=0 tx=0).stop 2024-09-19T05:45:05.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0708154c20 msgr2=0x7f07081f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0708154c20 0x7f07081f2040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c4c0 tx=0x7f06fc00c8c0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 shutdown_connections 2024-09-19T05:45:05.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f06f406c2b0 0x7f06f406e770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0708155000 0x7f07081f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0708154c20 0x7f07081f2040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 --2- 172.21.15.2:0/820776525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07080caf40 0x7f0708155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 >> 172.21.15.2:0/820776525 conn(0x7f070806d0e0 msgr2=0x7f07080d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:05.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 shutdown_connections 2024-09-19T05:45:05.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.584+0000 7f070e8a5640 1 -- 172.21.15.2:0/820776525 wait complete. 2024-09-19T05:45:05.597 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-19T05:45:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.646+0000 7fd2f37d6640 1 Processor -- start 2024-09-19T05:45:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.646+0000 7fd2f37d6640 1 -- start start 2024-09-19T05:45:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.647+0000 7fd2f37d6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec151670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.647+0000 7fd2f37d6640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2ec0c9cd0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.647+0000 7fd2f37d6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2ec1583e0 con 0x7fd2ec0c4bd0 2024-09-19T05:45:05.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.647+0000 7fd2f37d6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ec15ae30 con 0x7fd2ec0c9890 2024-09-19T05:45:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.647+0000 7fd2f0d71640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd2ec0c4fb0 0x7fd2ec1578c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44056/0 (socket says 172.21.15.2:44056) 2024-09-19T05:45:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2f0d71640 1 -- 172.21.15.2:0/382634822 learned_addr learned my addr 172.21.15.2:0/382634822 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2f1d73640 1 --2- 172.21.15.2:0/382634822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046842480 0 0) 0x7fd2ec0c9cd0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2d8005730 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290724273 0 0) 0x7fd2ec1583e0 con 0x7fd2ec0c4bd0 2024-09-19T05:45:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2ec0c9cd0 con 0x7fd2ec0c4bd0 2024-09-19T05:45:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2f1d73640 1 -- 172.21.15.2:0/382634822 >> v1:172.21.15.123:6789/0 conn(0x7fd2ec0c4bd0 legacy=0x7fd2ec073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2f1d73640 1 -- 172.21.15.2:0/382634822 >> v1:172.21.15.2:6789/0 conn(0x7fd2ec0c4fb0 legacy=0x7fd2ec1578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1188501926 0 0) 0x7fd2d8005730 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.648+0000 7fd2f1d73640 1 -- 172.21.15.2:0/382634822 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec1586c0 con 0x7fd2ec0c9890 2024-09-19T05:45:05.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.649+0000 7fd2f1d73640 1 --2- 172.21.15.2:0/382634822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec151670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009dc0 tx=0x7fd2e402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd1337987d03eab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.649+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8001690 con 0x7fd2ec0c9890 2024-09-19T05:45:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.649+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e4007540 con 0x7fd2ec0c9890 2024-09-19T05:45:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.649+0000 7fd2e3fff640 1 -- 172.21.15.2:0/382634822 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e4007910 con 0x7fd2ec0c9890 2024-09-19T05:45:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.650+0000 7fd2f37d6640 1 -- 172.21.15.2:0/382634822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 msgr2=0x7fd2ec151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.650+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/382634822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec151670 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009dc0 tx=0x7fd2e402a660 comp rx=0 tx=0).stop 2024-09-19T05:45:05.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.650+0000 7fd2f37d6640 1 -- 172.21.15.2:0/382634822 shutdown_connections 2024-09-19T05:45:05.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.650+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/382634822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec151670 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.650+0000 7fd2f37d6640 1 -- 172.21.15.2:0/382634822 >> 172.21.15.2:0/382634822 conn(0x7fd2ec06f3d0 msgr2=0x7fd2ec0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:05.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.650+0000 7fd2f37d6640 1 -- 172.21.15.2:0/382634822 shutdown_connections 2024-09-19T05:45:05.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.650+0000 7fd2f37d6640 1 -- 172.21.15.2:0/382634822 wait complete. 2024-09-19T05:45:05.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.651+0000 7fd2f37d6640 1 Processor -- start 2024-09-19T05:45:05.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.651+0000 7fd2f37d6640 1 -- start start 2024-09-19T05:45:05.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f37d6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec0c4bd0 0x7fd2ec1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f37d6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2ec0c4fb0 0x7fd2ec1f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f37d6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec14c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f37d6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec157fd0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f37d6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec15b360 con 0x7fd2ec0c4bd0 2024-09-19T05:45:05.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f37d6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ec158150 con 0x7fd2ec0c9890 2024-09-19T05:45:05.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2ec0c4fb0 0x7fd2ec1f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2ec0c4fb0 0x7fd2ec1f06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46570/0 (socket says 172.21.15.2:46570) 2024-09-19T05:45:05.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 -- 172.21.15.2:0/391534755 learned_addr learned my addr 172.21.15.2:0/391534755 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:05.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f1d73640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec14c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f1572640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec0c4bd0 0x7fd2ec1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 -- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 msgr2=0x7fd2ec14c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec14c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 -- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec0c4bd0 msgr2=0x7fd2ec1f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec0c4bd0 0x7fd2ec1f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 -- 172.21.15.2:0/391534755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec152230 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2f0d71640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2ec0c4fb0 0x7fd2ec1f06e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001200 tx=0x7fd2e800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.653+0000 7fd2e1ffb640 1 -- 172.21.15.2:0/391534755 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8018070 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.654+0000 7fd2e1ffb640 1 -- 172.21.15.2:0/391534755 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e80072f0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.654+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec1551a0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.654+0000 7fd2e1ffb640 1 -- 172.21.15.2:0/391534755 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80047e0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.654+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec1527d0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.655+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ec15bb60 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.655+0000 7fd2e1ffb640 1 -- 172.21.15.2:0/391534755 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd2e801b070 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.656+0000 7fd2e1ffb640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd2d006c2b0 0x7fd2d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.656+0000 7fd2e1ffb640 1 -- 172.21.15.2:0/391534755 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5799+0+0 (secure 0 0 0) 0x7fd2e808a870 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.656+0000 7fd2f1572640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd2d006c2b0 0x7fd2d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.656+0000 7fd2f1572640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd2d006c2b0 0x7fd2d006e770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0011e0 tx=0x7fd2dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.666 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.660+0000 7fd2e1ffb640 1 -- 172.21.15.2:0/391534755 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd2e8059010 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.742+0000 7fc60f63c640 1 Processor -- start 2024-09-19T05:45:05.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.742+0000 7fc60f63c640 1 -- start start 2024-09-19T05:45:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.743+0000 7fc60f63c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc6080680f0 0x7fc6080b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.743+0000 7fc60f63c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc60815cb90 0x7fc6080c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.743+0000 7fc60f63c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6080b4930 con 0x7fc608067d10 2024-09-19T05:45:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.743+0000 7fc60f63c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fc6080b4650 con 0x7fc6080680f0 2024-09-19T05:45:05.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.743+0000 7fc60f63c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc6080c90c0 con 0x7fc60815cb90 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.743+0000 7fc60cbd7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc6080680f0 0x7fc6080b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.743+0000 7fc60d3d8640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fc608067d10 0x7fc60815c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44072/0 (socket says 172.21.15.2:44072) 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.744+0000 7fc60d3d8640 1 -- 172.21.15.2:0/2939248236 learned_addr learned my addr 172.21.15.2:0/2939248236 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.744+0000 7fc60dbd9640 1 --2- 172.21.15.2:0/2939248236 >> v2:172.21.15.123:3300/0 conn(0x7fc60815cb90 0x7fc6080c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.744+0000 7fc60cbd7640 1 -- 172.21.15.2:0/2939248236 >> v2:172.21.15.123:3300/0 conn(0x7fc60815cb90 msgr2=0x7fc6080c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.744+0000 7fc60cbd7640 1 --2- 172.21.15.2:0/2939248236 >> v2:172.21.15.123:3300/0 conn(0x7fc60815cb90 0x7fc6080c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.744+0000 7fc60cbd7640 1 -- 172.21.15.2:0/2939248236 >> v1:172.21.15.2:6789/0 conn(0x7fc608067d10 legacy=0x7fc60815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.744+0000 7fc5fbfff640 1 -- 172.21.15.2:0/2939248236 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7470130 0 0) 0x7fc6080b4930 con 0x7fc608067d10 2024-09-19T05:45:05.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.744+0000 7fc60cbd7640 1 -- 172.21.15.2:0/2939248236 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6080c93a0 con 0x7fc6080680f0 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.745+0000 7fc60cbd7640 1 --2- 172.21.15.2:0/2939248236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6080680f0 0x7fc6080b3a90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc608068610 tx=0x7fc60402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2496adb92e748b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.745+0000 7fc5fbfff640 1 -- 172.21.15.2:0/2939248236 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604037070 con 0x7fc6080680f0 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.745+0000 7fc5fbfff640 1 -- 172.21.15.2:0/2939248236 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc604032440 con 0x7fc6080680f0 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.745+0000 7fc5fbfff640 1 -- 172.21.15.2:0/2939248236 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604032810 con 0x7fc6080680f0 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.745+0000 7fc60f63c640 1 -- 172.21.15.2:0/2939248236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6080680f0 msgr2=0x7fc6080b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.745+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2939248236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6080680f0 0x7fc6080b3a90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc608068610 tx=0x7fc60402a1a0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.746+0000 7fc60f63c640 1 -- 172.21.15.2:0/2939248236 shutdown_connections 2024-09-19T05:45:05.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.746+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2939248236 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6080680f0 0x7fc6080b3a90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.746+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2939248236 >> v2:172.21.15.123:3300/0 conn(0x7fc60815cb90 0x7fc6080c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.746+0000 7fc60f63c640 1 -- 172.21.15.2:0/2939248236 >> 172.21.15.2:0/2939248236 conn(0x7fc608070fc0 msgr2=0x7fc60806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:05.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.746+0000 7fc60f63c640 1 -- 172.21.15.2:0/2939248236 shutdown_connections 2024-09-19T05:45:05.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.746+0000 7fc60f63c640 1 -- 172.21.15.2:0/2939248236 wait complete. 2024-09-19T05:45:05.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.747+0000 7fc60f63c640 1 Processor -- start 2024-09-19T05:45:05.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.747+0000 7fc60f63c640 1 -- start start 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60f63c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc608067d10 0x7fc60815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60f63c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6080680f0 0x7fc6080c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60f63c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc60815cb90 0x7fc6080c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60f63c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080b41e0 con 0x7fc608067d10 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60f63c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080b4060 con 0x7fc60815cb90 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60f63c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc6080b4360 con 0x7fc6080680f0 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc608067d10 0x7fc60815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc608067d10 0x7fc60815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46598/0 (socket says 172.21.15.2:46598) 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 -- 172.21.15.2:0/2281281588 learned_addr learned my addr 172.21.15.2:0/2281281588 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60cbd7640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6080680f0 0x7fc6080c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60dbd9640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc60815cb90 0x7fc6080c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 -- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6080680f0 msgr2=0x7fc6080c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6080680f0 0x7fc6080c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 -- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc60815cb90 msgr2=0x7fc6080c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc60815cb90 0x7fc6080c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 -- 172.21.15.2:0/2281281588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6080c8020 con 0x7fc608067d10 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60d3d8640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc608067d10 0x7fc60815adc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00c460 tx=0x7fc5fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc5f9ffb640 1 -- 172.21.15.2:0/2281281588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0074e0 con 0x7fc608067d10 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc5f9ffb640 1 -- 172.21.15.2:0/2281281588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5fc0047e0 con 0x7fc608067d10 2024-09-19T05:45:05.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6081eed40 con 0x7fc608067d10 2024-09-19T05:45:05.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.748+0000 7fc5f9ffb640 1 -- 172.21.15.2:0/2281281588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc004bd0 con 0x7fc608067d10 2024-09-19T05:45:05.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.749+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6080c8570 con 0x7fc608067d10 2024-09-19T05:45:05.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.749+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6080c87b0 con 0x7fc608067d10 2024-09-19T05:45:05.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.750+0000 7fc5f9ffb640 1 -- 172.21.15.2:0/2281281588 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc5fc011070 con 0x7fc608067d10 2024-09-19T05:45:05.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.751+0000 7fc5f9ffb640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc5ec06c2b0 0x7fc5ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:05.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.751+0000 7fc5f9ffb640 1 -- 172.21.15.2:0/2281281588 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5799+0+0 (secure 0 0 0) 0x7fc5fc08aee0 con 0x7fc608067d10 2024-09-19T05:45:05.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.751+0000 7fc60cbd7640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc5ec06c2b0 0x7fc5ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:05.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.752+0000 7fc60cbd7640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc5ec06c2b0 0x7fc5ec06e770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc604034000 tx=0x7fc60402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:05.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.755+0000 7fc5f9ffb640 1 -- 172.21.15.2:0/2281281588 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc6080c87b0 con 0x7fc608067d10 2024-09-19T05:45:05.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.849+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7fd2ec0736d0 con 0x7fd2ec0c4fb0 2024-09-19T05:45:05.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:05.941+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fc6080b3a90 con 0x7fc608067d10 2024-09-19T05:45:06.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.588+0000 7fd2e1ffb640 1 -- 172.21.15.2:0/391534755 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v87) ==== 113+0+0 (secure 0 0 0) 0x7fd2e8058910 con 0x7fd2ec0c4fb0 2024-09-19T05:45:06.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.588+0000 7fc5f9ffb640 1 -- 172.21.15.2:0/2281281588 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v87) ==== 101+0+0 (secure 0 0 0) 0x7fc5fc059030 con 0x7fc608067d10 2024-09-19T05:45:06.591 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-19T05:45:06.591 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-19T05:45:06.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc5ec06c2b0 msgr2=0x7fc5ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc5ec06c2b0 0x7fc5ec06e770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc604034000 tx=0x7fc60402e000 comp rx=0 tx=0).stop 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc608067d10 msgr2=0x7fc60815adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc608067d10 0x7fc60815adc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00c460 tx=0x7fc5fc00c920 comp rx=0 tx=0).stop 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 shutdown_connections 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc5ec06c2b0 0x7fc5ec06e770 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc60815cb90 0x7fc6080c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc6080680f0 0x7fc6080c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 --2- 172.21.15.2:0/2281281588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc608067d10 0x7fc60815adc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 >> 172.21.15.2:0/2281281588 conn(0x7fc608070fc0 msgr2=0x7fc6080721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 shutdown_connections 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd2d006c2b0 msgr2=0x7fd2d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd2d006c2b0 0x7fd2d006e770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0011e0 tx=0x7fd2dc009040 comp rx=0 tx=0).stop 2024-09-19T05:45:06.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2ec0c4fb0 msgr2=0x7fd2ec1f06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fc60f63c640 1 -- 172.21.15.2:0/2281281588 wait complete. 2024-09-19T05:45:06.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2ec0c4fb0 0x7fd2ec1f06e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001200 tx=0x7fd2e800cbe0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 shutdown_connections 2024-09-19T05:45:06.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd2d006c2b0 0x7fd2d006e770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd2ec0c9890 0x7fd2ec14c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd2ec0c4fb0 0x7fd2ec1f06e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 --2- 172.21.15.2:0/391534755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2ec0c4bd0 0x7fd2ec1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.592+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 >> 172.21.15.2:0/391534755 conn(0x7fd2ec06f3d0 msgr2=0x7fd2ec150520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.593+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 shutdown_connections 2024-09-19T05:45:06.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.593+0000 7fd2f37d6640 1 -- 172.21.15.2:0/391534755 wait complete. 2024-09-19T05:45:06.610 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-19T05:45:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.761+0000 7f5ad2fe8640 1 Processor -- start 2024-09-19T05:45:06.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.761+0000 7f5ad2fe8640 1 -- start start 2024-09-19T05:45:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad2fe8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5acc15ede0 0x7f5acc0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad2fe8640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5acc15ab40 con 0x7f5acc0cba20 2024-09-19T05:45:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad2fe8640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5acc15ade0 con 0x7f5acc0cb640 2024-09-19T05:45:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad2fe8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f5acc0c5ef0 con 0x7f5acc15ede0 2024-09-19T05:45:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad1585640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5acc15ede0 0x7f5acc0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5acbfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5acc0cba20 0x7f5acc1549e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44082/0 (socket says 172.21.15.2:44082) 2024-09-19T05:45:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5acbfff640 1 -- 172.21.15.2:0/1981046481 learned_addr learned my addr 172.21.15.2:0/1981046481 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad1585640 1 -- 172.21.15.2:0/1981046481 >> v1:172.21.15.123:6789/0 conn(0x7f5acc0cb640 legacy=0x7f5acc0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad1585640 1 -- 172.21.15.2:0/1981046481 >> v1:172.21.15.2:6789/0 conn(0x7f5acc0cba20 legacy=0x7f5acc1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.762+0000 7f5ad1585640 1 -- 172.21.15.2:0/1981046481 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc15b0c0 con 0x7f5acc15ede0 2024-09-19T05:45:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.763+0000 7f5ad1585640 1 --2- 172.21.15.2:0/1981046481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc0c6380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009d90 tx=0x7f5ac402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810ab97dca0726af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.763+0000 7f5acb7fe640 1 -- 172.21.15.2:0/1981046481 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4007740 con 0x7f5acc15ede0 2024-09-19T05:45:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.763+0000 7f5acb7fe640 1 -- 172.21.15.2:0/1981046481 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac40078e0 con 0x7f5acc15ede0 2024-09-19T05:45:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.763+0000 7f5acb7fe640 1 -- 172.21.15.2:0/1981046481 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4007cb0 con 0x7f5acc15ede0 2024-09-19T05:45:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.764+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/1981046481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 msgr2=0x7f5acc0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.764+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/1981046481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc0c6380 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009d90 tx=0x7f5ac402a730 comp rx=0 tx=0).stop 2024-09-19T05:45:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.764+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/1981046481 shutdown_connections 2024-09-19T05:45:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.764+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/1981046481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc0c6380 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.764+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/1981046481 >> 172.21.15.2:0/1981046481 conn(0x7f5acc067950 msgr2=0x7f5acc067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.764+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/1981046481 shutdown_connections 2024-09-19T05:45:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.764+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/1981046481 wait complete. 2024-09-19T05:45:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.765+0000 7f5ad2fe8640 1 Processor -- start 2024-09-19T05:45:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.765+0000 7f5ad2fe8640 1 -- start start 2024-09-19T05:45:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad2fe8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5acc0cb640 0x7f5acc153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad2fe8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5acc0cba20 0x7f5acc156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad2fe8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad2fe8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc15b960 con 0x7f5acc15ede0 2024-09-19T05:45:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad2fe8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc15b680 con 0x7f5acc0cba20 2024-09-19T05:45:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad2fe8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc0c5bd0 con 0x7f5acc0cb640 2024-09-19T05:45:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46616/0 (socket says 172.21.15.2:46616) 2024-09-19T05:45:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 -- 172.21.15.2:0/55211457 learned_addr learned my addr 172.21.15.2:0/55211457 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 -- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5acc0cb640 msgr2=0x7f5acc153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5acbfff640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5acc0cba20 0x7f5acc156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad0d84640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5acc0cb640 0x7f5acc153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5acc0cb640 0x7f5acc153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 -- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5acc0cba20 msgr2=0x7f5acc156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5acc0cba20 0x7f5acc156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad1585640 1 -- 172.21.15.2:0/55211457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc0b6730 con 0x7f5acc15ede0 2024-09-19T05:45:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5ad0d84640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5acc0cb640 0x7f5acc153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.766+0000 7f5acbfff640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5acc0cba20 0x7f5acc156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.767+0000 7f5ad1585640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc157160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009d90 tx=0x7f5ac4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.767+0000 7f5ac97fa640 1 -- 172.21.15.2:0/55211457 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4040070 con 0x7f5acc15ede0 2024-09-19T05:45:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.767+0000 7f5ac97fa640 1 -- 172.21.15.2:0/55211457 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac4031c90 con 0x7f5acc15ede0 2024-09-19T05:45:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.767+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc0b5da0 con 0x7f5acc15ede0 2024-09-19T05:45:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.767+0000 7f5ac97fa640 1 -- 172.21.15.2:0/55211457 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac403c640 con 0x7f5acc15ede0 2024-09-19T05:45:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.767+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc0b5240 con 0x7f5acc15ede0 2024-09-19T05:45:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.768+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5acc0c54c0 con 0x7f5acc15ede0 2024-09-19T05:45:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.768+0000 7f5ac97fa640 1 -- 172.21.15.2:0/55211457 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5ac4039040 con 0x7f5acc15ede0 2024-09-19T05:45:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.769+0000 7f5ac97fa640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5ab006c2e0 0x7f5ab006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.769+0000 7f5ac97fa640 1 -- 172.21.15.2:0/55211457 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5799+0+0 (secure 0 0 0) 0x7f5ac40ac950 con 0x7f5acc15ede0 2024-09-19T05:45:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.769+0000 7f5ad0d84640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5ab006c2e0 0x7f5ab006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.770+0000 7f5ad0d84640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5ab006c2e0 0x7f5ab006e7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0003eb0 tx=0x7f5ac0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.773+0000 7f5ac97fa640 1 -- 172.21.15.2:0/55211457 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5ac407b1b0 con 0x7f5acc15ede0 2024-09-19T05:45:06.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:06.964+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) -- 0x7f5acc0cbf10 con 0x7f5acc15ede0 2024-09-19T05:45:07.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.590+0000 7f5ac97fa640 1 -- 172.21.15.2:0/55211457 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v88) ==== 133+0+0 (secure 0 0 0) 0x7f5ac407aab0 con 0x7f5acc15ede0 2024-09-19T05:45:07.592 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-19T05:45:07.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5ab006c2e0 msgr2=0x7f5ab006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5ab006c2e0 0x7f5ab006e7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0003eb0 tx=0x7f5ac0009040 comp rx=0 tx=0).stop 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 msgr2=0x7f5acc157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc157160 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009d90 tx=0x7f5ac4004a40 comp rx=0 tx=0).stop 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 shutdown_connections 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5ab006c2e0 0x7f5ab006e7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5acc0cba20 0x7f5acc156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5acc15ede0 0x7f5acc157160 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 --2- 172.21.15.2:0/55211457 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5acc0cb640 0x7f5acc153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 >> 172.21.15.2:0/55211457 conn(0x7f5acc067950 msgr2=0x7f5acc159220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 shutdown_connections 2024-09-19T05:45:07.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.593+0000 7f5ad2fe8640 1 -- 172.21.15.2:0/55211457 wait complete. 2024-09-19T05:45:07.610 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-19T05:45:07.720 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.718+0000 7f3cf68e6640 1 Processor -- start 2024-09-19T05:45:07.721 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.718+0000 7f3cf68e6640 1 -- start start 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cf68e6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3cf0154c20 0x7f3cf00cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cf68e6640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3cf0155000 0x7f3cf0073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cf68e6640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf00ccbc0 con 0x7f3cf00caf40 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cf68e6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf00cc600 con 0x7f3cf0154c20 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cf68e6640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf00cc8e0 con 0x7f3cf0155000 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3ceffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3cf0154c20 0x7f3cf00cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cf4e83640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3cf00caf40 0x7f3cf015aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44094/0 (socket says 172.21.15.2:44094) 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cf4e83640 1 -- 172.21.15.2:0/41134508 learned_addr learned my addr 172.21.15.2:0/41134508 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:07.722 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.719+0000 7f3cef7fe640 1 --2- 172.21.15.2:0/41134508 >> v2:172.21.15.123:3300/0 conn(0x7f3cf0155000 0x7f3cf0073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:07.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceffff640 1 -- 172.21.15.2:0/41134508 >> v2:172.21.15.123:3300/0 conn(0x7f3cf0155000 msgr2=0x7f3cf0073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:07.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceffff640 1 --2- 172.21.15.2:0/41134508 >> v2:172.21.15.123:3300/0 conn(0x7f3cf0155000 0x7f3cf0073340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceffff640 1 -- 172.21.15.2:0/41134508 >> v1:172.21.15.2:6789/0 conn(0x7f3cf00caf40 legacy=0x7f3cf015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:07.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceeffd640 1 -- 172.21.15.2:0/41134508 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739555959 0 0) 0x7f3cf00ccbc0 con 0x7f3cf00caf40 2024-09-19T05:45:07.723 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceffff640 1 -- 172.21.15.2:0/41134508 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf00cbc60 con 0x7f3cf0154c20 2024-09-19T05:45:07.727 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceffff640 1 --2- 172.21.15.2:0/41134508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf0154c20 0x7f3cf00cd9e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00b2f60 tx=0x7f3ce002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f56196bfde7fbca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceeffd640 1 -- 172.21.15.2:0/41134508 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0037070 con 0x7f3cf0154c20 2024-09-19T05:45:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceeffd640 1 -- 172.21.15.2:0/41134508 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce0032440 con 0x7f3cf0154c20 2024-09-19T05:45:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.720+0000 7f3ceeffd640 1 -- 172.21.15.2:0/41134508 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0032810 con 0x7f3cf0154c20 2024-09-19T05:45:07.728 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 -- 172.21.15.2:0/41134508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf0154c20 msgr2=0x7f3cf00cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/41134508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf0154c20 0x7f3cf00cd9e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00b2f60 tx=0x7f3ce002a1a0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 -- 172.21.15.2:0/41134508 shutdown_connections 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/41134508 >> v2:172.21.15.123:3300/0 conn(0x7f3cf0155000 0x7f3cf0073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/41134508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf0154c20 0x7f3cf00cd9e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 -- 172.21.15.2:0/41134508 >> 172.21.15.2:0/41134508 conn(0x7f3cf006d0e0 msgr2=0x7f3cf006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 -- 172.21.15.2:0/41134508 shutdown_connections 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.721+0000 7f3cf68e6640 1 -- 172.21.15.2:0/41134508 wait complete. 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.722+0000 7f3cf68e6640 1 Processor -- start 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.722+0000 7f3cf68e6640 1 -- start start 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf68e6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf00caf40 0x7f3cf01507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf68e6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3cf0154c20 0x7f3cf01e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf68e6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cf0155000 0x7f3cf01e3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf68e6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf015b1f0 con 0x7f3cf00caf40 2024-09-19T05:45:07.729 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf68e6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf00cb430 con 0x7f3cf0155000 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf68e6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf015b370 con 0x7f3cf0154c20 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf00caf40 0x7f3cf01507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf00caf40 0x7f3cf01507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46628/0 (socket says 172.21.15.2:46628) 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 -- 172.21.15.2:0/2990687867 learned_addr learned my addr 172.21.15.2:0/2990687867 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf4e83640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cf0155000 0x7f3cf01e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 -- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3cf0154c20 msgr2=0x7f3cf01e32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3cf0154c20 0x7f3cf01e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 -- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cf0155000 msgr2=0x7f3cf01e3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cf0155000 0x7f3cf01e3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3ceffff640 1 -- 172.21.15.2:0/2990687867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf01f09d0 con 0x7f3cf00caf40 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.723+0000 7f3cf4e83640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cf0155000 0x7f3cf01e3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.724+0000 7f3ceffff640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf00caf40 0x7f3cf01507e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0035040 tx=0x7f3ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.724+0000 7f3cecff9640 1 -- 172.21.15.2:0/2990687867 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0037050 con 0x7f3cf00caf40 2024-09-19T05:45:07.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.724+0000 7f3cecff9640 1 -- 172.21.15.2:0/2990687867 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce0038040 con 0x7f3cf00caf40 2024-09-19T05:45:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.724+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf01f0cb0 con 0x7f3cf00caf40 2024-09-19T05:45:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.724+0000 7f3cecff9640 1 -- 172.21.15.2:0/2990687867 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0004450 con 0x7f3cf00caf40 2024-09-19T05:45:07.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.724+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf01f1270 con 0x7f3cf00caf40 2024-09-19T05:45:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.725+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf0154370 con 0x7f3cf00caf40 2024-09-19T05:45:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.725+0000 7f3cecff9640 1 -- 172.21.15.2:0/2990687867 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3ce0030900 con 0x7f3cf00caf40 2024-09-19T05:45:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.726+0000 7f3cecff9640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3cdc06c2b0 0x7f3cdc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.726+0000 7f3cecff9640 1 -- 172.21.15.2:0/2990687867 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5799+0+0 (secure 0 0 0) 0x7f3ce00ac0b0 con 0x7f3cf00caf40 2024-09-19T05:45:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.726+0000 7f3cef7fe640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3cdc06c2b0 0x7f3cdc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.727+0000 7f3cef7fe640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3cdc06c2b0 0x7f3cdc06e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4006e20 tx=0x7f3ce4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.730+0000 7f3cecff9640 1 -- 172.21.15.2:0/2990687867 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3cf0154370 con 0x7f3cf00caf40 2024-09-19T05:45:07.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:07.917+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f3cf00c5770 con 0x7f3cf00caf40 2024-09-19T05:45:08.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.598+0000 7f3cecff9640 1 -- 172.21.15.2:0/2990687867 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v89) ==== 113+0+0 (secure 0 0 0) 0x7f3ce007a850 con 0x7f3cf00caf40 2024-09-19T05:45:08.601 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.1 to 1 (10000) 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3cdc06c2b0 msgr2=0x7f3cdc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3cdc06c2b0 0x7f3cdc06e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4006e20 tx=0x7f3ce4008040 comp rx=0 tx=0).stop 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf00caf40 msgr2=0x7f3cf01507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf00caf40 0x7f3cf01507e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0035040 tx=0x7f3ce00046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 shutdown_connections 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3cdc06c2b0 0x7f3cdc06e770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cf0155000 0x7f3cf01e3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3cf0154c20 0x7f3cf01e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 --2- 172.21.15.2:0/2990687867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3cf00caf40 0x7f3cf01507e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.602+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 >> 172.21.15.2:0/2990687867 conn(0x7f3cf006d0e0 msgr2=0x7f3cf01517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:08.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.603+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 shutdown_connections 2024-09-19T05:45:08.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.603+0000 7f3cf68e6640 1 -- 172.21.15.2:0/2990687867 wait complete. 2024-09-19T05:45:08.619 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-19T05:45:08.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.729+0000 7fd7c19b7640 1 Processor -- start 2024-09-19T05:45:08.731 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.729+0000 7fd7c19b7640 1 -- start start 2024-09-19T05:45:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.729+0000 7fd7c19b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc073c70 0x7fd7bc0c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.729+0000 7fd7c19b7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd7bc0724c0 0x7fd7bc072970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.729+0000 7fd7c19b7640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7bc0c4070 con 0x7fd7bc073890 2024-09-19T05:45:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.729+0000 7fd7c19b7640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd7bc1579d0 con 0x7fd7bc0724c0 2024-09-19T05:45:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.729+0000 7fd7c19b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc157cb0 con 0x7fd7bc073c70 2024-09-19T05:45:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7bb7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd7bc0724c0 0x7fd7bc072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7baffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd7bc073890 0x7fd7bc15b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:44104/0 (socket says 172.21.15.2:44104) 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7baffd640 1 -- 172.21.15.2:0/3966786781 learned_addr learned my addr 172.21.15.2:0/3966786781 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7ba7fc640 1 --2- 172.21.15.2:0/3966786781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc073c70 0x7fd7bc0c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7bb7fe640 1 -- 172.21.15.2:0/3966786781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc073c70 msgr2=0x7fd7bc0c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7bb7fe640 1 --2- 172.21.15.2:0/3966786781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc073c70 0x7fd7bc0c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7bb7fe640 1 -- 172.21.15.2:0/3966786781 >> v1:172.21.15.2:6789/0 conn(0x7fd7bc073890 legacy=0x7fd7bc15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7b9ffb640 1 -- 172.21.15.2:0/3966786781 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394905052 0 0) 0x7fd7bc0c4070 con 0x7fd7bc073890 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7bb7fe640 1 -- 172.21.15.2:0/3966786781 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc157f90 con 0x7fd7bc0724c0 2024-09-19T05:45:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.730+0000 7fd7ba7fc640 1 --2- 172.21.15.2:0/3966786781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc073c70 0x7fd7bc0c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:08.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7bb7fe640 1 --2- 172.21.15.2:0/3966786781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc0724c0 0x7fd7bc072970 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009d90 tx=0x7fd7b402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35acaad15f534f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:08.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7b9ffb640 1 -- 172.21.15.2:0/3966786781 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4007740 con 0x7fd7bc0724c0 2024-09-19T05:45:08.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7b9ffb640 1 -- 172.21.15.2:0/3966786781 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b40078e0 con 0x7fd7bc0724c0 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7b9ffb640 1 -- 172.21.15.2:0/3966786781 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4007cb0 con 0x7fd7bc0724c0 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3966786781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc0724c0 msgr2=0x7fd7bc072970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3966786781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc0724c0 0x7fd7bc072970 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009d90 tx=0x7fd7b402a6f0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3966786781 shutdown_connections 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3966786781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc0724c0 0x7fd7bc072970 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3966786781 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc073c70 0x7fd7bc0c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3966786781 >> 172.21.15.2:0/3966786781 conn(0x7fd7bc06bc10 msgr2=0x7fd7bc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3966786781 shutdown_connections 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.731+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3966786781 wait complete. 2024-09-19T05:45:08.735 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 Processor -- start 2024-09-19T05:45:08.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 -- start start 2024-09-19T05:45:08.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc0724c0 0x7fd7bc1ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:08.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7bc073890 0x7fd7bc148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:08.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc073c70 0x7fd7bc0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:08.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc071bc0 con 0x7fd7bc073c70 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd7bc158550 con 0x7fd7bc073890 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.733+0000 7fd7c19b7640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd7bc151670 con 0x7fd7bc0724c0 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc073c70 0x7fd7bc0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc073c70 0x7fd7bc0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46646/0 (socket says 172.21.15.2:46646) 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 -- 172.21.15.2:0/3680963984 learned_addr learned my addr 172.21.15.2:0/3680963984 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7baffd640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc0724c0 0x7fd7bc1ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7ba7fc640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7bc073890 0x7fd7bc148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:08.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 -- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc0724c0 msgr2=0x7fd7bc1ee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc0724c0 0x7fd7bc1ee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 -- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7bc073890 msgr2=0x7fd7bc148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7bc073890 0x7fd7bc148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 -- 172.21.15.2:0/3680963984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7bc1ee0e0 con 0x7fd7bc073c70 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7bb7fe640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc073c70 0x7fd7bc0c5b40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009eb0 tx=0x7fd7b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7a3fff640 1 -- 172.21.15.2:0/3680963984 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4040070 con 0x7fd7bc073c70 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7a3fff640 1 -- 172.21.15.2:0/3680963984 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4031c90 con 0x7fd7bc073c70 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7a3fff640 1 -- 172.21.15.2:0/3680963984 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b403c640 con 0x7fd7bc073c70 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7bc14f1a0 con 0x7fd7bc073c70 2024-09-19T05:45:08.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.734+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7bc1ee380 con 0x7fd7bc073c70 2024-09-19T05:45:08.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.736+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7bc1ee580 con 0x7fd7bc073c70 2024-09-19T05:45:08.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.736+0000 7fd7a3fff640 1 -- 172.21.15.2:0/3680963984 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd7b4039040 con 0x7fd7bc073c70 2024-09-19T05:45:08.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.736+0000 7fd7a3fff640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd7a806c2e0 0x7fd7a806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:08.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.737+0000 7fd7a3fff640 1 -- 172.21.15.2:0/3680963984 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5799+0+0 (secure 0 0 0) 0x7fd7b40aca00 con 0x7fd7bc073c70 2024-09-19T05:45:08.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.737+0000 7fd7baffd640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd7a806c2e0 0x7fd7a806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:08.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.737+0000 7fd7baffd640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd7a806c2e0 0x7fd7a806e7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac000f60 tx=0x7fd7ac007d50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:08.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.741+0000 7fd7a3fff640 1 -- 172.21.15.2:0/3680963984 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd7b407aae0 con 0x7fd7bc073c70 2024-09-19T05:45:08.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:08.927+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) -- 0x7fd7bc0c3950 con 0x7fd7bc073c70 2024-09-19T05:45:09.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.608+0000 7fd7a3fff640 1 -- 172.21.15.2:0/3680963984 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v90) ==== 133+0+0 (secure 0 0 0) 0x7fd7b407ede0 con 0x7fd7bc073c70 2024-09-19T05:45:09.611 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-19T05:45:09.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd7a806c2e0 msgr2=0x7fd7a806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:09.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd7a806c2e0 0x7fd7a806e7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac000f60 tx=0x7fd7ac007d50 comp rx=0 tx=0).stop 2024-09-19T05:45:09.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc073c70 msgr2=0x7fd7bc0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:09.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc073c70 0x7fd7bc0c5b40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4009eb0 tx=0x7fd7b4004a40 comp rx=0 tx=0).stop 2024-09-19T05:45:09.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 shutdown_connections 2024-09-19T05:45:09.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd7a806c2e0 0x7fd7a806e7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7bc073890 0x7fd7bc148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7bc073c70 0x7fd7bc0c5b40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 --2- 172.21.15.2:0/3680963984 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd7bc0724c0 0x7fd7bc1ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 >> 172.21.15.2:0/3680963984 conn(0x7fd7bc06bc10 msgr2=0x7fd7bc1e0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:09.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 shutdown_connections 2024-09-19T05:45:09.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.612+0000 7fd7c19b7640 1 -- 172.21.15.2:0/3680963984 wait complete. 2024-09-19T05:45:09.629 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-19T05:45:09.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.735+0000 7fd8291d8640 1 Processor -- start 2024-09-19T05:45:09.738 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.735+0000 7fd8291d8640 1 -- start start 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd8291d8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd8240c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd8291d8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd82415ede0 0x7fd8240c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd8291d8640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd82415ab40 con 0x7fd8240cba20 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd8291d8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd8240c5ef0 con 0x7fd82415ede0 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd8291d8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd82415ade0 con 0x7fd8240cb640 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd82359e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd82415ede0 0x7fd8240c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd82359e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd82415ede0 0x7fd8240c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46654/0 (socket says 172.21.15.2:46654) 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd82359e640 1 -- 172.21.15.2:0/2974121789 learned_addr learned my addr 172.21.15.2:0/2974121789 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.736+0000 7fd822d9d640 1 --2- 172.21.15.2:0/2974121789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd8240c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.737+0000 7fd82359e640 1 -- 172.21.15.2:0/2974121789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 msgr2=0x7fd8240c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.737+0000 7fd82359e640 1 --2- 172.21.15.2:0/2974121789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd8240c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.737+0000 7fd82359e640 1 -- 172.21.15.2:0/2974121789 >> v1:172.21.15.123:6789/0 conn(0x7fd8240cba20 legacy=0x7fd8241549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.737+0000 7fd82359e640 1 -- 172.21.15.2:0/2974121789 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82415b0c0 con 0x7fd82415ede0 2024-09-19T05:45:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.737+0000 7fd82359e640 1 --2- 172.21.15.2:0/2974121789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8240c6380 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009870 tx=0x7fd81c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de97bd59fdc53ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd821d9b640 1 -- 172.21.15.2:0/2974121789 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c037070 con 0x7fd82415ede0 2024-09-19T05:45:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd821d9b640 1 -- 172.21.15.2:0/2974121789 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81c02ae50 con 0x7fd82415ede0 2024-09-19T05:45:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd821d9b640 1 -- 172.21.15.2:0/2974121789 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c032860 con 0x7fd82415ede0 2024-09-19T05:45:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 -- 172.21.15.2:0/2974121789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 msgr2=0x7fd8240c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 --2- 172.21.15.2:0/2974121789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8240c6380 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009870 tx=0x7fd81c02a300 comp rx=0 tx=0).stop 2024-09-19T05:45:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 -- 172.21.15.2:0/2974121789 shutdown_connections 2024-09-19T05:45:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 --2- 172.21.15.2:0/2974121789 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8240c6380 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 --2- 172.21.15.2:0/2974121789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd8240c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 -- 172.21.15.2:0/2974121789 >> 172.21.15.2:0/2974121789 conn(0x7fd824067950 msgr2=0x7fd824067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 -- 172.21.15.2:0/2974121789 shutdown_connections 2024-09-19T05:45:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.738+0000 7fd8291d8640 1 -- 172.21.15.2:0/2974121789 wait complete. 2024-09-19T05:45:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 Processor -- start 2024-09-19T05:45:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 -- start start 2024-09-19T05:45:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd824155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8240cba20 0x7fd824153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8241f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd82415b960 con 0x7fd82415ede0 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd82415b680 con 0x7fd8240cba20 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd8291d8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd8241550f0 con 0x7fd8240cb640 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd82359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8241f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd82359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8241f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46668/0 (socket says 172.21.15.2:46668) 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd82359e640 1 -- 172.21.15.2:0/672065935 learned_addr learned my addr 172.21.15.2:0/672065935 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.740+0000 7fd822d9d640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd824155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd82259c640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8240cba20 0x7fd824153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd82359e640 1 -- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 msgr2=0x7fd824155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd82359e640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd824155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd82359e640 1 -- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8240cba20 msgr2=0x7fd824153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd82359e640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8240cba20 0x7fd824153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd82359e640 1 -- 172.21.15.2:0/672065935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8241f4b90 con 0x7fd82415ede0 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd82359e640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8241f0c70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009870 tx=0x7fd81c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd81b7fe640 1 -- 172.21.15.2:0/672065935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c041070 con 0x7fd82415ede0 2024-09-19T05:45:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd81b7fe640 1 -- 172.21.15.2:0/672065935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81c03c680 con 0x7fd82415ede0 2024-09-19T05:45:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8240b5a30 con 0x7fd82415ede0 2024-09-19T05:45:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd81b7fe640 1 -- 172.21.15.2:0/672065935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c037040 con 0x7fd82415ede0 2024-09-19T05:45:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.741+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8240b5ff0 con 0x7fd82415ede0 2024-09-19T05:45:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.742+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd824158e00 con 0x7fd82415ede0 2024-09-19T05:45:09.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.743+0000 7fd81b7fe640 1 -- 172.21.15.2:0/672065935 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd81c044070 con 0x7fd82415ede0 2024-09-19T05:45:09.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.743+0000 7fd81b7fe640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd80406c2b0 0x7fd80406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:09.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.743+0000 7fd81b7fe640 1 -- 172.21.15.2:0/672065935 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5799+0+0 (secure 0 0 0) 0x7fd81c0abd50 con 0x7fd82415ede0 2024-09-19T05:45:09.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.743+0000 7fd822d9d640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd80406c2b0 0x7fd80406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:09.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.744+0000 7fd822d9d640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd80406c2b0 0x7fd80406e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8140113e0 tx=0x7fd81400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:09.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.747+0000 7fd81b7fe640 1 -- 172.21.15.2:0/672065935 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd824158e00 con 0x7fd82415ede0 2024-09-19T05:45:09.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:09.934+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7fd8241549e0 con 0x7fd82415ede0 2024-09-19T05:45:10.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.614+0000 7fd81b7fe640 1 -- 172.21.15.2:0/672065935 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v91) ==== 113+0+0 (secure 0 0 0) 0x7fd81c0af030 con 0x7fd82415ede0 2024-09-19T05:45:10.616 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd80406c2b0 msgr2=0x7fd80406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd80406c2b0 0x7fd80406e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8140113e0 tx=0x7fd81400b040 comp rx=0 tx=0).stop 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 msgr2=0x7fd8241f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8241f0c70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009870 tx=0x7fd81c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 shutdown_connections 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd80406c2b0 0x7fd80406e770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8240cba20 0x7fd824153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd82415ede0 0x7fd8241f0c70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 --2- 172.21.15.2:0/672065935 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8240cb640 0x7fd824155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 >> 172.21.15.2:0/672065935 conn(0x7fd824067950 msgr2=0x7fd8240d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 shutdown_connections 2024-09-19T05:45:10.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.617+0000 7fd8291d8640 1 -- 172.21.15.2:0/672065935 wait complete. 2024-09-19T05:45:10.637 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-19T05:45:10.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.745+0000 7f3e7561e640 1 Processor -- start 2024-09-19T05:45:10.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.745+0000 7f3e7561e640 1 -- start start 2024-09-19T05:45:10.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e7561e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3e70073890 0x7f3e700c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:10.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e7561e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e700724c0 0x7f3e70072930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:10.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e7561e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e700c4070 con 0x7f3e70073c70 2024-09-19T05:45:10.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e7561e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3e70072e70 con 0x7f3e70073890 2024-09-19T05:45:10.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e7561e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3e701579d0 con 0x7f3e700724c0 2024-09-19T05:45:10.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e6effd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3e70073890 0x7f3e700c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:10.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e6e7fc640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3e70073c70 0x7f3e7015b7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:42554/0 (socket says 172.21.15.2:42554) 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e6f7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e700724c0 0x7f3e70072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.746+0000 7f3e6e7fc640 1 -- 172.21.15.2:0/1467477363 learned_addr learned my addr 172.21.15.2:0/1467477363 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6dffb640 1 -- 172.21.15.2:0/1467477363 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420058537 0 0) 0x7f3e700c4070 con 0x7f3e70073c70 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6dffb640 1 -- 172.21.15.2:0/1467477363 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e5c003520 con 0x7f3e70073c70 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6effd640 1 -- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e700724c0 msgr2=0x7f3e70072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6effd640 1 --2- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e700724c0 0x7f3e70072930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6effd640 1 -- 172.21.15.2:0/1467477363 >> v1:172.21.15.123:6789/0 conn(0x7f3e70073c70 legacy=0x7f3e7015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6effd640 1 -- 172.21.15.2:0/1467477363 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e70157cb0 con 0x7f3e70073890 2024-09-19T05:45:10.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6f7fe640 1 --2- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e700724c0 0x7f3e70072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:10.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.747+0000 7f3e6effd640 1 --2- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 0x7f3e700c3950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3e680097a0 tx=0x7f3e6802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fdb4f680f08e70b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e6dffb640 1 -- 172.21.15.2:0/1467477363 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68037070 con 0x7f3e70073890 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e6dffb640 1 -- 172.21.15.2:0/1467477363 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e68032440 con 0x7f3e70073890 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e6dffb640 1 -- 172.21.15.2:0/1467477363 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68032810 con 0x7f3e70073890 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 -- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 msgr2=0x7f3e700c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 --2- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 0x7f3e700c3950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3e680097a0 tx=0x7f3e6802a210 comp rx=0 tx=0).stop 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 -- 172.21.15.2:0/1467477363 shutdown_connections 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 --2- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e700724c0 0x7f3e70072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 --2- 172.21.15.2:0/1467477363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 0x7f3e700c3950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 -- 172.21.15.2:0/1467477363 >> 172.21.15.2:0/1467477363 conn(0x7f3e7006bc10 msgr2=0x7f3e7006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 -- 172.21.15.2:0/1467477363 shutdown_connections 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.748+0000 7f3e7561e640 1 -- 172.21.15.2:0/1467477363 wait complete. 2024-09-19T05:45:10.752 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 Processor -- start 2024-09-19T05:45:10.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 -- start start 2024-09-19T05:45:10.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e700724c0 0x7f3e701568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:10.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 0x7f3e70155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e70073c70 0x7f3e701effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70158550 con 0x7f3e700724c0 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3e70158270 con 0x7f3e70073890 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e7561e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3e70071bc0 con 0x7f3e70073c70 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.750+0000 7f3e6effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e700724c0 0x7f3e701568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e700724c0 0x7f3e701568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:46674/0 (socket says 172.21.15.2:46674) 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 -- 172.21.15.2:0/252625250 learned_addr learned my addr 172.21.15.2:0/252625250 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6f7fe640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e70073c70 0x7f3e701effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6e7fc640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 0x7f3e70155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 -- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e70073c70 msgr2=0x7f3e701effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e70073c70 0x7f3e701effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 -- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 msgr2=0x7f3e70155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:10.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 0x7f3e70155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 -- 172.21.15.2:0/252625250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7014f330 con 0x7f3e700724c0 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e6effd640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e700724c0 0x7f3e701568d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e680098c0 tx=0x7f3e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e677fe640 1 -- 172.21.15.2:0/252625250 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68041030 con 0x7f3e700724c0 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e677fe640 1 -- 172.21.15.2:0/252625250 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6803d420 con 0x7f3e700724c0 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e700cb950 con 0x7f3e700724c0 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e677fe640 1 -- 172.21.15.2:0/252625250 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68037040 con 0x7f3e700724c0 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.751+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e700cbf10 con 0x7f3e700724c0 2024-09-19T05:45:10.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.752+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e70072930 con 0x7f3e700724c0 2024-09-19T05:45:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.753+0000 7f3e677fe640 1 -- 172.21.15.2:0/252625250 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3e68044070 con 0x7f3e700724c0 2024-09-19T05:45:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.753+0000 7f3e677fe640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3e5006c2e0 0x7f3e5006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.753+0000 7f3e677fe640 1 -- 172.21.15.2:0/252625250 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5799+0+0 (secure 0 0 0) 0x7f3e680ab940 con 0x7f3e700724c0 2024-09-19T05:45:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.754+0000 7f3e6e7fc640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3e5006c2e0 0x7f3e5006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:10.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.754+0000 7f3e6e7fc640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3e5006c2e0 0x7f3e5006e7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e70157900 tx=0x7f3e58009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:10.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.758+0000 7f3e677fe640 1 -- 172.21.15.2:0/252625250 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3e6807a060 con 0x7f3e700724c0 2024-09-19T05:45:10.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:10.947+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) -- 0x7f3e7014ecb0 con 0x7f3e700724c0 2024-09-19T05:45:11.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.615+0000 7f3e677fe640 1 -- 172.21.15.2:0/252625250 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v92) ==== 133+0+0 (secure 0 0 0) 0x7f3e68079960 con 0x7f3e700724c0 2024-09-19T05:45:11.618 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-19T05:45:11.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3e5006c2e0 msgr2=0x7f3e5006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:11.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3e5006c2e0 0x7f3e5006e7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3e70157900 tx=0x7f3e58009040 comp rx=0 tx=0).stop 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e700724c0 msgr2=0x7f3e701568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e700724c0 0x7f3e701568d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e680098c0 tx=0x7f3e680046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 shutdown_connections 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3e5006c2e0 0x7f3e5006e7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3e70073890 0x7f3e70155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3e70073c70 0x7f3e701effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 --2- 172.21.15.2:0/252625250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e700724c0 0x7f3e701568d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 >> 172.21.15.2:0/252625250 conn(0x7f3e7006bc10 msgr2=0x7f3e70151440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 shutdown_connections 2024-09-19T05:45:11.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.619+0000 7f3e7561e640 1 -- 172.21.15.2:0/252625250 wait complete. 2024-09-19T05:45:11.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-19T05:45:11.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.745+0000 7f591029b640 1 Processor -- start 2024-09-19T05:45:11.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.745+0000 7f591029b640 1 -- start start 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f591029b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5908154c20 0x7f59080cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f591029b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5908155000 0x7f5908073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f591029b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59080ccbc0 con 0x7f59080caf40 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f591029b640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f59080cc600 con 0x7f5908155000 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f591029b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f59080cc8e0 con 0x7f5908154c20 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590d836640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5908155000 0x7f5908073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590d836640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5908155000 0x7f5908073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53380/0 (socket says 172.21.15.2:53380) 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590d836640 1 -- 172.21.15.2:0/2598548011 learned_addr learned my addr 172.21.15.2:0/2598548011 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590e037640 1 --2- 172.21.15.2:0/2598548011 >> v2:172.21.15.123:3300/0 conn(0x7f5908154c20 0x7f59080cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590d836640 1 -- 172.21.15.2:0/2598548011 >> v2:172.21.15.123:3300/0 conn(0x7f5908154c20 msgr2=0x7f59080cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590d836640 1 --2- 172.21.15.2:0/2598548011 >> v2:172.21.15.123:3300/0 conn(0x7f5908154c20 0x7f59080cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590d836640 1 -- 172.21.15.2:0/2598548011 >> v1:172.21.15.123:6789/0 conn(0x7f59080caf40 legacy=0x7f590815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:11.749 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.746+0000 7f590d836640 1 -- 172.21.15.2:0/2598548011 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080cbc60 con 0x7f5908155000 2024-09-19T05:45:11.750 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.747+0000 7f590d836640 1 --2- 172.21.15.2:0/2598548011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5908155000 0x7f5908073340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5908073ed0 tx=0x7f58f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57fb59e30de1737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.747+0000 7f590d035640 1 -- 172.21.15.2:0/2598548011 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8004030 con 0x7f5908155000 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.747+0000 7f590d035640 1 -- 172.21.15.2:0/2598548011 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f80041d0 con 0x7f5908155000 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.747+0000 7f590d035640 1 -- 172.21.15.2:0/2598548011 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8003610 con 0x7f5908155000 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 -- 172.21.15.2:0/2598548011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5908155000 msgr2=0x7f5908073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 --2- 172.21.15.2:0/2598548011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5908155000 0x7f5908073340 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5908073ed0 tx=0x7f58f802a270 comp rx=0 tx=0).stop 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 -- 172.21.15.2:0/2598548011 shutdown_connections 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 --2- 172.21.15.2:0/2598548011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5908155000 0x7f5908073340 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 --2- 172.21.15.2:0/2598548011 >> v2:172.21.15.123:3300/0 conn(0x7f5908154c20 0x7f59080cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 -- 172.21.15.2:0/2598548011 >> 172.21.15.2:0/2598548011 conn(0x7f590806d0e0 msgr2=0x7f590806d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 -- 172.21.15.2:0/2598548011 shutdown_connections 2024-09-19T05:45:11.751 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.748+0000 7f591029b640 1 -- 172.21.15.2:0/2598548011 wait complete. 2024-09-19T05:45:11.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.749+0000 7f591029b640 1 Processor -- start 2024-09-19T05:45:11.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.749+0000 7f591029b640 1 -- start start 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f591029b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59080caf40 0x7f59081f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f591029b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5908154c20 0x7f590814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f591029b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908155000 0x7f5908158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f591029b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f590815b1f0 con 0x7f59080caf40 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f591029b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59080cb430 con 0x7f5908155000 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f591029b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f590815b370 con 0x7f5908154c20 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59080caf40 0x7f59081f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59080caf40 0x7f59081f13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53394/0 (socket says 172.21.15.2:53394) 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 -- 172.21.15.2:0/1769745103 learned_addr learned my addr 172.21.15.2:0/1769745103 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 -- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5908154c20 msgr2=0x7f590814b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5908154c20 0x7f590814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 -- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908155000 msgr2=0x7f5908158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908155000 0x7f5908158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:11.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 -- 172.21.15.2:0/1769745103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081f1900 con 0x7f59080caf40 2024-09-19T05:45:11.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f590e037640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59080caf40 0x7f59081f13c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f590400d830 tx=0x7f590400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:11.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f58feffd640 1 -- 172.21.15.2:0/1769745103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590400b6c0 con 0x7f59080caf40 2024-09-19T05:45:11.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.750+0000 7f58feffd640 1 -- 172.21.15.2:0/1769745103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5904019070 con 0x7f59080caf40 2024-09-19T05:45:11.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.751+0000 7f58feffd640 1 -- 172.21.15.2:0/1769745103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590400bac0 con 0x7f59080caf40 2024-09-19T05:45:11.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.751+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59081e0cb0 con 0x7f59080caf40 2024-09-19T05:45:11.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.751+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081e1230 con 0x7f59080caf40 2024-09-19T05:45:11.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.751+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5908152210 con 0x7f59080caf40 2024-09-19T05:45:11.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.752+0000 7f58feffd640 1 -- 172.21.15.2:0/1769745103 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f590401e460 con 0x7f59080caf40 2024-09-19T05:45:11.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.752+0000 7f58feffd640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:11.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.752+0000 7f58feffd640 1 -- 172.21.15.2:0/1769745103 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5799+0+0 (secure 0 0 0) 0x7f590408bc30 con 0x7f59080caf40 2024-09-19T05:45:11.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.753+0000 7f590d836640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:11.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.753+0000 7f590d836640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f58f8033000 tx=0x7f58f802c000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:11.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.757+0000 7f58feffd640 1 -- 172.21.15.2:0/1769745103 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5908152210 con 0x7f59080caf40 2024-09-19T05:45:11.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:11.945+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7f59080c5770 con 0x7f59080caf40 2024-09-19T05:45:12.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.621+0000 7f58feffd640 1 -- 172.21.15.2:0/1769745103 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v93) ==== 113+0+0 (secure 0 0 0) 0x7f590405a2b0 con 0x7f59080caf40 2024-09-19T05:45:12.624 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-19T05:45:12.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f58ec06c2b0 msgr2=0x7f58ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:12.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f58f8033000 tx=0x7f58f802c000 comp rx=0 tx=0).stop 2024-09-19T05:45:12.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59080caf40 msgr2=0x7f59081f13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59080caf40 0x7f59081f13c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f590400d830 tx=0x7f590400dcf0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 shutdown_connections 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5908155000 0x7f5908158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5908154c20 0x7f590814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 --2- 172.21.15.2:0/1769745103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59080caf40 0x7f59081f13c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 >> 172.21.15.2:0/1769745103 conn(0x7f590806d0e0 msgr2=0x7f5908153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 shutdown_connections 2024-09-19T05:45:12.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.625+0000 7f591029b640 1 -- 172.21.15.2:0/1769745103 wait complete. 2024-09-19T05:45:12.642 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-19T05:45:12.753 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.751+0000 7fafdda23640 1 Processor -- start 2024-09-19T05:45:12.754 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.751+0000 7fafdda23640 1 -- start start 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafdda23640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafd8154d50 0x7fafd8073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafdda23640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafdda23640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd8155130 con 0x7fafd80cc3f0 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafdda23640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fafd80cbc60 con 0x7fafd8154d50 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafdda23640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fafd80cb050 con 0x7fafd815b660 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafd67fc640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fafd80cc3f0 0x7fafd815af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:52774/0 (socket says 172.21.15.2:52774) 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafd67fc640 1 -- 172.21.15.2:0/42882548 learned_addr learned my addr 172.21.15.2:0/42882548 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafd6ffd640 1 --2- 172.21.15.2:0/42882548 >> v2:172.21.15.123:3300/0 conn(0x7fafd8154d50 0x7fafd8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.752+0000 7fafd77fe640 1 --2- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.753+0000 7fafd5ffb640 1 -- 172.21.15.2:0/42882548 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1705636935 0 0) 0x7fafd8155130 con 0x7fafd80cc3f0 2024-09-19T05:45:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.753+0000 7fafd5ffb640 1 -- 172.21.15.2:0/42882548 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc4003520 con 0x7fafd80cc3f0 2024-09-19T05:45:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.753+0000 7fafd6ffd640 1 -- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 msgr2=0x7fafd80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.753+0000 7fafd6ffd640 1 --2- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.753+0000 7fafd6ffd640 1 -- 172.21.15.2:0/42882548 >> v1:172.21.15.2:6789/0 conn(0x7fafd80cc3f0 legacy=0x7fafd815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.753+0000 7fafd6ffd640 1 -- 172.21.15.2:0/42882548 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd80cb330 con 0x7fafd8154d50 2024-09-19T05:45:12.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafd6ffd640 1 --2- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd8154d50 0x7fafd8073390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009d90 tx=0x7fafc802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51964831fc63cebd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:12.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafd5ffb640 1 -- 172.21.15.2:0/42882548 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8007740 con 0x7fafd8154d50 2024-09-19T05:45:12.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafd5ffb640 1 -- 172.21.15.2:0/42882548 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc80078e0 con 0x7fafd8154d50 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafd5ffb640 1 -- 172.21.15.2:0/42882548 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8007c90 con 0x7fafd8154d50 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 -- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd8154d50 msgr2=0x7fafd8073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 --2- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd8154d50 0x7fafd8073390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009d90 tx=0x7fafc802a6f0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 -- 172.21.15.2:0/42882548 shutdown_connections 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 --2- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 --2- 172.21.15.2:0/42882548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd8154d50 0x7fafd8073390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 -- 172.21.15.2:0/42882548 >> 172.21.15.2:0/42882548 conn(0x7fafd806d250 msgr2=0x7fafd806d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 -- 172.21.15.2:0/42882548 shutdown_connections 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.754+0000 7fafdda23640 1 -- 172.21.15.2:0/42882548 wait complete. 2024-09-19T05:45:12.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.755+0000 7fafdda23640 1 Processor -- start 2024-09-19T05:45:12.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafdda23640 1 -- start start 2024-09-19T05:45:12.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafdda23640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd80cc3f0 0x7fafd81596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:12.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafdda23640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafd8154d50 0x7fafd81f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:12.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafdda23640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd8148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:12.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafdda23640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fafd815aa00 con 0x7fafd8154d50 2024-09-19T05:45:12.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafdda23640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafd80c5c40 con 0x7fafd80cc3f0 2024-09-19T05:45:12.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafdda23640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fafd80cbec0 con 0x7fafd815b660 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.756+0000 7fafd67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafd8154d50 0x7fafd81f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafd8154d50 0x7fafd81f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53408/0 (socket says 172.21.15.2:53408) 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 -- 172.21.15.2:0/2074285947 learned_addr learned my addr 172.21.15.2:0/2074285947 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd6ffd640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd80cc3f0 0x7fafd81596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd77fe640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd8148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 -- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 msgr2=0x7fafd8148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd8148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 -- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd80cc3f0 msgr2=0x7fafd81596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd80cc3f0 0x7fafd81596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 -- 172.21.15.2:0/2074285947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd81f4880 con 0x7fafd8154d50 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd77fe640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd8148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafd67fc640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafd8154d50 0x7fafd81f3dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c530 tx=0x7fafc000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:12.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafcffff640 1 -- 172.21.15.2:0/2074285947 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0017070 con 0x7fafd8154d50 2024-09-19T05:45:12.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafcffff640 1 -- 172.21.15.2:0/2074285947 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc0005ce0 con 0x7fafd8154d50 2024-09-19T05:45:12.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd80ca400 con 0x7fafd8154d50 2024-09-19T05:45:12.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafcffff640 1 -- 172.21.15.2:0/2074285947 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00072b0 con 0x7fafd8154d50 2024-09-19T05:45:12.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.757+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81505a0 con 0x7fafd8154d50 2024-09-19T05:45:12.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.758+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd80c69f0 con 0x7fafd8154d50 2024-09-19T05:45:12.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.759+0000 7fafcffff640 1 -- 172.21.15.2:0/2074285947 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fafc001a070 con 0x7fafd8154d50 2024-09-19T05:45:12.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.759+0000 7fafcffff640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fafb806c2b0 0x7fafb806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:12.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.759+0000 7fafcffff640 1 -- 172.21.15.2:0/2074285947 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5799+0+0 (secure 0 0 0) 0x7fafc008a8d0 con 0x7fafd8154d50 2024-09-19T05:45:12.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.760+0000 7fafd6ffd640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fafb806c2b0 0x7fafb806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:12.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.760+0000 7fafd6ffd640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fafb806c2b0 0x7fafb806e770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafc8034000 tx=0x7fafc802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:12.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.764+0000 7fafcffff640 1 -- 172.21.15.2:0/2074285947 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fafc00589c0 con 0x7fafd8154d50 2024-09-19T05:45:12.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:12.954+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) -- 0x7fafd80c61f0 con 0x7fafd8154d50 2024-09-19T05:45:13.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.627+0000 7fafcffff640 1 -- 172.21.15.2:0/2074285947 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v94) ==== 133+0+0 (secure 0 0 0) 0x7fafc005ccc0 con 0x7fafd8154d50 2024-09-19T05:45:13.630 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-19T05:45:13.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fafb806c2b0 msgr2=0x7fafb806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:13.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fafb806c2b0 0x7fafb806e770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fafc8034000 tx=0x7fafc802f000 comp rx=0 tx=0).stop 2024-09-19T05:45:13.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafd8154d50 msgr2=0x7fafd81f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:13.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafd8154d50 0x7fafd81f3dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c530 tx=0x7fafc000c9f0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 shutdown_connections 2024-09-19T05:45:13.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fafb806c2b0 0x7fafb806e770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafd815b660 0x7fafd8148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafd8154d50 0x7fafd81f3dc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 --2- 172.21.15.2:0/2074285947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd80cc3f0 0x7fafd81596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 >> 172.21.15.2:0/2074285947 conn(0x7fafd806d250 msgr2=0x7fafd814b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:13.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 shutdown_connections 2024-09-19T05:45:13.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.631+0000 7fafdda23640 1 -- 172.21.15.2:0/2074285947 wait complete. 2024-09-19T05:45:13.647 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-19T05:45:13.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.757+0000 7fdd07e2e640 1 Processor -- start 2024-09-19T05:45:13.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.757+0000 7fdd07e2e640 1 -- start start 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd07e2e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd00154c20 0x7fdd000cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd07e2e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd000caf40 0x7fdd0015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd07e2e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd000ccbc0 con 0x7fdd00155000 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd07e2e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdd000cc600 con 0x7fdd00154c20 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd07e2e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdd000cc8e0 con 0x7fdd000caf40 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd053c9640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdd00155000 0x7fdd00073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:56110/0 (socket says 172.21.15.2:56110) 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd05bca640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdd00154c20 0x7fdd000cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd053c9640 1 -- 172.21.15.2:0/1686330117 learned_addr learned my addr 172.21.15.2:0/1686330117 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd063cb640 1 --2- 172.21.15.2:0/1686330117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd000caf40 0x7fdd0015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd04bc8640 1 -- 172.21.15.2:0/1686330117 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696777332 0 0) 0x7fdd000ccbc0 con 0x7fdd00155000 2024-09-19T05:45:13.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd04bc8640 1 -- 172.21.15.2:0/1686330117 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcec003520 con 0x7fdd00155000 2024-09-19T05:45:13.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd063cb640 1 -- 172.21.15.2:0/1686330117 >> v2:172.21.15.123:3300/0 conn(0x7fdd00154c20 msgr2=0x7fdd000cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:13.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd063cb640 1 --2- 172.21.15.2:0/1686330117 >> v2:172.21.15.123:3300/0 conn(0x7fdd00154c20 0x7fdd000cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd063cb640 1 -- 172.21.15.2:0/1686330117 >> v1:172.21.15.123:6789/0 conn(0x7fdd00155000 legacy=0x7fdd00073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:13.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd063cb640 1 -- 172.21.15.2:0/1686330117 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd000cbc60 con 0x7fdd000caf40 2024-09-19T05:45:13.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.758+0000 7fdd05bca640 1 --2- 172.21.15.2:0/1686330117 >> v2:172.21.15.123:3300/0 conn(0x7fdd00154c20 0x7fdd000cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:13.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.759+0000 7fdd063cb640 1 --2- 172.21.15.2:0/1686330117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd000caf40 0x7fdd0015aae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c56a41d070d510 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:13.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.759+0000 7fdd04bc8640 1 -- 172.21.15.2:0/1686330117 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8007740 con 0x7fdd000caf40 2024-09-19T05:45:13.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.759+0000 7fdd04bc8640 1 -- 172.21.15.2:0/1686330117 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf80078e0 con 0x7fdd000caf40 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.759+0000 7fdd04bc8640 1 -- 172.21.15.2:0/1686330117 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8007cb0 con 0x7fdd000caf40 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 -- 172.21.15.2:0/1686330117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd000caf40 msgr2=0x7fdd0015aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/1686330117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd000caf40 0x7fdd0015aae0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009d90 tx=0x7fdcf802a660 comp rx=0 tx=0).stop 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 -- 172.21.15.2:0/1686330117 shutdown_connections 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/1686330117 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd000caf40 0x7fdd0015aae0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/1686330117 >> v2:172.21.15.123:3300/0 conn(0x7fdd00154c20 0x7fdd000cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 -- 172.21.15.2:0/1686330117 >> 172.21.15.2:0/1686330117 conn(0x7fdd0006d0e0 msgr2=0x7fdd0006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 -- 172.21.15.2:0/1686330117 shutdown_connections 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.760+0000 7fdd07e2e640 1 -- 172.21.15.2:0/1686330117 wait complete. 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.761+0000 7fdd07e2e640 1 Processor -- start 2024-09-19T05:45:13.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.761+0000 7fdd07e2e640 1 -- start start 2024-09-19T05:45:13.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd07e2e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd000caf40 0x7fdd001473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:13.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd07e2e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd00154c20 0x7fdd001e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd07e2e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd00155000 0x7fdd00149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd07e2e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00073a50 con 0x7fdd000caf40 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd07e2e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd000cb430 con 0x7fdd00154c20 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd07e2e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdd00073bd0 con 0x7fdd00155000 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd000caf40 0x7fdd001473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd000caf40 0x7fdd001473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53418/0 (socket says 172.21.15.2:53418) 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 -- 172.21.15.2:0/3159716661 learned_addr learned my addr 172.21.15.2:0/3159716661 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd063cb640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd00155000 0x7fdd00149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd053c9640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd00154c20 0x7fdd001e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 -- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd00155000 msgr2=0x7fdd00149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd00155000 0x7fdd00149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 -- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd00154c20 msgr2=0x7fdd001e0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:13.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd00154c20 0x7fdd001e0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.762+0000 7fdd05bca640 1 -- 172.21.15.2:0/3159716661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00146300 con 0x7fdd000caf40 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.763+0000 7fdd053c9640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd00154c20 0x7fdd001e0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.763+0000 7fdd05bca640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd000caf40 0x7fdd001473a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00ec10 tx=0x7fdcfc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.763+0000 7fdcf67fc640 1 -- 172.21.15.2:0/3159716661 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc017070 con 0x7fdd000caf40 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.763+0000 7fdcf67fc640 1 -- 172.21.15.2:0/3159716661 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcfc00cbd0 con 0x7fdd000caf40 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.763+0000 7fdcf67fc640 1 -- 172.21.15.2:0/3159716661 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc013470 con 0x7fdd000caf40 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.763+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd001465e0 con 0x7fdd000caf40 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.763+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd00146ba0 con 0x7fdd000caf40 2024-09-19T05:45:13.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.764+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd00151f10 con 0x7fdd000caf40 2024-09-19T05:45:13.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.764+0000 7fdcf67fc640 1 -- 172.21.15.2:0/3159716661 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdcfc01a070 con 0x7fdd000caf40 2024-09-19T05:45:13.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.765+0000 7fdcf67fc640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdce406c2b0 0x7fdce406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:13.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.765+0000 7fdcf67fc640 1 -- 172.21.15.2:0/3159716661 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5799+0+0 (secure 0 0 0) 0x7fdcfc08b5f0 con 0x7fdd000caf40 2024-09-19T05:45:13.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.765+0000 7fdd053c9640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdce406c2b0 0x7fdce406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:13.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.766+0000 7fdd053c9640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdce406c2b0 0x7fdce406e770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd000b3b20 tx=0x7fdcf0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:13.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.769+0000 7fdcf67fc640 1 -- 172.21.15.2:0/3159716661 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdd00151f10 con 0x7fdd000caf40 2024-09-19T05:45:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:13.954+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7fdd000c5770 con 0x7fdd000caf40 2024-09-19T05:45:14.636 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.634+0000 7fdcf67fc640 1 -- 172.21.15.2:0/3159716661 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v95) ==== 113+0+0 (secure 0 0 0) 0x7fdcfc059740 con 0x7fdd000caf40 2024-09-19T05:45:14.636 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.4 to 1 (10000) 2024-09-19T05:45:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdce406c2b0 msgr2=0x7fdce406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdce406c2b0 0x7fdce406e770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdd000b3b20 tx=0x7fdcf0009040 comp rx=0 tx=0).stop 2024-09-19T05:45:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd000caf40 msgr2=0x7fdd001473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd000caf40 0x7fdd001473a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00ec10 tx=0x7fdcfc00c520 comp rx=0 tx=0).stop 2024-09-19T05:45:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 shutdown_connections 2024-09-19T05:45:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdce406c2b0 0x7fdce406e770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd00154c20 0x7fdd001e0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdd00155000 0x7fdd00149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 --2- 172.21.15.2:0/3159716661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd000caf40 0x7fdd001473a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 >> 172.21.15.2:0/3159716661 conn(0x7fdd0006d0e0 msgr2=0x7fdd00151800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 shutdown_connections 2024-09-19T05:45:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.637+0000 7fdd07e2e640 1 -- 172.21.15.2:0/3159716661 wait complete. 2024-09-19T05:45:14.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-19T05:45:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.762+0000 7f66edfe4640 1 Processor -- start 2024-09-19T05:45:14.765 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.762+0000 7f66edfe4640 1 -- start start 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66edfe4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f66e80cb640 0x7f66e80c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66edfe4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f66e80cba20 0x7f66e81549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66edfe4640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e815ab40 con 0x7f66e815ede0 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66edfe4640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f66e80c5ef0 con 0x7f66e80cba20 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66edfe4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f66e815ade0 con 0x7f66e80cb640 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e6ffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f66e80cba20 0x7f66e81549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e6ffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f66e80cba20 0x7f66e81549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53428/0 (socket says 172.21.15.2:53428) 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e6ffd640 1 -- 172.21.15.2:0/3864898165 learned_addr learned my addr 172.21.15.2:0/3864898165 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e77fe640 1 --2- 172.21.15.2:0/3864898165 >> v2:172.21.15.123:3300/0 conn(0x7f66e80cb640 0x7f66e80c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e6ffd640 1 -- 172.21.15.2:0/3864898165 >> v2:172.21.15.123:3300/0 conn(0x7f66e80cb640 msgr2=0x7f66e80c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e6ffd640 1 --2- 172.21.15.2:0/3864898165 >> v2:172.21.15.123:3300/0 conn(0x7f66e80cb640 0x7f66e80c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e6ffd640 1 -- 172.21.15.2:0/3864898165 >> v1:172.21.15.123:6789/0 conn(0x7f66e815ede0 legacy=0x7f66e80c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:14.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.763+0000 7f66e6ffd640 1 -- 172.21.15.2:0/3864898165 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e815b0c0 con 0x7f66e80cba20 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.764+0000 7f66e6ffd640 1 --2- 172.21.15.2:0/3864898165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e81549e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f66e8073fd0 tx=0x7f66dc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acde6d754a514154 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.764+0000 7f66e67fc640 1 -- 172.21.15.2:0/3864898165 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc037070 con 0x7f66e80cba20 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.764+0000 7f66e67fc640 1 -- 172.21.15.2:0/3864898165 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66dc02ae50 con 0x7f66e80cba20 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.764+0000 7f66e67fc640 1 -- 172.21.15.2:0/3864898165 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc032860 con 0x7f66e80cba20 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 -- 172.21.15.2:0/3864898165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 msgr2=0x7f66e81549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 --2- 172.21.15.2:0/3864898165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e81549e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f66e8073fd0 tx=0x7f66dc02a300 comp rx=0 tx=0).stop 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 -- 172.21.15.2:0/3864898165 shutdown_connections 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 --2- 172.21.15.2:0/3864898165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e81549e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 --2- 172.21.15.2:0/3864898165 >> v2:172.21.15.123:3300/0 conn(0x7f66e80cb640 0x7f66e80c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 -- 172.21.15.2:0/3864898165 >> 172.21.15.2:0/3864898165 conn(0x7f66e8067950 msgr2=0x7f66e8067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 -- 172.21.15.2:0/3864898165 shutdown_connections 2024-09-19T05:45:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.765+0000 7f66edfe4640 1 -- 172.21.15.2:0/3864898165 wait complete. 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.766+0000 7f66edfe4640 1 Processor -- start 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.766+0000 7f66edfe4640 1 -- start start 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66edfe4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66e80cb640 0x7f66e81f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66edfe4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e8153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66edfe4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e815ede0 0x7f66e80cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66edfe4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f66e815b960 con 0x7f66e80cba20 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66edfe4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66e815b680 con 0x7f66e815ede0 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66edfe4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f66e80c6a90 con 0x7f66e80cb640 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e8153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e8153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53436/0 (socket says 172.21.15.2:53436) 2024-09-19T05:45:14.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 -- 172.21.15.2:0/1941666995 learned_addr learned my addr 172.21.15.2:0/1941666995 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e7fff640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e815ede0 0x7f66e80cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e77fe640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66e80cb640 0x7f66e81f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 -- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66e80cb640 msgr2=0x7f66e81f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66e80cb640 0x7f66e81f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 -- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e815ede0 msgr2=0x7f66e80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e815ede0 0x7f66e80cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e6ffd640 1 -- 172.21.15.2:0/1941666995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e81504c0 con 0x7f66e80cba20 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.767+0000 7f66e77fe640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66e80cb640 0x7f66e81f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.768+0000 7f66e6ffd640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e8153fe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f66dc02a340 tx=0x7f66dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.768+0000 7f66cffff640 1 -- 172.21.15.2:0/1941666995 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc041070 con 0x7f66e80cba20 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.768+0000 7f66cffff640 1 -- 172.21.15.2:0/1941666995 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66dc03c680 con 0x7f66e80cba20 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.768+0000 7f66cffff640 1 -- 172.21.15.2:0/1941666995 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc037040 con 0x7f66e80cba20 2024-09-19T05:45:14.772 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.768+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e8153800 con 0x7f66e80cba20 2024-09-19T05:45:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.768+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e8155550 con 0x7f66e80cba20 2024-09-19T05:45:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.769+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e80c6380 con 0x7f66e80cba20 2024-09-19T05:45:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.769+0000 7f66cffff640 1 -- 172.21.15.2:0/1941666995 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f66dc044070 con 0x7f66e80cba20 2024-09-19T05:45:14.773 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.770+0000 7f66cffff640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f66d406c2b0 0x7f66d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:14.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.770+0000 7f66cffff640 1 -- 172.21.15.2:0/1941666995 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5799+0+0 (secure 0 0 0) 0x7f66dc0ab900 con 0x7f66e80cba20 2024-09-19T05:45:14.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.770+0000 7f66e77fe640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f66d406c2b0 0x7f66d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:14.774 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.770+0000 7f66e77fe640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f66d406c2b0 0x7f66d406e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009b90 tx=0x7f66d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:14.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.774+0000 7f66cffff640 1 -- 172.21.15.2:0/1941666995 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f66e81504c0 con 0x7f66e80cba20 2024-09-19T05:45:14.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:14.961+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) -- 0x7f66e80cbf10 con 0x7f66e80cba20 2024-09-19T05:45:15.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.644+0000 7f66cffff640 1 -- 172.21.15.2:0/1941666995 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v96) ==== 133+0+0 (secure 0 0 0) 0x7f66dc079b70 con 0x7f66e80cba20 2024-09-19T05:45:15.647 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f66d406c2b0 msgr2=0x7f66d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f66d406c2b0 0x7f66d406e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009b90 tx=0x7f66d8008040 comp rx=0 tx=0).stop 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 msgr2=0x7f66e8153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e8153fe0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f66dc02a340 tx=0x7f66dc0046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 shutdown_connections 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f66d406c2b0 0x7f66d406e770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e815ede0 0x7f66e80cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f66e80cba20 0x7f66e8153fe0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.651 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 --2- 172.21.15.2:0/1941666995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f66e80cb640 0x7f66e81f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.648+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 >> 172.21.15.2:0/1941666995 conn(0x7f66e8067950 msgr2=0x7f66e8067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:15.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.649+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 shutdown_connections 2024-09-19T05:45:15.652 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.649+0000 7f66edfe4640 1 -- 172.21.15.2:0/1941666995 wait complete. 2024-09-19T05:45:15.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-19T05:45:15.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.775+0000 7fe21359e640 1 Processor -- start 2024-09-19T05:45:15.777 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.775+0000 7fe21359e640 1 -- start start 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21359e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe214154a20 0x7fe2140cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21359e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe214154e00 0x7fe2140cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21359e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe21415ad80 con 0x7fe21415a8d0 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21359e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe2140ccbe0 con 0x7fe214154a20 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21359e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe21406c9a0 con 0x7fe214154e00 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe214154a20 0x7fe2140cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe214154a20 0x7fe2140cb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53444/0 (socket says 172.21.15.2:53444) 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21259c640 1 -- 172.21.15.2:0/1115152002 learned_addr learned my addr 172.21.15.2:0/1115152002 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe211d9b640 1 --2- 172.21.15.2:0/1115152002 >> v2:172.21.15.123:3300/0 conn(0x7fe214154e00 0x7fe2140cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21259c640 1 -- 172.21.15.2:0/1115152002 >> v2:172.21.15.123:3300/0 conn(0x7fe214154e00 msgr2=0x7fe2140cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21259c640 1 --2- 172.21.15.2:0/1115152002 >> v2:172.21.15.123:3300/0 conn(0x7fe214154e00 0x7fe2140cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21259c640 1 -- 172.21.15.2:0/1115152002 >> v1:172.21.15.123:6789/0 conn(0x7fe21415a8d0 legacy=0x7fe2140ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:15.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe21259c640 1 -- 172.21.15.2:0/1115152002 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21415b060 con 0x7fe214154a20 2024-09-19T05:45:15.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.776+0000 7fe211d9b640 1 --2- 172.21.15.2:0/1115152002 >> v2:172.21.15.123:3300/0 conn(0x7fe214154e00 0x7fe2140cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:15.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.777+0000 7fe21259c640 1 --2- 172.21.15.2:0/1115152002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe214154a20 0x7fe2140cb0e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe2140b2d80 tx=0x7fe20402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c818be070067e0c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:15.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.777+0000 7fe21159a640 1 -- 172.21.15.2:0/1115152002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204037070 con 0x7fe214154a20 2024-09-19T05:45:15.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.777+0000 7fe21159a640 1 -- 172.21.15.2:0/1115152002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe204032440 con 0x7fe214154a20 2024-09-19T05:45:15.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.777+0000 7fe21159a640 1 -- 172.21.15.2:0/1115152002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204032810 con 0x7fe214154a20 2024-09-19T05:45:15.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 -- 172.21.15.2:0/1115152002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe214154a20 msgr2=0x7fe2140cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:15.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 --2- 172.21.15.2:0/1115152002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe214154a20 0x7fe2140cb0e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe2140b2d80 tx=0x7fe20402a230 comp rx=0 tx=0).stop 2024-09-19T05:45:15.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 -- 172.21.15.2:0/1115152002 shutdown_connections 2024-09-19T05:45:15.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 --2- 172.21.15.2:0/1115152002 >> v2:172.21.15.123:3300/0 conn(0x7fe214154e00 0x7fe2140cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 --2- 172.21.15.2:0/1115152002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe214154a20 0x7fe2140cb0e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 -- 172.21.15.2:0/1115152002 >> 172.21.15.2:0/1115152002 conn(0x7fe214067950 msgr2=0x7fe214067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:15.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 -- 172.21.15.2:0/1115152002 shutdown_connections 2024-09-19T05:45:15.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.778+0000 7fe21359e640 1 -- 172.21.15.2:0/1115152002 wait complete. 2024-09-19T05:45:15.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.779+0000 7fe21359e640 1 Processor -- start 2024-09-19T05:45:15.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.779+0000 7fe21359e640 1 -- start start 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe21359e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe214154a20 0x7fe2140b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe21359e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe214154e00 0x7fe2141469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe21359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21415a8d0 0x7fe21415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe21359e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe2140cd4f0 con 0x7fe21415a8d0 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe21359e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe21415b620 con 0x7fe214154a20 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe21359e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe2140cd670 con 0x7fe214154e00 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21415a8d0 0x7fe21415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:15.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21415a8d0 0x7fe21415c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53448/0 (socket says 172.21.15.2:53448) 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 -- 172.21.15.2:0/4176377141 learned_addr learned my addr 172.21.15.2:0/4176377141 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe21259c640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe214154a20 0x7fe2140b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe211d9b640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe214154e00 0x7fe2141469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 -- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe214154e00 msgr2=0x7fe2141469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe214154e00 0x7fe2141469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 -- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe214154a20 msgr2=0x7fe2140b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe214154a20 0x7fe2140b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.780+0000 7fe212d9d640 1 -- 172.21.15.2:0/4176377141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2140d0080 con 0x7fe21415a8d0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.781+0000 7fe212d9d640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21415a8d0 0x7fe21415c150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00be60 tx=0x7fe20c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.781+0000 7fe202ffd640 1 -- 172.21.15.2:0/4176377141 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c019070 con 0x7fe21415a8d0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.781+0000 7fe202ffd640 1 -- 172.21.15.2:0/4176377141 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe20c014440 con 0x7fe21415a8d0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.781+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2141f0a60 con 0x7fe21415a8d0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.781+0000 7fe202ffd640 1 -- 172.21.15.2:0/4176377141 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c014870 con 0x7fe21415a8d0 2024-09-19T05:45:15.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.781+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2140ce530 con 0x7fe21415a8d0 2024-09-19T05:45:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.782+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2140cfe60 con 0x7fe21415a8d0 2024-09-19T05:45:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.782+0000 7fe202ffd640 1 -- 172.21.15.2:0/4176377141 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe20c01c070 con 0x7fe21415a8d0 2024-09-19T05:45:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.783+0000 7fe202ffd640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe1f406c2b0 0x7fe1f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.783+0000 7fe202ffd640 1 -- 172.21.15.2:0/4176377141 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5799+0+0 (secure 0 0 0) 0x7fe20c08b440 con 0x7fe21415a8d0 2024-09-19T05:45:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.783+0000 7fe21259c640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe1f406c2b0 0x7fe1f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:15.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.784+0000 7fe21259c640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe1f406c2b0 0x7fe1f406e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe204034000 tx=0x7fe20402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:15.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.787+0000 7fe202ffd640 1 -- 172.21.15.2:0/4176377141 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe20c059500 con 0x7fe21415a8d0 2024-09-19T05:45:15.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:15.973+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7fe2140cbc40 con 0x7fe21415a8d0 2024-09-19T05:45:16.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.651+0000 7fe202ffd640 1 -- 172.21.15.2:0/4176377141 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v97) ==== 113+0+0 (secure 0 0 0) 0x7fe20c05d800 con 0x7fe21415a8d0 2024-09-19T05:45:16.653 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe1f406c2b0 msgr2=0x7fe1f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe1f406c2b0 0x7fe1f406e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe204034000 tx=0x7fe20402e000 comp rx=0 tx=0).stop 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21415a8d0 msgr2=0x7fe21415c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21415a8d0 0x7fe21415c150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00be60 tx=0x7fe20c00bf50 comp rx=0 tx=0).stop 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 shutdown_connections 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe1f406c2b0 0x7fe1f406e770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe21415a8d0 0x7fe21415c150 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe214154e00 0x7fe2141469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 --2- 172.21.15.2:0/4176377141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe214154a20 0x7fe2140b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 >> 172.21.15.2:0/4176377141 conn(0x7fe214067950 msgr2=0x7fe21414a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:16.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 shutdown_connections 2024-09-19T05:45:16.658 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.655+0000 7fe21359e640 1 -- 172.21.15.2:0/4176377141 wait complete. 2024-09-19T05:45:16.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-19T05:45:16.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.774+0000 7f7280477640 1 Processor -- start 2024-09-19T05:45:16.776 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.774+0000 7f7280477640 1 -- start start 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f7280477640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7278154dd0 0x7f72780cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f7280477640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f727815e200 0x7f727815e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f7280477640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72780cb9b0 con 0x7f72780ca940 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f7280477640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f727815b920 con 0x7f7278154dd0 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f7280477640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f727815eb70 con 0x7f727815e200 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f727e213640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7278154dd0 0x7f72780cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f727e213640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7278154dd0 0x7f72780cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53464/0 (socket says 172.21.15.2:53464) 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f727e213640 1 -- 172.21.15.2:0/2808803151 learned_addr learned my addr 172.21.15.2:0/2808803151 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.775+0000 7f727ea14640 1 --2- 172.21.15.2:0/2808803151 >> v2:172.21.15.123:3300/0 conn(0x7f727815e200 0x7f727815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:16.778 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727e213640 1 -- 172.21.15.2:0/2808803151 >> v2:172.21.15.123:3300/0 conn(0x7f727815e200 msgr2=0x7f727815e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:16.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727e213640 1 --2- 172.21.15.2:0/2808803151 >> v2:172.21.15.123:3300/0 conn(0x7f727815e200 0x7f727815e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727e213640 1 -- 172.21.15.2:0/2808803151 >> v1:172.21.15.123:6789/0 conn(0x7f72780ca940 legacy=0x7f72780718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:16.779 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727e213640 1 -- 172.21.15.2:0/2808803151 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72780cbba0 con 0x7f7278154dd0 2024-09-19T05:45:16.780 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727e213640 1 --2- 172.21.15.2:0/2808803151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7278154dd0 0x7f72780cb390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f727806eec0 tx=0x7f726802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44adfc02e2416d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727d211640 1 -- 172.21.15.2:0/2808803151 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268036070 con 0x7f7278154dd0 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727d211640 1 -- 172.21.15.2:0/2808803151 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7268031600 con 0x7f7278154dd0 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.776+0000 7f727d211640 1 -- 172.21.15.2:0/2808803151 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680319f0 con 0x7f7278154dd0 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 -- 172.21.15.2:0/2808803151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7278154dd0 msgr2=0x7f72780cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 --2- 172.21.15.2:0/2808803151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7278154dd0 0x7f72780cb390 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f727806eec0 tx=0x7f726802a7c0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 -- 172.21.15.2:0/2808803151 shutdown_connections 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 --2- 172.21.15.2:0/2808803151 >> v2:172.21.15.123:3300/0 conn(0x7f727815e200 0x7f727815e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 --2- 172.21.15.2:0/2808803151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7278154dd0 0x7f72780cb390 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.781 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 -- 172.21.15.2:0/2808803151 >> 172.21.15.2:0/2808803151 conn(0x7f727806bdf0 msgr2=0x7f727806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:16.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 -- 172.21.15.2:0/2808803151 shutdown_connections 2024-09-19T05:45:16.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.777+0000 7f7280477640 1 -- 172.21.15.2:0/2808803151 wait complete. 2024-09-19T05:45:16.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.778+0000 7f7280477640 1 Processor -- start 2024-09-19T05:45:16.782 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.778+0000 7f7280477640 1 -- start start 2024-09-19T05:45:16.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f7280477640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72780ca940 0x7f72781f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:16.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f7280477640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7278154dd0 0x7f72781f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:16.785 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f7280477640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f727815e200 0x7f727815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f7280477640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f727815b2d0 con 0x7f72780ca940 2024-09-19T05:45:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f7280477640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f727815aff0 con 0x7f7278154dd0 2024-09-19T05:45:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f7280477640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f727815b5b0 con 0x7f727815e200 2024-09-19T05:45:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72780ca940 0x7f72781f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72780ca940 0x7f72781f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53466/0 (socket says 172.21.15.2:53466) 2024-09-19T05:45:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 -- 172.21.15.2:0/1346394085 learned_addr learned my addr 172.21.15.2:0/1346394085 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727da12640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7278154dd0 0x7f72781f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727ea14640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f727815e200 0x7f727815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 -- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f727815e200 msgr2=0x7f727815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f727815e200 0x7f727815ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 -- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7278154dd0 msgr2=0x7f72781f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7278154dd0 0x7f72781f4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727e213640 1 -- 172.21.15.2:0/1346394085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727815a9f0 con 0x7f72780ca940 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.779+0000 7f727da12640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7278154dd0 0x7f72781f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.780+0000 7f727e213640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72780ca940 0x7f72781f1630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f726802adb0 tx=0x7f7268004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.780+0000 7f7276ffd640 1 -- 172.21.15.2:0/1346394085 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268036050 con 0x7f72780ca940 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.780+0000 7f7276ffd640 1 -- 172.21.15.2:0/1346394085 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7268042070 con 0x7f72780ca940 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.780+0000 7f7276ffd640 1 -- 172.21.15.2:0/1346394085 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268009510 con 0x7f72780ca940 2024-09-19T05:45:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.780+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727814e750 con 0x7f72780ca940 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.780+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72780ce5b0 con 0x7f72780ca940 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.781+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727814ee50 con 0x7f72780ca940 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.781+0000 7f7276ffd640 1 -- 172.21.15.2:0/1346394085 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7268004470 con 0x7f72780ca940 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.782+0000 7f7276ffd640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f726406c2b0 0x7f726406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.782+0000 7f7276ffd640 1 -- 172.21.15.2:0/1346394085 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5799+0+0 (secure 0 0 0) 0x7f72680ac340 con 0x7f72780ca940 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.782+0000 7f727da12640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f726406c2b0 0x7f726406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.783+0000 7f727da12640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f726406c2b0 0x7f726406e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f72781ee7e0 tx=0x7f726c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:16.788 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.786+0000 7f7276ffd640 1 -- 172.21.15.2:0/1346394085 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f727814ee50 con 0x7f72780ca940 2024-09-19T05:45:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:16.973+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) -- 0x7f7278153f20 con 0x7f72780ca940 2024-09-19T05:45:17.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.657+0000 7f7276ffd640 1 -- 172.21.15.2:0/1346394085 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v98) ==== 133+0+0 (secure 0 0 0) 0x7f7278153f20 con 0x7f72780ca940 2024-09-19T05:45:17.660 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-19T05:45:17.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f726406c2b0 msgr2=0x7f726406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:17.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f726406c2b0 0x7f726406e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f72781ee7e0 tx=0x7f726c009040 comp rx=0 tx=0).stop 2024-09-19T05:45:17.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72780ca940 msgr2=0x7f72781f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:17.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72780ca940 0x7f72781f1630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f726802adb0 tx=0x7f7268004a40 comp rx=0 tx=0).stop 2024-09-19T05:45:17.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 shutdown_connections 2024-09-19T05:45:17.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f726406c2b0 0x7f726406e770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.663 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7278154dd0 0x7f72781f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f727815e200 0x7f727815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 --2- 172.21.15.2:0/1346394085 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72780ca940 0x7f72781f1630 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 >> 172.21.15.2:0/1346394085 conn(0x7f727806bdf0 msgr2=0x7f72781530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:17.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 shutdown_connections 2024-09-19T05:45:17.664 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.661+0000 7f7280477640 1 -- 172.21.15.2:0/1346394085 wait complete. 2024-09-19T05:45:17.677 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-19T05:45:17.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.786+0000 7f686225a640 1 Processor -- start 2024-09-19T05:45:17.789 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.786+0000 7f686225a640 1 -- start start 2024-09-19T05:45:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f686225a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f685c155000 0x7f685c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f686225a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c0caf40 0x7f685c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f686225a640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685c0ccbc0 con 0x7f685c154c20 2024-09-19T05:45:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f686225a640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f685c0cc600 con 0x7f685c155000 2024-09-19T05:45:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f686225a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f685c0cc8e0 con 0x7f685c0caf40 2024-09-19T05:45:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f685bfff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c0caf40 0x7f685c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:17.790 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f685affd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f685c155000 0x7f685c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f685affd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f685c155000 0x7f685c0cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36588/0 (socket says 172.21.15.2:36588) 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f685b7fe640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f685c154c20 0x7f685c073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:56132/0 (socket says 172.21.15.2:56132) 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f685bfff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c0caf40 0x7f685c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:33116/0 (socket says 172.21.15.2:33116) 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.787+0000 7f685affd640 1 -- 172.21.15.2:0/4046232321 learned_addr learned my addr 172.21.15.2:0/4046232321 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.788+0000 7f685affd640 1 -- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c0caf40 msgr2=0x7f685c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.788+0000 7f685affd640 1 --2- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c0caf40 0x7f685c15aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.788+0000 7f685affd640 1 -- 172.21.15.2:0/4046232321 >> v1:172.21.15.123:6789/0 conn(0x7f685c154c20 legacy=0x7f685c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.788+0000 7f685affd640 1 -- 172.21.15.2:0/4046232321 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c0cbc60 con 0x7f685c155000 2024-09-19T05:45:17.791 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.788+0000 7f685bfff640 1 --2- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c0caf40 0x7f685c15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f685affd640 1 --2- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 0x7f685c0cd9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6850009850 tx=0x7f685002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56db9fc57e3f9f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f685a7fc640 1 -- 172.21.15.2:0/4046232321 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850004030 con 0x7f685c155000 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f685a7fc640 1 -- 172.21.15.2:0/4046232321 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68500041d0 con 0x7f685c155000 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f685a7fc640 1 -- 172.21.15.2:0/4046232321 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850003610 con 0x7f685c155000 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f686225a640 1 -- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 msgr2=0x7f685c0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f686225a640 1 --2- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 0x7f685c0cd9e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6850009850 tx=0x7f685002a230 comp rx=0 tx=0).stop 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f686225a640 1 -- 172.21.15.2:0/4046232321 shutdown_connections 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f686225a640 1 --2- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c0caf40 0x7f685c15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f686225a640 1 --2- 172.21.15.2:0/4046232321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 0x7f685c0cd9e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f686225a640 1 -- 172.21.15.2:0/4046232321 >> 172.21.15.2:0/4046232321 conn(0x7f685c06d0e0 msgr2=0x7f685c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.789+0000 7f686225a640 1 -- 172.21.15.2:0/4046232321 shutdown_connections 2024-09-19T05:45:17.793 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.790+0000 7f686225a640 1 -- 172.21.15.2:0/4046232321 wait complete. 2024-09-19T05:45:17.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.791+0000 7f686225a640 1 Processor -- start 2024-09-19T05:45:17.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.791+0000 7f686225a640 1 -- start start 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.791+0000 7f686225a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f685c0caf40 0x7f685c151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.791+0000 7f686225a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c154c20 0x7f685c1e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.791+0000 7f686225a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 0x7f685c0b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f686225a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f685c073a50 con 0x7f685c0caf40 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f686225a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f685c0cb430 con 0x7f685c155000 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f686225a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f685c073bd0 con 0x7f685c154c20 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f685c0caf40 0x7f685c151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f685c0caf40 0x7f685c151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53468/0 (socket says 172.21.15.2:53468) 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 -- 172.21.15.2:0/3253353092 learned_addr learned my addr 172.21.15.2:0/3253353092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685bfff640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 0x7f685c0b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:17.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685affd640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c154c20 0x7f685c1e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 -- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c154c20 msgr2=0x7f685c1e0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c154c20 0x7f685c1e0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 -- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 msgr2=0x7f685c0b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 0x7f685c0b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 -- 172.21.15.2:0/3253353092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c1f0c60 con 0x7f685c0caf40 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685affd640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c154c20 0x7f685c1e0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f685b7fe640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f685c0caf40 0x7f685c151440 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c430 tx=0x7f684c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f6843fff640 1 -- 172.21.15.2:0/3253353092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c0074e0 con 0x7f685c0caf40 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f6843fff640 1 -- 172.21.15.2:0/3253353092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684c007680 con 0x7f685c0caf40 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f6843fff640 1 -- 172.21.15.2:0/3253353092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c004a00 con 0x7f685c0caf40 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c1f0f40 con 0x7f685c0caf40 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.792+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c1f1500 con 0x7f685c0caf40 2024-09-19T05:45:17.797 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.793+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c0c5770 con 0x7f685c0caf40 2024-09-19T05:45:17.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.794+0000 7f6843fff640 1 -- 172.21.15.2:0/3253353092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f684c01a020 con 0x7f685c0caf40 2024-09-19T05:45:17.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.794+0000 7f6843fff640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f684806c2b0 0x7f684806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:17.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.795+0000 7f6843fff640 1 -- 172.21.15.2:0/3253353092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5799+0+0 (secure 0 0 0) 0x7f684c08aac0 con 0x7f685c0caf40 2024-09-19T05:45:17.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.795+0000 7f685affd640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f684806c2b0 0x7f684806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:17.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.795+0000 7f685affd640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f684806c2b0 0x7f684806e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6850033000 tx=0x7f685002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:17.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.799+0000 7f6843fff640 1 -- 172.21.15.2:0/3253353092 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f684c059260 con 0x7f685c0caf40 2024-09-19T05:45:17.991 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:17.989+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7f685c1e4be0 con 0x7f685c0caf40 2024-09-19T05:45:18.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.663+0000 7f6843fff640 1 -- 172.21.15.2:0/3253353092 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v99) ==== 113+0+0 (secure 0 0 0) 0x7f684c058b60 con 0x7f685c0caf40 2024-09-19T05:45:18.666 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-19T05:45:18.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f684806c2b0 msgr2=0x7f684806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:18.669 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f684806c2b0 0x7f684806e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6850033000 tx=0x7f685002e000 comp rx=0 tx=0).stop 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f685c0caf40 msgr2=0x7f685c151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f685c0caf40 0x7f685c151440 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c430 tx=0x7f684c00c830 comp rx=0 tx=0).stop 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 shutdown_connections 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f684806c2b0 0x7f684806e770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c155000 0x7f685c0b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f685c154c20 0x7f685c1e0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 --2- 172.21.15.2:0/3253353092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f685c0caf40 0x7f685c151440 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 >> 172.21.15.2:0/3253353092 conn(0x7f685c06d0e0 msgr2=0x7f685c153000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:18.670 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 shutdown_connections 2024-09-19T05:45:18.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.667+0000 7f686225a640 1 -- 172.21.15.2:0/3253353092 wait complete. 2024-09-19T05:45:18.683 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-19T05:45:18.794 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.792+0000 7fea97e8c640 1 Processor -- start 2024-09-19T05:45:18.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.792+0000 7fea97e8c640 1 -- start start 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.793+0000 7fea97e8c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea90155000 0x7fea900cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.793+0000 7fea97e8c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea900caf40 0x7fea9015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.793+0000 7fea97e8c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea900ccbc0 con 0x7fea90154c20 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.793+0000 7fea97e8c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fea900cc600 con 0x7fea90155000 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.793+0000 7fea97e8c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fea900cc8e0 con 0x7fea900caf40 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea95427640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea90155000 0x7fea900cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea96429640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea900caf40 0x7fea9015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea95427640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea90155000 0x7fea900cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36608/0 (socket says 172.21.15.2:36608) 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea96429640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea900caf40 0x7fea9015aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:33130/0 (socket says 172.21.15.2:33130) 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea95427640 1 -- 172.21.15.2:0/3430015305 learned_addr learned my addr 172.21.15.2:0/3430015305 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea95c28640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fea90154c20 0x7fea90073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:56146/0 (socket says 172.21.15.2:56146) 2024-09-19T05:45:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea94c26640 1 -- 172.21.15.2:0/3430015305 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014956449 0 0) 0x7fea900ccbc0 con 0x7fea90154c20 2024-09-19T05:45:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea94c26640 1 -- 172.21.15.2:0/3430015305 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea78003520 con 0x7fea90154c20 2024-09-19T05:45:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea96429640 1 -- 172.21.15.2:0/3430015305 >> v2:172.21.15.123:3300/0 conn(0x7fea90155000 msgr2=0x7fea900cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea96429640 1 --2- 172.21.15.2:0/3430015305 >> v2:172.21.15.123:3300/0 conn(0x7fea90155000 0x7fea900cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea96429640 1 -- 172.21.15.2:0/3430015305 >> v1:172.21.15.123:6789/0 conn(0x7fea90154c20 legacy=0x7fea90073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea96429640 1 -- 172.21.15.2:0/3430015305 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea900cbc60 con 0x7fea900caf40 2024-09-19T05:45:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.794+0000 7fea95427640 1 --2- 172.21.15.2:0/3430015305 >> v2:172.21.15.123:3300/0 conn(0x7fea90155000 0x7fea900cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.795+0000 7fea96429640 1 --2- 172.21.15.2:0/3430015305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea900caf40 0x7fea9015aae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea80009d90 tx=0x7fea8002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908206674edf3031 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.795+0000 7fea94c26640 1 -- 172.21.15.2:0/3430015305 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80007740 con 0x7fea900caf40 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.795+0000 7fea94c26640 1 -- 172.21.15.2:0/3430015305 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea800078e0 con 0x7fea900caf40 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.795+0000 7fea94c26640 1 -- 172.21.15.2:0/3430015305 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80007c90 con 0x7fea900caf40 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 -- 172.21.15.2:0/3430015305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea900caf40 msgr2=0x7fea9015aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3430015305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea900caf40 0x7fea9015aae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fea80009d90 tx=0x7fea8002a660 comp rx=0 tx=0).stop 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 -- 172.21.15.2:0/3430015305 shutdown_connections 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3430015305 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea900caf40 0x7fea9015aae0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3430015305 >> v2:172.21.15.123:3300/0 conn(0x7fea90155000 0x7fea900cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 -- 172.21.15.2:0/3430015305 >> 172.21.15.2:0/3430015305 conn(0x7fea9006d0e0 msgr2=0x7fea9006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 -- 172.21.15.2:0/3430015305 shutdown_connections 2024-09-19T05:45:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.796+0000 7fea97e8c640 1 -- 172.21.15.2:0/3430015305 wait complete. 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.797+0000 7fea97e8c640 1 Processor -- start 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.797+0000 7fea97e8c640 1 -- start start 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea97e8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea900caf40 0x7fea9014ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea97e8c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea90154c20 0x7fea901e0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea97e8c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea90155000 0x7fea901e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea97e8c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fea90073a50 con 0x7fea900caf40 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea97e8c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea900cb430 con 0x7fea90155000 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea97e8c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fea90073bd0 con 0x7fea90154c20 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea900caf40 0x7fea9014ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea900caf40 0x7fea9014ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53480/0 (socket says 172.21.15.2:53480) 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 -- 172.21.15.2:0/3429399016 learned_addr learned my addr 172.21.15.2:0/3429399016 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea96429640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea90155000 0x7fea901e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95427640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea90154c20 0x7fea901e0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 -- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea90154c20 msgr2=0x7fea901e0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea90154c20 0x7fea901e0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 -- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea90155000 msgr2=0x7fea901e14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea90155000 0x7fea901e14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 -- 172.21.15.2:0/3429399016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea900b2ef0 con 0x7fea900caf40 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.798+0000 7fea95c28640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea900caf40 0x7fea9014ad40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c430 tx=0x7fea8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.799+0000 7fea8e7fc640 1 -- 172.21.15.2:0/3429399016 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880074e0 con 0x7fea900caf40 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.799+0000 7fea8e7fc640 1 -- 172.21.15.2:0/3429399016 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea880047e0 con 0x7fea900caf40 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.799+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea900b5630 con 0x7fea900caf40 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.799+0000 7fea8e7fc640 1 -- 172.21.15.2:0/3429399016 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88004bd0 con 0x7fea900caf40 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.799+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901e2e50 con 0x7fea900caf40 2024-09-19T05:45:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.800+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea900b5ff0 con 0x7fea900caf40 2024-09-19T05:45:18.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.800+0000 7fea8e7fc640 1 -- 172.21.15.2:0/3429399016 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fea8801a020 con 0x7fea900caf40 2024-09-19T05:45:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.801+0000 7fea8e7fc640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fea7c06c2e0 0x7fea7c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.801+0000 7fea8e7fc640 1 -- 172.21.15.2:0/3429399016 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5799+0+0 (secure 0 0 0) 0x7fea8808ab50 con 0x7fea900caf40 2024-09-19T05:45:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.801+0000 7fea95427640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fea7c06c2e0 0x7fea7c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:18.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.801+0000 7fea95427640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fea7c06c2e0 0x7fea7c06e7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea90152ff0 tx=0x7fea84009210 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:18.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.805+0000 7fea8e7fc640 1 -- 172.21.15.2:0/3429399016 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fea88058ce0 con 0x7fea900caf40 2024-09-19T05:45:18.995 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:18.993+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) -- 0x7fea900c5770 con 0x7fea900caf40 2024-09-19T05:45:19.671 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.669+0000 7fea8e7fc640 1 -- 172.21.15.2:0/3429399016 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v100) ==== 133+0+0 (secure 0 0 0) 0x7fea8805cfe0 con 0x7fea900caf40 2024-09-19T05:45:19.672 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-19T05:45:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fea7c06c2e0 msgr2=0x7fea7c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fea7c06c2e0 0x7fea7c06e7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea90152ff0 tx=0x7fea84009210 comp rx=0 tx=0).stop 2024-09-19T05:45:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea900caf40 msgr2=0x7fea9014ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea900caf40 0x7fea9014ad40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fea8800c430 tx=0x7fea8800c8f0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 shutdown_connections 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fea7c06c2e0 0x7fea7c06e7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea90155000 0x7fea901e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea90154c20 0x7fea901e0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 --2- 172.21.15.2:0/3429399016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea900caf40 0x7fea9014ad40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 >> 172.21.15.2:0/3429399016 conn(0x7fea9006d0e0 msgr2=0x7fea90067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 shutdown_connections 2024-09-19T05:45:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.673+0000 7fea97e8c640 1 -- 172.21.15.2:0/3429399016 wait complete. 2024-09-19T05:45:19.689 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-19T05:45:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.798+0000 7f40c1d8d640 1 Processor -- start 2024-09-19T05:45:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.798+0000 7f40c1d8d640 1 -- start start 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40c1d8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc15c150 0x7f40bc0c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40c1d8d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40bc0c3a60 con 0x7f40bc152b70 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40c1d8d640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40bc0c3d40 con 0x7f40bc15c530 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40c1d8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc0c4c00 con 0x7f40bc15c150 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40bbfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f40bc152b70 0x7f40bc06f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:52784/0 (socket says 172.21.15.2:52784) 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40bbfff640 1 -- 172.21.15.2:0/3942993372 learned_addr learned my addr 172.21.15.2:0/3942993372 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40bb7fe640 1 --2- 172.21.15.2:0/3942993372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc15c150 0x7f40bc0c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:19.802 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770313487 0 0) 0x7f40bc0c3a60 con 0x7f40bc152b70 2024-09-19T05:45:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.799+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40a4003520 con 0x7f40bc152b70 2024-09-19T05:45:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.800+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326603727 0 0) 0x7f40bc0c3d40 con 0x7f40bc15c530 2024-09-19T05:45:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.800+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40bc0c3a60 con 0x7f40bc15c530 2024-09-19T05:45:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.800+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2833869294 0 0) 0x7f40a4003520 con 0x7f40bc152b70 2024-09-19T05:45:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.800+0000 7f40bb7fe640 1 -- 172.21.15.2:0/3942993372 >> v1:172.21.15.123:6789/0 conn(0x7f40bc15c530 legacy=0x7f40bc152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.800+0000 7f40bb7fe640 1 -- 172.21.15.2:0/3942993372 >> v1:172.21.15.2:6789/0 conn(0x7f40bc152b70 legacy=0x7f40bc06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:19.803 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.800+0000 7f40bb7fe640 1 -- 172.21.15.2:0/3942993372 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc0c4ee0 con 0x7f40bc15c150 2024-09-19T05:45:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.800+0000 7f40bb7fe640 1 --2- 172.21.15.2:0/3942993372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc15c150 0x7f40bc0c6e90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0097d0 tx=0x7f40ac02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0797c1a8da3278d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:19.804 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac037070 con 0x7f40bc15c150 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ac02ac30 con 0x7f40bc15c150 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40ba7fc640 1 -- 172.21.15.2:0/3942993372 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b40019f0 con 0x7f40bc15c150 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40c1d8d640 1 -- 172.21.15.2:0/3942993372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc15c150 msgr2=0x7f40bc0c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/3942993372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc15c150 0x7f40bc0c6e90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0097d0 tx=0x7f40ac02a1a0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40c1d8d640 1 -- 172.21.15.2:0/3942993372 shutdown_connections 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/3942993372 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc15c150 0x7f40bc0c6e90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40c1d8d640 1 -- 172.21.15.2:0/3942993372 >> 172.21.15.2:0/3942993372 conn(0x7f40bc06bc10 msgr2=0x7f40bc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40c1d8d640 1 -- 172.21.15.2:0/3942993372 shutdown_connections 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.801+0000 7f40c1d8d640 1 -- 172.21.15.2:0/3942993372 wait complete. 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.802+0000 7f40c1d8d640 1 Processor -- start 2024-09-19T05:45:19.805 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40c1d8d640 1 -- start start 2024-09-19T05:45:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40c1d8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc152b70 0x7f40bc072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40c1d8d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40bc15c150 0x7f40bc0730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40c1d8d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40bc15c530 0x7f40bc1f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40c1d8d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc0c7570 con 0x7f40bc15c530 2024-09-19T05:45:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40c1d8d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc0c51c0 con 0x7f40bc15c150 2024-09-19T05:45:19.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40c1d8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc0c76f0 con 0x7f40bc152b70 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40bbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40bc15c530 0x7f40bc1f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40bbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40bc15c530 0x7f40bc1f08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53494/0 (socket says 172.21.15.2:53494) 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40bbfff640 1 -- 172.21.15.2:0/4238495462 learned_addr learned my addr 172.21.15.2:0/4238495462 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40baffd640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40bc15c150 0x7f40bc0730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.803+0000 7f40bb7fe640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc152b70 0x7f40bc072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40bbfff640 1 -- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc152b70 msgr2=0x7f40bc072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40bbfff640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc152b70 0x7f40bc072b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40bbfff640 1 -- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40bc15c150 msgr2=0x7f40bc0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40bbfff640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40bc15c150 0x7f40bc0730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40bbfff640 1 -- 172.21.15.2:0/4238495462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc0736d0 con 0x7f40bc15c530 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40bbfff640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40bc15c530 0x7f40bc1f08f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f40b4001350 tx=0x7f40b400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40a3fff640 1 -- 172.21.15.2:0/4238495462 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4017070 con 0x7f40bc15c530 2024-09-19T05:45:19.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40a3fff640 1 -- 172.21.15.2:0/4238495462 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b400ce20 con 0x7f40bc15c530 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc1582f0 con 0x7f40bc15c530 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40a3fff640 1 -- 172.21.15.2:0/4238495462 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4012640 con 0x7f40bc15c530 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.804+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc1493a0 con 0x7f40bc15c530 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.805+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40bc1f05c0 con 0x7f40bc15c530 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.806+0000 7f40a3fff640 1 -- 172.21.15.2:0/4238495462 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f40b401a070 con 0x7f40bc15c530 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.806+0000 7f40a3fff640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f40a806c2b0 0x7f40a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.806+0000 7f40a3fff640 1 -- 172.21.15.2:0/4238495462 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5799+0+0 (secure 0 0 0) 0x7f40b408b380 con 0x7f40bc15c530 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.806+0000 7f40bb7fe640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f40a806c2b0 0x7f40a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:19.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.807+0000 7f40bb7fe640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f40a806c2b0 0x7f40a806e770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40ac034000 tx=0x7f40ac02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:19.812 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.810+0000 7f40a3fff640 1 -- 172.21.15.2:0/4238495462 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f40b40593b0 con 0x7f40bc15c530 2024-09-19T05:45:19.999 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:19.997+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7f40bc0c6e90 con 0x7f40bc15c530 2024-09-19T05:45:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.675+0000 7f40a3fff640 1 -- 172.21.15.2:0/4238495462 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v101) ==== 113+0+0 (secure 0 0 0) 0x7f40b405d6b0 con 0x7f40bc15c530 2024-09-19T05:45:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-19T05:45:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.679+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f40a806c2b0 msgr2=0x7f40a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.679+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f40a806c2b0 0x7f40a806e770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40ac034000 tx=0x7f40ac02e000 comp rx=0 tx=0).stop 2024-09-19T05:45:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.679+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40bc15c530 msgr2=0x7f40bc1f08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.679+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40bc15c530 0x7f40bc1f08f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f40b4001350 tx=0x7f40b400a430 comp rx=0 tx=0).stop 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.679+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 shutdown_connections 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.680+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f40a806c2b0 0x7f40a806e770 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.680+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40bc15c150 0x7f40bc0730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.680+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40bc15c530 0x7f40bc1f08f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.680+0000 7f40c1d8d640 1 --2- 172.21.15.2:0/4238495462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40bc152b70 0x7f40bc072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.680+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 >> 172.21.15.2:0/4238495462 conn(0x7f40bc06bc10 msgr2=0x7f40bc150d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.680+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 shutdown_connections 2024-09-19T05:45:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.680+0000 7f40c1d8d640 1 -- 172.21.15.2:0/4238495462 wait complete. 2024-09-19T05:45:20.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-19T05:45:20.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.806+0000 7f5c3b52a640 1 Processor -- start 2024-09-19T05:45:20.809 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.806+0000 7f5c3b52a640 1 -- start start 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c3b52a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5c340cb640 0x7f5c340c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c3b52a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c340cba20 0x7f5c341549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c3b52a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c3415ede0 0x7f5c34154f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c3b52a640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f5c340c5ef0 con 0x7f5c3415ede0 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c3b52a640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5c3415ab40 con 0x7f5c340cb640 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c3b52a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3415ade0 con 0x7f5c340cba20 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c39ac7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c3415ede0 0x7f5c34154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:20.810 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c39ac7640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5c3415ede0 0x7f5c34154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53498/0 (socket says 172.21.15.2:53498) 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c39ac7640 1 -- 172.21.15.2:0/887416239 learned_addr learned my addr 172.21.15.2:0/887416239 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c392c6640 1 --2- 172.21.15.2:0/887416239 >> v2:172.21.15.123:3300/0 conn(0x7f5c340cb640 0x7f5c340c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.807+0000 7f5c38ac5640 1 --2- 172.21.15.2:0/887416239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c340cba20 0x7f5c341549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.808+0000 7f5c39ac7640 1 -- 172.21.15.2:0/887416239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c340cba20 msgr2=0x7f5c341549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.808+0000 7f5c39ac7640 1 --2- 172.21.15.2:0/887416239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c340cba20 0x7f5c341549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.808+0000 7f5c39ac7640 1 -- 172.21.15.2:0/887416239 >> v2:172.21.15.123:3300/0 conn(0x7f5c340cb640 msgr2=0x7f5c340c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.808+0000 7f5c39ac7640 1 --2- 172.21.15.2:0/887416239 >> v2:172.21.15.123:3300/0 conn(0x7f5c340cb640 0x7f5c340c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.811 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.808+0000 7f5c39ac7640 1 -- 172.21.15.2:0/887416239 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3415b0c0 con 0x7f5c3415ede0 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.808+0000 7f5c39ac7640 1 --2- 172.21.15.2:0/887416239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3415ede0 0x7f5c34154f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5c34073fd0 tx=0x7f5c2c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b020a3119ce3d969 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c2bfff640 1 -- 172.21.15.2:0/887416239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c036070 con 0x7f5c3415ede0 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c2bfff640 1 -- 172.21.15.2:0/887416239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c031600 con 0x7f5c3415ede0 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c2bfff640 1 -- 172.21.15.2:0/887416239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0319d0 con 0x7f5c3415ede0 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 -- 172.21.15.2:0/887416239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3415ede0 msgr2=0x7f5c34154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/887416239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3415ede0 0x7f5c34154f20 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5c34073fd0 tx=0x7f5c2c02a730 comp rx=0 tx=0).stop 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 -- 172.21.15.2:0/887416239 shutdown_connections 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/887416239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c3415ede0 0x7f5c34154f20 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/887416239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c340cba20 0x7f5c341549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/887416239 >> v2:172.21.15.123:3300/0 conn(0x7f5c340cb640 0x7f5c340c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.813 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 -- 172.21.15.2:0/887416239 >> 172.21.15.2:0/887416239 conn(0x7f5c34067950 msgr2=0x7f5c34067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:20.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 -- 172.21.15.2:0/887416239 shutdown_connections 2024-09-19T05:45:20.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.809+0000 7f5c3b52a640 1 -- 172.21.15.2:0/887416239 wait complete. 2024-09-19T05:45:20.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 Processor -- start 2024-09-19T05:45:20.814 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 -- start start 2024-09-19T05:45:20.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c340cb640 0x7f5c34153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:20.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c340cba20 0x7f5c341526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3415ede0 0x7f5c3415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3415b960 con 0x7f5c340cb640 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3415b680 con 0x7f5c340cba20 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c3b52a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5c340c5a00 con 0x7f5c3415ede0 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c392c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c340cb640 0x7f5c34153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c392c6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c340cb640 0x7f5c34153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53512/0 (socket says 172.21.15.2:53512) 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c392c6640 1 -- 172.21.15.2:0/886208521 learned_addr learned my addr 172.21.15.2:0/886208521 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.811+0000 7f5c38ac5640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c340cba20 0x7f5c341526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c39ac7640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3415ede0 0x7f5c3415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c392c6640 1 -- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3415ede0 msgr2=0x7f5c3415cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:20.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c392c6640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3415ede0 0x7f5c3415cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c392c6640 1 -- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c340cba20 msgr2=0x7f5c341526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c392c6640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c340cba20 0x7f5c341526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c392c6640 1 -- 172.21.15.2:0/886208521 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c340cdb30 con 0x7f5c340cb640 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c392c6640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c340cb640 0x7f5c34153ec0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000e8c0 tx=0x7f5c3000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c29ffb640 1 -- 172.21.15.2:0/886208521 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30017070 con 0x7f5c340cb640 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c29ffb640 1 -- 172.21.15.2:0/886208521 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3000c940 con 0x7f5c340cb640 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c340cfca0 con 0x7f5c340cb640 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c29ffb640 1 -- 172.21.15.2:0/886208521 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000cd10 con 0x7f5c340cb640 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.812+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c340d0260 con 0x7f5c340cb640 2024-09-19T05:45:20.818 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.813+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c340500d0 con 0x7f5c340cb640 2024-09-19T05:45:20.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.814+0000 7f5c29ffb640 1 -- 172.21.15.2:0/886208521 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5c3001a070 con 0x7f5c340cb640 2024-09-19T05:45:20.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.814+0000 7f5c29ffb640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1806c2b0 0x7f5c1806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:20.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.814+0000 7f5c29ffb640 1 -- 172.21.15.2:0/886208521 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5799+0+0 (secure 0 0 0) 0x7f5c3008b740 con 0x7f5c340cb640 2024-09-19T05:45:20.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.814+0000 7f5c38ac5640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1806c2b0 0x7f5c1806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:20.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.815+0000 7f5c38ac5640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1806c2b0 0x7f5c1806e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c340b4200 tx=0x7f5c24009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:20.821 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:20.819+0000 7f5c29ffb640 1 -- 172.21.15.2:0/886208521 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f5c340500d0 con 0x7f5c340cb640 2024-09-19T05:45:21.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.007+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7f5c34158d90 con 0x7f5c340cb640 2024-09-19T05:45:21.685 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.683+0000 7f5c29ffb640 1 -- 172.21.15.2:0/886208521 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v102) ==== 133+0+0 (secure 0 0 0) 0x7f5c34158d90 con 0x7f5c340cb640 2024-09-19T05:45:21.685 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-19T05:45:21.689 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1806c2b0 msgr2=0x7f5c1806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1806c2b0 0x7f5c1806e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c340b4200 tx=0x7f5c24009190 comp rx=0 tx=0).stop 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c340cb640 msgr2=0x7f5c34153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c340cb640 0x7f5c34153ec0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000e8c0 tx=0x7f5c3000ed80 comp rx=0 tx=0).stop 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 shutdown_connections 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f5c1806c2b0 0x7f5c1806e770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c340cba20 0x7f5c341526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5c3415ede0 0x7f5c3415cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 --2- 172.21.15.2:0/886208521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5c340cb640 0x7f5c34153ec0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 >> 172.21.15.2:0/886208521 conn(0x7f5c34067950 msgr2=0x7f5c340cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 shutdown_connections 2024-09-19T05:45:21.690 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.687+0000 7f5c3b52a640 1 -- 172.21.15.2:0/886208521 wait complete. 2024-09-19T05:45:21.703 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-19T05:45:21.703 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-19T05:45:21.704 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-19T05:45:21.704 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-19T05:45:21.704 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T05:45:21.704 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:45:21.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.813+0000 7f4482d1a640 1 Processor -- start 2024-09-19T05:45:21.815 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.813+0000 7f4482d1a640 1 -- start start 2024-09-19T05:45:21.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f4482d1a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f447c0cce60 0x7f447c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:21.816 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f4482d1a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c15afe0 0x7f447c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f4482d1a640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f447c0cd780 con 0x7f447c154e70 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f4482d1a640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f447c0cd260 con 0x7f447c0cce60 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f4482d1a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f447c155250 con 0x7f447c15afe0 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f447bfff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f447c0cce60 0x7f447c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f4480ab6640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f447c154e70 0x7f447c0cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54048/0 (socket says 172.21.15.2:54048) 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f4480ab6640 1 -- 172.21.15.2:0/3989265504 learned_addr learned my addr 172.21.15.2:0/3989265504 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f44812b7640 1 --2- 172.21.15.2:0/3989265504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c15afe0 0x7f447c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f447b7fe640 1 -- 172.21.15.2:0/3989265504 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 103403045 0 0) 0x7f447c0cd780 con 0x7f447c154e70 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f447b7fe640 1 -- 172.21.15.2:0/3989265504 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4464003520 con 0x7f447c154e70 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f447bfff640 1 -- 172.21.15.2:0/3989265504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c15afe0 msgr2=0x7f447c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f447bfff640 1 --2- 172.21.15.2:0/3989265504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c15afe0 0x7f447c0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f447bfff640 1 -- 172.21.15.2:0/3989265504 >> v1:172.21.15.2:6789/0 conn(0x7f447c154e70 legacy=0x7f447c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.814+0000 7f447bfff640 1 -- 172.21.15.2:0/3989265504 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c0cd4a0 con 0x7f447c0cce60 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.815+0000 7f447bfff640 1 --2- 172.21.15.2:0/3989265504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c0cce60 0x7f447c15aaa0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4470009dc0 tx=0x7f447002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ad5056ca3b467 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.815+0000 7f447b7fe640 1 -- 172.21.15.2:0/3989265504 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447002ae60 con 0x7f447c0cce60 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.815+0000 7f447b7fe640 1 -- 172.21.15.2:0/3989265504 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4470007740 con 0x7f447c0cce60 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.815+0000 7f447b7fe640 1 -- 172.21.15.2:0/3989265504 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470007b10 con 0x7f447c0cce60 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 -- 172.21.15.2:0/3989265504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c0cce60 msgr2=0x7f447c15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 --2- 172.21.15.2:0/3989265504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c0cce60 0x7f447c15aaa0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4470009dc0 tx=0x7f447002a680 comp rx=0 tx=0).stop 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 -- 172.21.15.2:0/3989265504 shutdown_connections 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 --2- 172.21.15.2:0/3989265504 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c15afe0 0x7f447c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.819 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 --2- 172.21.15.2:0/3989265504 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c0cce60 0x7f447c15aaa0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 -- 172.21.15.2:0/3989265504 >> 172.21.15.2:0/3989265504 conn(0x7f447c06da40 msgr2=0x7f447c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:21.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 -- 172.21.15.2:0/3989265504 shutdown_connections 2024-09-19T05:45:21.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.816+0000 7f4482d1a640 1 -- 172.21.15.2:0/3989265504 wait complete. 2024-09-19T05:45:21.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.817+0000 7f4482d1a640 1 Processor -- start 2024-09-19T05:45:21.820 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.817+0000 7f4482d1a640 1 -- start start 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f4482d1a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c0cce60 0x7f447c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f4482d1a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c154e70 0x7f447c1f18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f4482d1a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c15afe0 0x7f447c1f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f4482d1a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f447c15e670 con 0x7f447c15afe0 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f4482d1a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f447c15e390 con 0x7f447c0cce60 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f4482d1a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f447c15e950 con 0x7f447c154e70 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c15afe0 0x7f447c1f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c15afe0 0x7f447c1f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54344/0 (socket says 172.21.15.2:54344) 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 -- 172.21.15.2:0/867986578 learned_addr learned my addr 172.21.15.2:0/867986578 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f447bfff640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c154e70 0x7f447c1f18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:21.822 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f4480ab6640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c0cce60 0x7f447c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 -- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c154e70 msgr2=0x7f447c1f18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c154e70 0x7f447c1f18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 -- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c0cce60 msgr2=0x7f447c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c0cce60 0x7f447c15cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.818+0000 7f44812b7640 1 -- 172.21.15.2:0/867986578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c15dc20 con 0x7f447c15afe0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.819+0000 7f44812b7640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c15afe0 0x7f447c1f1e00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f447400d600 tx=0x7f447400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.819+0000 7f44797fa640 1 -- 172.21.15.2:0/867986578 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44740125f0 con 0x7f447c15afe0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.819+0000 7f44797fa640 1 -- 172.21.15.2:0/867986578 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4474019070 con 0x7f447c15afe0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.819+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c155fc0 con 0x7f447c15afe0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.819+0000 7f44797fa640 1 -- 172.21.15.2:0/867986578 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44740129d0 con 0x7f447c15afe0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.819+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c156580 con 0x7f447c15afe0 2024-09-19T05:45:21.823 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.820+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c15ee70 con 0x7f447c15afe0 2024-09-19T05:45:21.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.820+0000 7f44797fa640 1 -- 172.21.15.2:0/867986578 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4474012b70 con 0x7f447c15afe0 2024-09-19T05:45:21.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.821+0000 7f44797fa640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f446806c2b0 0x7f446806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:21.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.821+0000 7f44797fa640 1 -- 172.21.15.2:0/867986578 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f447408b8b0 con 0x7f447c15afe0 2024-09-19T05:45:21.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.821+0000 7f4480ab6640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f446806c2b0 0x7f446806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:21.824 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.821+0000 7f4480ab6640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f446806c2b0 0x7f446806e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f446c000f60 tx=0x7f446c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:21.827 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:21.825+0000 7f44797fa640 1 -- 172.21.15.2:0/867986578 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f447405a050 con 0x7f447c15afe0 2024-09-19T05:45:22.005 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.003+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f447c0cb910 con 0x7f447c15afe0 2024-09-19T05:45:22.006 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.004+0000 7f44797fa640 1 -- 172.21.15.2:0/867986578 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16762 (secure 0 0 0) 0x7f4474059950 con 0x7f447c15afe0 2024-09-19T05:45:22.007 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:22.007 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":102,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:45:21.682862+0000","last_up_change":"2024-09-19T05:44:52.039950+0000","last_in_change":"2024-09-19T05:43:19.394170+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":78,"up_thru":82,"down_at":68,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6801","nonce":2989037926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6803","nonce":2989037926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6807","nonce":2989037926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6805","nonce":2989037926}]},"public_addr":"172.21.15.123:6801/2989037926","cluster_addr":"172.21.15.123:6803/2989037926","heartbeat_back_addr":"172.21.15.123:6807/2989037926","heartbeat_front_addr":"172.21.15.123:6805/2989037926","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":68},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2a","osds":[5,0]}],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f446806c2b0 msgr2=0x7f446806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f446806c2b0 0x7f446806e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f446c000f60 tx=0x7f446c009040 comp rx=0 tx=0).stop 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c15afe0 msgr2=0x7f447c1f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c15afe0 0x7f447c1f1e00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f447400d600 tx=0x7f447400dac0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 shutdown_connections 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f446806c2b0 0x7f446806e770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f447c15afe0 0x7f447c1f1e00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.009 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f447c154e70 0x7f447c1f18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 --2- 172.21.15.2:0/867986578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f447c0cce60 0x7f447c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.006+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 >> 172.21.15.2:0/867986578 conn(0x7f447c06da40 msgr2=0x7f447c150e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.007+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 shutdown_connections 2024-09-19T05:45:22.010 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.007+0000 7f4482d1a640 1 -- 172.21.15.2:0/867986578 wait complete. 2024-09-19T05:45:22.020 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T05:45:22.020 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:45:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.120+0000 7eff90c74640 1 Processor -- start 2024-09-19T05:45:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.120+0000 7eff90c74640 1 -- start start 2024-09-19T05:45:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff90c74640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7eff8c154e70 0x7eff8c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff90c74640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7eff8c0cc670 0x7eff8c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff90c74640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff8c0cd780 con 0x7eff8c15afe0 2024-09-19T05:45:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff90c74640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7eff8c155250 con 0x7eff8c0cc670 2024-09-19T05:45:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff90c74640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7eff8c0cd4a0 con 0x7eff8c154e70 2024-09-19T05:45:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff81d9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7eff8c0cc670 0x7eff8c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff81d9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7eff8c0cc670 0x7eff8c15aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54358/0 (socket says 172.21.15.2:54358) 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff81d9b640 1 -- 172.21.15.2:0/3401194539 learned_addr learned my addr 172.21.15.2:0/3401194539 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff8a59c640 1 --2- 172.21.15.2:0/3401194539 >> v2:172.21.15.123:3300/0 conn(0x7eff8c154e70 0x7eff8c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff81d9b640 1 -- 172.21.15.2:0/3401194539 >> v2:172.21.15.123:3300/0 conn(0x7eff8c154e70 msgr2=0x7eff8c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff81d9b640 1 --2- 172.21.15.2:0/3401194539 >> v2:172.21.15.123:3300/0 conn(0x7eff8c154e70 0x7eff8c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff81d9b640 1 -- 172.21.15.2:0/3401194539 >> v1:172.21.15.123:6789/0 conn(0x7eff8c15afe0 legacy=0x7eff8c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff81d9b640 1 -- 172.21.15.2:0/3401194539 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c15e0b0 con 0x7eff8c0cc670 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.121+0000 7eff8a59c640 1 --2- 172.21.15.2:0/3401194539 >> v2:172.21.15.123:3300/0 conn(0x7eff8c154e70 0x7eff8c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.122+0000 7eff81d9b640 1 --2- 172.21.15.2:0/3401194539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c15aaa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7eff8c08e100 tx=0x7eff7402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5630c8f7e4c805 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.122+0000 7eff89d9b640 1 -- 172.21.15.2:0/3401194539 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74037070 con 0x7eff8c0cc670 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.122+0000 7eff89d9b640 1 -- 172.21.15.2:0/3401194539 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74032440 con 0x7eff8c0cc670 2024-09-19T05:45:22.125 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.122+0000 7eff89d9b640 1 -- 172.21.15.2:0/3401194539 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74032810 con 0x7eff8c0cc670 2024-09-19T05:45:22.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.122+0000 7eff90c74640 1 -- 172.21.15.2:0/3401194539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 msgr2=0x7eff8c15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.126 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.123+0000 7eff90c74640 1 --2- 172.21.15.2:0/3401194539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c15aaa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7eff8c08e100 tx=0x7eff7402a230 comp rx=0 tx=0).stop 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.123+0000 7eff90c74640 1 -- 172.21.15.2:0/3401194539 shutdown_connections 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.123+0000 7eff90c74640 1 --2- 172.21.15.2:0/3401194539 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c15aaa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.123+0000 7eff90c74640 1 --2- 172.21.15.2:0/3401194539 >> v2:172.21.15.123:3300/0 conn(0x7eff8c154e70 0x7eff8c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.123+0000 7eff90c74640 1 -- 172.21.15.2:0/3401194539 >> 172.21.15.2:0/3401194539 conn(0x7eff8c06da40 msgr2=0x7eff8c073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.123+0000 7eff90c74640 1 -- 172.21.15.2:0/3401194539 shutdown_connections 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.123+0000 7eff90c74640 1 -- 172.21.15.2:0/3401194539 wait complete. 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.124+0000 7eff90c74640 1 Processor -- start 2024-09-19T05:45:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.124+0000 7eff90c74640 1 -- start start 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff90c74640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff90c74640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff8c154e70 0x7eff8c14be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff90c74640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c15afe0 0x7eff8c14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff90c74640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c15e950 con 0x7eff8c0cc670 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff90c74640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c15e670 con 0x7eff8c15afe0 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff90c74640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7eff8c15ee70 con 0x7eff8c154e70 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c14d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54364/0 (socket says 172.21.15.2:54364) 2024-09-19T05:45:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 -- 172.21.15.2:0/3101481638 learned_addr learned my addr 172.21.15.2:0/3101481638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff81d9b640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff8c154e70 0x7eff8c14be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8ad9d640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c15afe0 0x7eff8c14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 -- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff8c154e70 msgr2=0x7eff8c14be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff8c154e70 0x7eff8c14be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 -- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c15afe0 msgr2=0x7eff8c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c15afe0 0x7eff8c14db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8a59c640 1 -- 172.21.15.2:0/3101481638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c14d670 con 0x7eff8c0cc670 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.125+0000 7eff8ad9d640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c15afe0 0x7eff8c14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.126+0000 7eff8a59c640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c14d130 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c7f0 tx=0x7eff7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.126+0000 7eff837fe640 1 -- 172.21.15.2:0/3101481638 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c015070 con 0x7eff8c0cc670 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.126+0000 7eff837fe640 1 -- 172.21.15.2:0/3101481638 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff7c007a30 con 0x7eff8c0cc670 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.126+0000 7eff837fe640 1 -- 172.21.15.2:0/3101481638 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c007e00 con 0x7eff8c0cc670 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.126+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c0d0d70 con 0x7eff8c0cc670 2024-09-19T05:45:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.126+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c0ce800 con 0x7eff8c0cc670 2024-09-19T05:45:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.127+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c0cca90 con 0x7eff8c0cc670 2024-09-19T05:45:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.127+0000 7eff837fe640 1 -- 172.21.15.2:0/3101481638 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7eff7c018070 con 0x7eff8c0cc670 2024-09-19T05:45:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.128+0000 7eff837fe640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7eff6c06c2e0 0x7eff6c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.128+0000 7eff837fe640 1 -- 172.21.15.2:0/3101481638 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7eff7c08b480 con 0x7eff8c0cc670 2024-09-19T05:45:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.128+0000 7eff81d9b640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7eff6c06c2e0 0x7eff6c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.129+0000 7eff81d9b640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7eff6c06c2e0 0x7eff6c06e7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff74034000 tx=0x7eff7402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.134 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.132+0000 7eff837fe640 1 -- 172.21.15.2:0/3101481638 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7eff7c059b90 con 0x7eff8c0cc670 2024-09-19T05:45:22.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.310+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff8c0cb910 con 0x7eff8c0cc670 2024-09-19T05:45:22.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.311+0000 7eff837fe640 1 -- 172.21.15.2:0/3101481638 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16762 (secure 0 0 0) 0x7eff7c059490 con 0x7eff8c0cc670 2024-09-19T05:45:22.313 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:22.313 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":102,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:45:21.682862+0000","last_up_change":"2024-09-19T05:44:52.039950+0000","last_in_change":"2024-09-19T05:43:19.394170+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":78,"up_thru":82,"down_at":68,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6801","nonce":2989037926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6803","nonce":2989037926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6807","nonce":2989037926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6805","nonce":2989037926}]},"public_addr":"172.21.15.123:6801/2989037926","cluster_addr":"172.21.15.123:6803/2989037926","heartbeat_back_addr":"172.21.15.123:6807/2989037926","heartbeat_front_addr":"172.21.15.123:6805/2989037926","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":68},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2a","osds":[5,0]}],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:45:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7eff6c06c2e0 msgr2=0x7eff6c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7eff6c06c2e0 0x7eff6c06e7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7eff74034000 tx=0x7eff7402e000 comp rx=0 tx=0).stop 2024-09-19T05:45:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 msgr2=0x7eff8c14d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c14d130 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c7f0 tx=0x7eff7c00cbf0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 shutdown_connections 2024-09-19T05:45:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7eff6c06c2e0 0x7eff6c06e7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c15afe0 0x7eff8c14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7eff8c154e70 0x7eff8c14be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 --2- 172.21.15.2:0/3101481638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff8c0cc670 0x7eff8c14d130 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 >> 172.21.15.2:0/3101481638 conn(0x7eff8c06da40 msgr2=0x7eff8c1e5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 shutdown_connections 2024-09-19T05:45:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.313+0000 7eff90c74640 1 -- 172.21.15.2:0/3101481638 wait complete. 2024-09-19T05:45:22.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T05:45:22.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T05:45:22.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T05:45:22.328 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T05:45:22.328 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-19T05:45:22.328 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-19T05:45:22.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-19T05:45:22.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-19T05:45:22.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.500+0000 7fe11679a640 1 Processor -- start 2024-09-19T05:45:22.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.500+0000 7fe11679a640 1 -- start start 2024-09-19T05:45:22.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.501+0000 7fac09a3c640 1 Processor -- start 2024-09-19T05:45:22.504 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.501+0000 7fac09a3c640 1 -- start start 2024-09-19T05:45:22.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fe11679a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe11015f7c0 0x7fe1100c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fe11679a640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1100c92d0 con 0x7fe1100c5420 2024-09-19T05:45:22.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fe11679a640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe11006d760 con 0x7fe110159300 2024-09-19T05:45:22.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fe11679a640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe1100ca1f0 con 0x7fe11015f7c0 2024-09-19T05:45:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac09a3c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fac04162790 0x7fac04162b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac09a3c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac041630b0 0x7fac04158de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac09a3c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04072ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac09a3c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fac04159790 con 0x7fac04162790 2024-09-19T05:45:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac09a3c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fac040c47f0 con 0x7fac041630b0 2024-09-19T05:45:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fe10ffff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe110159300 0x7fe110067aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:44972/0 (socket says 172.21.15.2:44972) 2024-09-19T05:45:22.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fe10ffff640 1 -- 172.21.15.2:0/697675424 learned_addr learned my addr 172.21.15.2:0/697675424 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac09a3c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac040c98b0 con 0x7fac04159320 2024-09-19T05:45:22.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac037fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04072ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac027fc640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac041630b0 0x7fac04158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.507 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac037fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04072ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40694/0 (socket says 172.21.15.2:40694) 2024-09-19T05:45:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.503+0000 7fe114d37640 1 --2- 172.21.15.2:0/697675424 >> v2:172.21.15.2:3300/0 conn(0x7fe11015f7c0 0x7fe1100c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.503+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499046310 0 0) 0x7fe11006d760 con 0x7fe110159300 2024-09-19T05:45:22.508 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.502+0000 7fac037fe640 1 -- 172.21.15.2:0/1179996205 learned_addr learned my addr 172.21.15.2:0/1179996205 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.503+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0f8003520 con 0x7fe110159300 2024-09-19T05:45:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.503+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414794416 0 0) 0x7fe1100c92d0 con 0x7fe1100c5420 2024-09-19T05:45:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.503+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe11006d760 con 0x7fe1100c5420 2024-09-19T05:45:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 383110760 0 0) 0x7fe0f8003520 con 0x7fe110159300 2024-09-19T05:45:22.509 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7f61beb9f640 1 Processor -- start 2024-09-19T05:45:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fe114d37640 1 -- 172.21.15.2:0/697675424 >> v1:172.21.15.123:6789/0 conn(0x7fe110159300 legacy=0x7fe110067aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fe114d37640 1 -- 172.21.15.2:0/697675424 >> v1:172.21.15.2:6789/0 conn(0x7fe1100c5420 legacy=0x7fe11015f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7f61beb9f640 1 -- start start 2024-09-19T05:45:22.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fe114d37640 1 -- 172.21.15.2:0/697675424 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11006da40 con 0x7fe11015f7c0 2024-09-19T05:45:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.503+0000 7fac02ffd640 1 --2- 172.21.15.2:0/1179996205 >> v2:172.21.15.2:3300/0 conn(0x7fac04162790 0x7fac04162b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fac037fe640 1 -- 172.21.15.2:0/1179996205 >> v2:172.21.15.123:3300/0 conn(0x7fac041630b0 msgr2=0x7fac04158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fac037fe640 1 --2- 172.21.15.2:0/1179996205 >> v2:172.21.15.123:3300/0 conn(0x7fac041630b0 0x7fac04158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fac037fe640 1 -- 172.21.15.2:0/1179996205 >> v2:172.21.15.2:3300/0 conn(0x7fac04162790 msgr2=0x7fac04162b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.511 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f95eb0fd640 1 Processor -- start 2024-09-19T05:45:22.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7fe114d37640 1 --2- 172.21.15.2:0/697675424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe11015f7c0 0x7fe1100c9cb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe1000097a0 tx=0x7fe10002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8dfd8555ca09c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fac037fe640 1 --2- 172.21.15.2:0/1179996205 >> v2:172.21.15.2:3300/0 conn(0x7fac04162790 0x7fac04162b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fac037fe640 1 -- 172.21.15.2:0/1179996205 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac040c9b90 con 0x7fac04159320 2024-09-19T05:45:22.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.504+0000 7fac02ffd640 1 --2- 172.21.15.2:0/1179996205 >> v2:172.21.15.2:3300/0 conn(0x7fac04162790 0x7fac04162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f61beb9f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80cd220 0x7f61b80b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f8d3e8a1640 1 Processor -- start 2024-09-19T05:45:22.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f95eb0fd640 1 -- start start 2024-09-19T05:45:22.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe100037070 con 0x7fe11015f7c0 2024-09-19T05:45:22.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe10002ab60 con 0x7fe11015f7c0 2024-09-19T05:45:22.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7fe10effd640 1 -- 172.21.15.2:0/697675424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108001970 con 0x7fe11015f7c0 2024-09-19T05:45:22.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f61beb9f640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f61b80b4630 0x7f61b815f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f61beb9f640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b81627c0 con 0x7f61b80b30e0 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f61beb9f640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f61b80d0450 con 0x7f61b80b4630 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.505+0000 7f61beb9f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f61b8162aa0 con 0x7f61b80cd220 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f61b7fff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f61b80b30e0 0x7f61b80d08e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:44974/0 (socket says 172.21.15.2:44974) 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f61b7fff640 1 -- 172.21.15.2:0/1751267768 learned_addr learned my addr 172.21.15.2:0/1751267768 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f61bc93b640 1 --2- 172.21.15.2:0/1751267768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80cd220 0x7f61b80b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f61bc93b640 1 -- 172.21.15.2:0/1751267768 >> v2:172.21.15.123:3300/0 conn(0x7f61b80b4630 msgr2=0x7f61b815f630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f61bc93b640 1 --2- 172.21.15.2:0/1751267768 >> v2:172.21.15.123:3300/0 conn(0x7f61b80b4630 0x7f61b815f630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f61bc93b640 1 -- 172.21.15.2:0/1751267768 >> v1:172.21.15.123:6789/0 conn(0x7f61b80b30e0 legacy=0x7f61b80d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f8d3e8a1640 1 -- start start 2024-09-19T05:45:22.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f61bc93b640 1 -- 172.21.15.2:0/1751267768 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b8162d80 con 0x7f61b80cd220 2024-09-19T05:45:22.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7fe11679a640 1 -- 172.21.15.2:0/697675424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe11015f7c0 msgr2=0x7fe1100c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7fe11679a640 1 --2- 172.21.15.2:0/697675424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe11015f7c0 0x7fe1100c9cb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe1000097a0 tx=0x7fe10002a0d0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7fac037fe640 1 --2- 172.21.15.2:0/1179996205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04072ae0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac0415cbe0 tx=0x7fabfc02a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4eaf45b2724a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7fe11679a640 1 -- 172.21.15.2:0/697675424 shutdown_connections 2024-09-19T05:45:22.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7fe11679a640 1 --2- 172.21.15.2:0/697675424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe11015f7c0 0x7fe1100c9cb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7fe11679a640 1 -- 172.21.15.2:0/697675424 >> 172.21.15.2:0/697675424 conn(0x7fe110070fc0 msgr2=0x7fe11006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7fac01ffb640 1 -- 172.21.15.2:0/1179996205 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabfc036070 con 0x7fac04159320 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3e8a1640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d30007380 0x7f8d300de630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3e8a1640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d3004f170 0x7f8d300deb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3e8a1640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d300502e0 con 0x7f8d300d8a90 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3e8a1640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8d30054f80 con 0x7f8d3004f170 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3e8a1640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8d30050550 con 0x7f8d30007380 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f95eb0fd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e4158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f95eb0fd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f95e40c6ef0 0x7f95e4159000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f95eb0fd640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e4072380 con 0x7f95e40c9110 2024-09-19T05:45:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f95eb0fd640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f95e4073650 con 0x7f95e40c6ef0 2024-09-19T05:45:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f95eb0fd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f95e4072d20 con 0x7f95e40c8d30 2024-09-19T05:45:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3d09e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8d30007380 0x7f8d300de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.506+0000 7f677a952640 1 Processor -- start 2024-09-19T05:45:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f677a952640 1 -- start start 2024-09-19T05:45:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f95ea0fb640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e4158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f95ea0fb640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e4158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40718/0 (socket says 172.21.15.2:40718) 2024-09-19T05:45:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f95ea0fb640 1 -- 172.21.15.2:0/143171508 learned_addr learned my addr 172.21.15.2:0/143171508 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3e0a0640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d3004f170 0x7f8d300deb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3d89f640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8d300d8a90 0x7f8d30048fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54058/0 (socket says 172.21.15.2:54058) 2024-09-19T05:45:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7f8d3d89f640 1 -- 172.21.15.2:0/929825598 learned_addr learned my addr 172.21.15.2:0/929825598 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f61bc93b640 1 --2- 172.21.15.2:0/1751267768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80cd220 0x7f61b80b5a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61b00097a0 tx=0x7f61b002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ee25691dff4a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fe11679a640 1 -- 172.21.15.2:0/697675424 shutdown_connections 2024-09-19T05:45:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fe11679a640 1 -- 172.21.15.2:0/697675424 wait complete. 2024-09-19T05:45:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f7a18b31640 1 Processor -- start 2024-09-19T05:45:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f61b77fe640 1 -- 172.21.15.2:0/1751267768 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0037070 con 0x7f61b80cd220 2024-09-19T05:45:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f61b77fe640 1 -- 172.21.15.2:0/1751267768 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b002adc0 con 0x7f61b80cd220 2024-09-19T05:45:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f61b77fe640 1 -- 172.21.15.2:0/1751267768 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0032660 con 0x7f61b80cd220 2024-09-19T05:45:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.507+0000 7fac01ffb640 1 -- 172.21.15.2:0/1179996205 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabfc031600 con 0x7fac04159320 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fac09a3c640 1 -- 172.21.15.2:0/1179996205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 msgr2=0x7fac04072ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fac09a3c640 1 --2- 172.21.15.2:0/1179996205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04072ae0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac0415cbe0 tx=0x7fabfc02a6a0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fac09a3c640 1 -- 172.21.15.2:0/1179996205 shutdown_connections 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fac09a3c640 1 --2- 172.21.15.2:0/1179996205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04072ae0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fac09a3c640 1 --2- 172.21.15.2:0/1179996205 >> v2:172.21.15.123:3300/0 conn(0x7fac041630b0 0x7fac04158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fac09a3c640 1 --2- 172.21.15.2:0/1179996205 >> v2:172.21.15.2:3300/0 conn(0x7fac04162790 0x7fac04162b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7fac09a3c640 1 -- 172.21.15.2:0/1179996205 >> 172.21.15.2:0/1179996205 conn(0x7fac0406bc10 msgr2=0x7fac0406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f8d3e0a0640 1 -- 172.21.15.2:0/929825598 >> v2:172.21.15.123:3300/0 conn(0x7f8d30007380 msgr2=0x7f8d300de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f8d3e0a0640 1 --2- 172.21.15.2:0/929825598 >> v2:172.21.15.123:3300/0 conn(0x7f8d30007380 0x7f8d300de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f8d3e0a0640 1 -- 172.21.15.2:0/929825598 >> v1:172.21.15.2:6789/0 conn(0x7f8d300d8a90 legacy=0x7f8d30048fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f8d3e0a0640 1 -- 172.21.15.2:0/929825598 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d300068a0 con 0x7f8d3004f170 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f677a952640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67740cc520 0x7f677406cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f677a952640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b3260 0x7f677406d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f67737fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67740cc520 0x7f677406cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f67737fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f67740cc520 0x7f677406cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:36982/0 (socket says 172.21.15.2:36982) 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f6773fff640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f67740b5ce0 0x7f67740cb2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:44988/0 (socket says 172.21.15.2:44988) 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f67737fe640 1 -- 172.21.15.2:0/1676684086 learned_addr learned my addr 172.21.15.2:0/1676684086 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f677415f040 con 0x7f67740b5ce0 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f95ea0fb640 1 -- 172.21.15.2:0/143171508 >> v2:172.21.15.2:3300/0 conn(0x7f95e40c6ef0 msgr2=0x7f95e4159000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f95e98fa640 1 --2- 172.21.15.2:0/143171508 >> v2:172.21.15.2:3300/0 conn(0x7f95e40c6ef0 0x7f95e4159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f95ea0fb640 1 --2- 172.21.15.2:0/143171508 >> v2:172.21.15.2:3300/0 conn(0x7f95e40c6ef0 0x7f95e4159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f95ea0fb640 1 -- 172.21.15.2:0/143171508 >> v1:172.21.15.123:6789/0 conn(0x7f95e40c9110 legacy=0x7f95e415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f95ea0fb640 1 -- 172.21.15.2:0/143171508 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e4073000 con 0x7f95e40c8d30 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f677406ca50 con 0x7f67740cc520 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f677415fe60 con 0x7f67740b3260 2024-09-19T05:45:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f6778eef640 1 --2- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b3260 0x7f677406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f8d3e0a0640 1 --2- 172.21.15.2:0/929825598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d3004f170 0x7f8d300deb70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000faa0 tx=0x7f8d3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d634a9b4e0b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f61beb9f640 1 -- 172.21.15.2:0/1751267768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80cd220 msgr2=0x7f61b80b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f61beb9f640 1 --2- 172.21.15.2:0/1751267768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80cd220 0x7f61b80b5a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f61b00097a0 tx=0x7f61b002a130 comp rx=0 tx=0).stop 2024-09-19T05:45:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f61beb9f640 1 -- 172.21.15.2:0/1751267768 shutdown_connections 2024-09-19T05:45:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f61beb9f640 1 --2- 172.21.15.2:0/1751267768 >> v2:172.21.15.123:3300/0 conn(0x7f61b80b4630 0x7f61b815f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f61beb9f640 1 --2- 172.21.15.2:0/1751267768 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80cd220 0x7f61b80b5a50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f61beb9f640 1 -- 172.21.15.2:0/1751267768 >> 172.21.15.2:0/1751267768 conn(0x7f61b806d0e0 msgr2=0x7f61b806d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f6772ffd640 1 -- 172.21.15.2:0/1676684086 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000385167 0 0) 0x7f677415f040 con 0x7f67740b5ce0 2024-09-19T05:45:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.509+0000 7f6772ffd640 1 -- 172.21.15.2:0/1676684086 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f675c003520 con 0x7f67740b5ce0 2024-09-19T05:45:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f8d3c89d640 1 -- 172.21.15.2:0/929825598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34037070 con 0x7f8d3004f170 2024-09-19T05:45:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f8d3c89d640 1 -- 172.21.15.2:0/929825598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d34032440 con 0x7f8d3004f170 2024-09-19T05:45:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f8d3c89d640 1 -- 172.21.15.2:0/929825598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34032810 con 0x7f8d3004f170 2024-09-19T05:45:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.508+0000 7f7a18b31640 1 -- start start 2024-09-19T05:45:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f67737fe640 1 -- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b3260 msgr2=0x7f677406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f67737fe640 1 --2- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b3260 0x7f677406d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f67737fe640 1 -- 172.21.15.2:0/1676684086 >> v1:172.21.15.123:6789/0 conn(0x7f67740b5ce0 legacy=0x7f67740cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f67737fe640 1 -- 172.21.15.2:0/1676684086 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677415f2e0 con 0x7f67740cc520 2024-09-19T05:45:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7fac09a3c640 1 -- 172.21.15.2:0/1179996205 shutdown_connections 2024-09-19T05:45:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7fe11679a640 1 Processor -- start 2024-09-19T05:45:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f61beb9f640 1 -- 172.21.15.2:0/1751267768 shutdown_connections 2024-09-19T05:45:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f7a1259c640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7a0c0d6880 0x7f7a0c048590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:45002/0 (socket says 172.21.15.2:45002) 2024-09-19T05:45:22.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f61beb9f640 1 -- 172.21.15.2:0/1751267768 wait complete. 2024-09-19T05:45:22.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f7a11d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f7a0c04cc40 0x7f7a0c0dfa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54060/0 (socket says 172.21.15.2:54060) 2024-09-19T05:45:22.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f7a18b31640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7a0c005a40 0x7f7a0c046d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f7a1259c640 1 -- 172.21.15.2:0/368024007 learned_addr learned my addr 172.21.15.2:0/368024007 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f8bd9c1e640 1 Processor -- start 2024-09-19T05:45:22.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7fe11679a640 1 -- start start 2024-09-19T05:45:22.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f8bd9c1e640 1 -- start start 2024-09-19T05:45:22.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0c0e04d0 con 0x7f7a0c04cc40 2024-09-19T05:45:22.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a0c0e07b0 con 0x7f7a0c0d6880 2024-09-19T05:45:22.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f7a12d9d640 1 --2- 172.21.15.2:0/368024007 >> v2:172.21.15.123:3300/0 conn(0x7f7a0c005a40 0x7f7a0c046d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7a0c047860 con 0x7f7a0c005a40 2024-09-19T05:45:22.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7f95ea0fb640 1 --2- 172.21.15.2:0/143171508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e4158ac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95e40737d0 tx=0x7f95d802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a0572ec0f95c32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.510+0000 7fac09a3c640 1 -- 172.21.15.2:0/1179996205 wait complete. 2024-09-19T05:45:22.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342121018 0 0) 0x7f7a0c0e04d0 con 0x7f7a0c04cc40 2024-09-19T05:45:22.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79fc003520 con 0x7f7a0c04cc40 2024-09-19T05:45:22.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 67940934 0 0) 0x7f7a0c0e07b0 con 0x7f7a0c0d6880 2024-09-19T05:45:22.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a0c0e04d0 con 0x7f7a0c0d6880 2024-09-19T05:45:22.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f67737fe640 1 --2- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740cc520 0x7f677406cee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f677406cbd0 tx=0x7f676802a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538e53ee7c18aabf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f95e90f9640 1 -- 172.21.15.2:0/143171508 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8037070 con 0x7f95e40c8d30 2024-09-19T05:45:22.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f95e90f9640 1 -- 172.21.15.2:0/143171508 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d8032440 con 0x7f95e40c8d30 2024-09-19T05:45:22.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/929825598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d3004f170 msgr2=0x7f8d300deb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f8d3e8a1640 1 --2- 172.21.15.2:0/929825598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d3004f170 0x7f8d300deb70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000faa0 tx=0x7f8d3402a1a0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f95e90f9640 1 -- 172.21.15.2:0/143171508 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8032810 con 0x7f95e40c8d30 2024-09-19T05:45:22.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 122483781 0 0) 0x7f79fc003520 con 0x7f7a0c04cc40 2024-09-19T05:45:22.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 >> v2:172.21.15.123:3300/0 conn(0x7f7a0c005a40 msgr2=0x7f7a0c046d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 --2- 172.21.15.2:0/368024007 >> v2:172.21.15.123:3300/0 conn(0x7f7a0c005a40 0x7f7a0c046d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 >> v1:172.21.15.123:6789/0 conn(0x7f7a0c0d6880 legacy=0x7f7a0c048590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c047ab0 con 0x7f7a0c04cc40 2024-09-19T05:45:22.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f7a080015a0 con 0x7f7a0c04cc40 2024-09-19T05:45:22.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 >> v1:172.21.15.2:6789/0 conn(0x7f7a0c04cc40 legacy=0x7f7a0c0dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f79fc003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79fc004800 0x7f79fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0051c0 0x7f79fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c047860 con 0x7f79fc004800 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc00c040 con 0x7f79fc0051c0 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc010650 con 0x7f79fc003b70 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7fe11679a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1100c5420 0x7fe1100c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f6772ffd640 1 -- 172.21.15.2:0/1676684086 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768037070 con 0x7f67740cc520 2024-09-19T05:45:22.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe11679a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe110159300 0x7fe1101f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f6772ffd640 1 -- 172.21.15.2:0/1676684086 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768032640 con 0x7f67740cc520 2024-09-19T05:45:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1100c5420 0x7fe1100c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe11679a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe11015f7c0 0x7fe1101f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1100c5420 0x7fe1100c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37002/0 (socket says 172.21.15.2:37002) 2024-09-19T05:45:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 -- 172.21.15.2:0/2159175595 learned_addr learned my addr 172.21.15.2:0/2159175595 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe11679a640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe1100c9590 con 0x7fe11015f7c0 2024-09-19T05:45:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe11679a640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1101596e0 con 0x7fe1100c5420 2024-09-19T05:45:22.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe11679a640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe1100cbfc0 con 0x7fe110159300 2024-09-19T05:45:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f7a1259c640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f79fc003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.511+0000 7f6772ffd640 1 -- 172.21.15.2:0/1676684086 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768032a10 con 0x7f67740cc520 2024-09-19T05:45:22.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f7a11d9b640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79fc004800 0x7f79fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 -- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe110159300 msgr2=0x7fe1101f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe110159300 0x7fe1101f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 -- 172.21.15.2:0/2159175595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe11015f7c0 msgr2=0x7fe1101f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe11015f7c0 0x7fe1101f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fe10ffff640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe110153030 con 0x7fe1100c5420 2024-09-19T05:45:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8bd9c1e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0dcb50 0x7f8bcc04bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8bd9c1e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8bcc0e5fa0 0x7f8bcc0e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8bd9c1e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc04a950 con 0x7f8bcc0dcf30 2024-09-19T05:45:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8bd9c1e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc005a40 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8bd9c1e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc0e34a0 con 0x7f8bcc0dcb50 2024-09-19T05:45:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/929825598 shutdown_connections 2024-09-19T05:45:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8d3e8a1640 1 --2- 172.21.15.2:0/929825598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d3004f170 0x7f8d300deb70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8d3e8a1640 1 --2- 172.21.15.2:0/929825598 >> v2:172.21.15.123:3300/0 conn(0x7f8d30007380 0x7f8d300de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/929825598 >> 172.21.15.2:0/929825598 conn(0x7f8d30054060 msgr2=0x7f8d30054470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f61beb9f640 1 Processor -- start 2024-09-19T05:45:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f61beb9f640 1 -- start start 2024-09-19T05:45:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f7a1259c640 1 -- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0051c0 msgr2=0x7f79fc00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:45:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f7a1259c640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79fc0051c0 0x7f79fc00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f7a1259c640 1 -- 172.21.15.2:0/368024007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79fc004800 msgr2=0x7f79fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f7a1259c640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f79fc004800 0x7f79fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f7a1259c640 1 -- 172.21.15.2:0/368024007 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c047ab0 con 0x7f79fc003b70 2024-09-19T05:45:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740cc520 msgr2=0x7f677406cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 --2- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740cc520 0x7f677406cee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f677406cbd0 tx=0x7f676802a2e0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fe10ffff640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1100c5420 0x7fe1100c8340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c760 tx=0x7fe10400cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8bd2ffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8bcc0dcf30 0x7f8bcc045c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54070/0 (socket says 172.21.15.2:54070) 2024-09-19T05:45:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8bd2ffd640 1 -- 172.21.15.2:0/2271776474 learned_addr learned my addr 172.21.15.2:0/2271776474 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8bd37fe640 1 --2- 172.21.15.2:0/2271776474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0dcb50 0x7f8bcc04bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fac09a3c640 1 Processor -- start 2024-09-19T05:45:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7fac09a3c640 1 -- start start 2024-09-19T05:45:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f95eb0fd640 1 -- 172.21.15.2:0/143171508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 msgr2=0x7f95e4158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/929825598 shutdown_connections 2024-09-19T05:45:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f7a12d9d640 1 -- 172.21.15.2:0/368024007 reap_dead start 2024-09-19T05:45:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 shutdown_connections 2024-09-19T05:45:22.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 --2- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b3260 0x7f677406d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 --2- 172.21.15.2:0/1676684086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740cc520 0x7f677406cee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 >> 172.21.15.2:0/1676684086 conn(0x7f6774067950 msgr2=0x7f6774067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.512+0000 7f95eb0fd640 1 --2- 172.21.15.2:0/143171508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e4158ac0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f95e40737d0 tx=0x7f95d802a1e0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8bd37fe640 1 -- 172.21.15.2:0/2271776474 >> v2:172.21.15.2:3300/0 conn(0x7f8bcc0e5fa0 msgr2=0x7f8bcc0e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8bd37fe640 1 --2- 172.21.15.2:0/2271776474 >> v2:172.21.15.2:3300/0 conn(0x7f8bcc0e5fa0 0x7f8bcc0e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8bd37fe640 1 -- 172.21.15.2:0/2271776474 >> v1:172.21.15.2:6789/0 conn(0x7f8bcc0dcf30 legacy=0x7f8bcc045c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8bd37fe640 1 -- 172.21.15.2:0/2271776474 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0e36d0 con 0x7f8bcc0dcb50 2024-09-19T05:45:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f95eb0fd640 1 -- 172.21.15.2:0/143171508 shutdown_connections 2024-09-19T05:45:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f95eb0fd640 1 --2- 172.21.15.2:0/143171508 >> v2:172.21.15.2:3300/0 conn(0x7f95e40c6ef0 0x7f95e4159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f95eb0fd640 1 --2- 172.21.15.2:0/143171508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e4158ac0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f95eb0fd640 1 -- 172.21.15.2:0/143171508 >> 172.21.15.2:0/143171508 conn(0x7f95e40ccc60 msgr2=0x7f95e40cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f95eb0fd640 1 -- 172.21.15.2:0/143171508 shutdown_connections 2024-09-19T05:45:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f95eb0fd640 1 -- 172.21.15.2:0/143171508 wait complete. 2024-09-19T05:45:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 shutdown_connections 2024-09-19T05:45:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/929825598 wait complete. 2024-09-19T05:45:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61beb9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61b80b30e0 0x7f61b80c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61beb9f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80b4630 0x7f61b80c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61beb9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b80cd220 0x7f61b80c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61bc93b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61b80b30e0 0x7f61b80c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80b4630 0x7f61b80c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61bc93b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61b80b30e0 0x7f61b80c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54450/0 (socket says 172.21.15.2:54450) 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80b4630 0x7f61b80c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40776/0 (socket says 172.21.15.2:40776) 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80b5f90 con 0x7f61b80b30e0 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61b8163340 con 0x7f61b80cd220 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f61b80cd600 con 0x7f61b80b4630 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f61bc93b640 1 -- 172.21.15.2:0/1097240011 learned_addr learned my addr 172.21.15.2:0/1097240011 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104005ce0 con 0x7fe1100c5420 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fe11679a640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1100c68e0 con 0x7fe1100c5420 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7fe11679a640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1100c8880 con 0x7fe1100c5420 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe104007070 con 0x7fe1100c5420 2024-09-19T05:45:22.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104007460 con 0x7fe1100c5420 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fac09a3c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fac09a3c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac04162790 0x7fac041f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fac09a3c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac041630b0 0x7fac041e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fac09a3c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fac040722e0 con 0x7fac04162790 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7fac09a3c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac04072000 con 0x7fac041630b0 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7fac09a3c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac040725c0 con 0x7fac04159320 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7fac02ffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40768/0 (socket says 172.21.15.2:40768) 2024-09-19T05:45:22.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7fac02ffd640 1 -- 172.21.15.2:0/2435911212 learned_addr learned my addr 172.21.15.2:0/2435911212 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.514+0000 7f7a1259c640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f79fc003f50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c190 tx=0x7f79fc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620900fa721be311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f95eb0fd640 1 Processor -- start 2024-09-19T05:45:22.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.513+0000 7f677a952640 1 -- 172.21.15.2:0/1676684086 wait complete. 2024-09-19T05:45:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f95eb0fd640 1 -- start start 2024-09-19T05:45:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7fac02ffd640 1 -- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac041630b0 msgr2=0x7fac041e9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7fac02ffd640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac041630b0 0x7fac041e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7fac02ffd640 1 -- 172.21.15.2:0/2435911212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac04162790 msgr2=0x7fac041f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bc93b640 1 -- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80b4630 msgr2=0x7f61b80c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7fac02ffd640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac04162790 0x7fac041f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7fac02ffd640 1 -- 172.21.15.2:0/2435911212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac04154720 con 0x7fac04159320 2024-09-19T05:45:22.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bc93b640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f61b80b4630 0x7f61b80c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f8d3e8a1640 1 Processor -- start 2024-09-19T05:45:22.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bd13c640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b80cd220 0x7f61b80c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bc93b640 1 -- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b80cd220 msgr2=0x7f61b80c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bc93b640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b80cd220 0x7f61b80c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7fe11679a640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0f0001030 con 0x7fe1100c5420 2024-09-19T05:45:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bc93b640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b80c7ae0 con 0x7f61b80b30e0 2024-09-19T05:45:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bd13c640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61b80cd220 0x7f61b80c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7fac02ffd640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04159920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e9b0 tx=0x7fabf400ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04007c20 con 0x7f79fc003b70 2024-09-19T05:45:22.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a04004ac0 con 0x7f79fc003b70 2024-09-19T05:45:22.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f7a1159a640 1 -- 172.21.15.2:0/368024007 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040070d0 con 0x7f79fc003b70 2024-09-19T05:45:22.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f61bc93b640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61b80b30e0 0x7f61b80c8b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f61b002a720 tx=0x7f61b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f677a952640 1 Processor -- start 2024-09-19T05:45:22.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4017070 con 0x7fac04159320 2024-09-19T05:45:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0037040 con 0x7f61b80b30e0 2024-09-19T05:45:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b0004230 con 0x7f61b80b30e0 2024-09-19T05:45:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b80c7dc0 con 0x7f61b80b30e0 2024-09-19T05:45:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0037040 con 0x7f61b80b30e0 2024-09-19T05:45:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7fac09a3c640 1 -- 172.21.15.2:0/2435911212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac041549c0 con 0x7fac04159320 2024-09-19T05:45:22.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7fac09a3c640 1 -- 172.21.15.2:0/2435911212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fac041ea660 con 0x7fac04159320 2024-09-19T05:45:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f95eb0fd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e40c6ef0 0x7f95e41597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f95eb0fd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f677a952640 1 -- start start 2024-09-19T05:45:22.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b80c8380 con 0x7f61b80b30e0 2024-09-19T05:45:22.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f95eb0fd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40c9110 0x7f95e41f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f95eb0fd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f95e40740c0 con 0x7f95e40c6ef0 2024-09-19T05:45:22.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f95eb0fd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95e40c8070 con 0x7f95e40c9110 2024-09-19T05:45:22.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f95eb0fd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f95e4074240 con 0x7f95e40c8d30 2024-09-19T05:45:22.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.515+0000 7f8d3e8a1640 1 -- start start 2024-09-19T05:45:22.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 msgr2=0x7f79fc003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f7a18b31640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f79fc003f50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c190 tx=0x7f79fc005830 comp rx=0 tx=0).stop 2024-09-19T05:45:22.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f8d3e8a1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d30007380 0x7f8d300db3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 shutdown_connections 2024-09-19T05:45:22.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f7a18b31640 1 --2- 172.21.15.2:0/368024007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f79fc003f50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 >> 172.21.15.2:0/368024007 conn(0x7f7a0c051e50 msgr2=0x7f7a0c052260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f8d3e8a1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d3004f170 0x7f8d300396a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 shutdown_connections 2024-09-19T05:45:22.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f8d3e8a1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d300d8a90 0x7f8d30039be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f8d3e8a1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3004f730 con 0x7f8d300d8a90 2024-09-19T05:45:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf400ca00 con 0x7fac04159320 2024-09-19T05:45:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f8d3e8a1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30006e60 con 0x7f8d3004f170 2024-09-19T05:45:22.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf400cdd0 con 0x7fac04159320 2024-09-19T05:45:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.516+0000 7f8d3e8a1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3004f8b0 con 0x7f8d30007380 2024-09-19T05:45:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7fac09a3c640 1 -- 172.21.15.2:0/2435911212 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabe4001030 con 0x7fac04159320 2024-09-19T05:45:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f95ea0fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e40c6ef0 0x7f95e41597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f7a18b31640 1 -- 172.21.15.2:0/368024007 wait complete. 2024-09-19T05:45:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f95ea0fb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e40c6ef0 0x7f95e41597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54458/0 (socket says 172.21.15.2:54458) 2024-09-19T05:45:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f95ea0fb640 1 -- 172.21.15.2:0/559477579 learned_addr learned my addr 172.21.15.2:0/559477579 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f61aeffd640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6190001030 con 0x7f61b80b30e0 2024-09-19T05:45:22.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f677a952640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740b3260 0x7f6774160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f8bd37fe640 1 --2- 172.21.15.2:0/2271776474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0dcb50 0x7f8bcc04bf10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0e38a0 tx=0x7f8bd40a6ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c29d04c49ae1ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f8d3e0a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d300d8a90 0x7f8d30039be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f8d3e0a0640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d300d8a90 0x7f8d30039be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54464/0 (socket says 172.21.15.2:54464) 2024-09-19T05:45:22.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7f8bd27fc640 1 -- 172.21.15.2:0/2271776474 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40a89c0 con 0x7f8bcc0dcb50 2024-09-19T05:45:22.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f8bd27fc640 1 -- 172.21.15.2:0/2271776474 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd40a8b60 con 0x7f8bcc0dcb50 2024-09-19T05:45:22.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f677a952640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b5ce0 0x7f6774148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f6773fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740b3260 0x7f6774160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f95ea0fb640 1 -- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 msgr2=0x7f95e414cad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:45:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f677a952640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67740cc520 0x7f67740cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f95ea0fb640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e40c8d30 0x7f95e414cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.517+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe104007600 con 0x7fe1100c5420 2024-09-19T05:45:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7fe10cff9640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe0fc06c380 0x7fe0fc06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fe10408b680 con 0x7fe1100c5420 2024-09-19T05:45:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7fe10cff9640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe0fc074720 0x7fe0fc076b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe104016e50 con 0x7fe0fc074720 2024-09-19T05:45:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fe10408ba70 con 0x7fe1100c5420 2024-09-19T05:45:22.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f95ea0fb640 1 -- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40c9110 msgr2=0x7f95e41f7e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T05:45:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f677a952640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f677415fb80 con 0x7f67740cc520 2024-09-19T05:45:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f677a952640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f677415f8a0 con 0x7f67740b3260 2024-09-19T05:45:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f677a952640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f67740cc140 con 0x7f67740b5ce0 2024-09-19T05:45:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f6773fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740b3260 0x7f6774160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37036/0 (socket says 172.21.15.2:37036) 2024-09-19T05:45:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f6773fff640 1 -- 172.21.15.2:0/121416060 learned_addr learned my addr 172.21.15.2:0/121416060 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f67737fe640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b5ce0 0x7f6774148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f95ea0fb640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e40c9110 0x7f95e41f7e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f95ea0fb640 1 -- 172.21.15.2:0/559477579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e414edd0 con 0x7f95e40c6ef0 2024-09-19T05:45:22.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f8bd27fc640 1 -- 172.21.15.2:0/2271776474 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40b0420 con 0x7f8bcc0dcb50 2024-09-19T05:45:22.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f8d3e0a0640 1 -- 172.21.15.2:0/109025167 learned_addr learned my addr 172.21.15.2:0/109025167 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7fe10f7fe640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe0fc06c380 0x7fe0fc06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f8d3d09e640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d3004f170 0x7f8d300396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f8d3d89f640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d30007380 0x7f8d300db3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f67737fe640 1 -- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740b3260 msgr2=0x7f6774160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f67737fe640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67740b3260 0x7f6774160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f67737fe640 1 -- 172.21.15.2:0/121416060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67740cc520 msgr2=0x7f67740cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.518+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/2271776474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0dcb50 msgr2=0x7f8bcc04bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8bd9c1e640 1 --2- 172.21.15.2:0/2271776474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0dcb50 0x7f8bcc04bf10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0e38a0 tx=0x7f8bd40a6ba0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f6778eef640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67740cc520 0x7f67740cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/2271776474 shutdown_connections 2024-09-19T05:45:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8bd9c1e640 1 --2- 172.21.15.2:0/2271776474 >> v2:172.21.15.2:3300/0 conn(0x7f8bcc0e5fa0 0x7f8bcc0e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8bd9c1e640 1 --2- 172.21.15.2:0/2271776474 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0dcb50 0x7f8bcc04bf10 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0e38a0 tx=0x7f8bd40a6ba0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/2271776474 >> 172.21.15.2:0/2271776474 conn(0x7f8bcc04fc40 msgr2=0x7f8bcc050050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7fe114d37640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe0fc074720 0x7fe0fc076b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7fe10f7fe640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe0fc06c380 0x7fe0fc06e840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe108001460 tx=0x7fe108001820 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f67737fe640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67740cc520 0x7f67740cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f67737fe640 1 -- 172.21.15.2:0/121416060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67740cf060 con 0x7f67740b5ce0 2024-09-19T05:45:22.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f6778eef640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67740cc520 0x7f67740cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:22.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f7a18b31640 1 Processor -- start 2024-09-19T05:45:22.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8d3d09e640 1 -- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d30007380 msgr2=0x7f8d300db3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8d3d09e640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d30007380 0x7f8d300db3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8d3d09e640 1 -- 172.21.15.2:0/109025167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d300d8a90 msgr2=0x7f8d30039be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8d3d09e640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d300d8a90 0x7f8d30039be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8d3d09e640 1 -- 172.21.15.2:0/109025167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d300df1e0 con 0x7f8d3004f170 2024-09-19T05:45:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8d3d89f640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8d30007380 0x7f8d300db3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f7a18b31640 1 -- start start 2024-09-19T05:45:22.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f95ea0fb640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e40c6ef0 0x7f95e41597f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000c00 tx=0x7f95d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f67737fe640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b5ce0 0x7f6774148a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6768031000 tx=0x7f67680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fabf401a070 con 0x7fac04159320 2024-09-19T05:45:22.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.519+0000 7f8d3d09e640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d3004f170 0x7f8d300396a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d380504d0 tx=0x7f8d38069890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768040020 con 0x7f67740b5ce0 2024-09-19T05:45:22.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f61b0003610 con 0x7f61b80b30e0 2024-09-19T05:45:22.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7fe114d37640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe0fc074720 0x7fe0fc076b80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768038040 con 0x7f67740b5ce0 2024-09-19T05:45:22.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67740cf320 con 0x7f67740b5ce0 2024-09-19T05:45:22.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7fabebfff640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fabf006c330 0x7fabf006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741f5430 con 0x7f67740b5ce0 2024-09-19T05:45:22.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fabf408b480 con 0x7fac04159320 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7fabebfff640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fabf00746d0 0x7fabf0076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabf400ca00 con 0x7fabf00746d0 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fabf408b960 con 0x7fac04159320 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8037040 con 0x7f95e40c6ef0 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f95eb0fd640 1 -- 172.21.15.2:0/559477579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e4157a40 con 0x7f95e40c6ef0 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d8004250 con 0x7f95e40c6ef0 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f95eb0fd640 1 -- 172.21.15.2:0/559477579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e414c430 con 0x7f95e40c6ef0 2024-09-19T05:45:22.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8003610 con 0x7f95e40c6ef0 2024-09-19T05:45:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f61b57fa640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f619806c4b0 0x7f619806e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f61b00ac630 con 0x7f61b80b30e0 2024-09-19T05:45:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f61b57fa640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f61980747d0 0x7f6198076c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61b0007e80 con 0x7f61980747d0 2024-09-19T05:45:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f61b00acac0 con 0x7f61b80b30e0 2024-09-19T05:45:22.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7fac027fc640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fabf006c330 0x7fabf006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d380738b0 con 0x7f8d3004f170 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d38073a50 con 0x7f8d3004f170 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f7a18b31640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f7a0c0093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f7a18b31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0370c0 0x7f7a0c0c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f7a18b31640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c050c80 0x7f7a0c051100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f7a18b31640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c0d6c60 con 0x7f7a0c050c80 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f7a18b31640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c048100 con 0x7f7a0c0370c0 2024-09-19T05:45:22.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.520+0000 7f7a18b31640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7a0c048590 con 0x7f79fc003b70 2024-09-19T05:45:22.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768037070 con 0x7f67740b5ce0 2024-09-19T05:45:22.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/2271776474 shutdown_connections 2024-09-19T05:45:22.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f7a1259c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f7a0c0093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f7a1259c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f7a0c0093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40806/0 (socket says 172.21.15.2:40806) 2024-09-19T05:45:22.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f7a1259c640 1 -- 172.21.15.2:0/1731446614 learned_addr learned my addr 172.21.15.2:0/1731446614 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f7a12d9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c050c80 0x7f7a0c051100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f7a11d9b640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0370c0 0x7f7a0c0c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f61b7fff640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f619806c4b0 0x7f619806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a1259c640 1 -- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0370c0 msgr2=0x7f7a0c0c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a1259c640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a0c0370c0 0x7f7a0c0c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a1259c640 1 -- 172.21.15.2:0/1731446614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c050c80 msgr2=0x7f7a0c051100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a1259c640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c050c80 0x7f7a0c051100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38073e40 con 0x7f8d3004f170 2024-09-19T05:45:22.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a1259c640 1 -- 172.21.15.2:0/1731446614 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c007950 con 0x7f79fc003b70 2024-09-19T05:45:22.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a12d9d640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7a0c050c80 0x7f7a0c051100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:22.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f61bd13c640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f61980747d0 0x7f6198076c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3004dfe0 con 0x7f8d3004f170 2024-09-19T05:45:22.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a1259c640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f7a0c0093f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c2b0 tx=0x7f7a04001790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d300dd870 con 0x7f8d3004f170 2024-09-19T05:45:22.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6754001030 con 0x7f67740b5ce0 2024-09-19T05:45:22.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f95e40c8570 con 0x7f95e40c6ef0 2024-09-19T05:45:22.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7fac027fc640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fabf006c330 0x7fabf006e7f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fabf8006fd0 tx=0x7fabf8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f61b7fff640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f619806c4b0 0x7f619806e970 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f61b815e6f0 tx=0x7f61a8001830 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe104016e50 con 0x7fe0fc074720 2024-09-19T05:45:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04007c20 con 0x7f79fc003b70 2024-09-19T05:45:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a04004c70 con 0x7f79fc003b70 2024-09-19T05:45:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0c0dfdb0 con 0x7f79fc003b70 2024-09-19T05:45:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6768043020 con 0x7f67740b5ce0 2024-09-19T05:45:22.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04004e20 con 0x7f79fc003b70 2024-09-19T05:45:22.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0c0d1ff0 con 0x7f79fc003b70 2024-09-19T05:45:22.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f61bd13c640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f61980747d0 0x7f6198076c30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7fac037fe640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fabf00746d0 0x7fabf0076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8d30048fe0 con 0x7f8d3004f170 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.522+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f95d80037b0 con 0x7f95e40c6ef0 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f95e2ffd640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95d006c2b0 0x7f95d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f95d8078320 con 0x7f95e40c6ef0 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f95e98fa640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95d006c2b0 0x7f95d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f95e2ffd640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f95d0075150 0x7f95d00775f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95d0077cc0 con 0x7f95d0075150 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f6770ff9640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.521+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/2271776474 wait complete. 2024-09-19T05:45:22.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f8bd9c1e640 1 Processor -- start 2024-09-19T05:45:22.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7f8bd9c1e640 1 -- start start 2024-09-19T05:45:22.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.523+0000 7fac037fe640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fabf00746d0 0x7fabf0076b30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f6773fff640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f67680ab9a0 con 0x7f67740b5ce0 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f6770ff9640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6760074650 0x7f6760076ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f676800af30 con 0x7f6760074650 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f67680b02a0 con 0x7f67740b5ce0 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f95d8076cb0 con 0x7f95e40c6ef0 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f95ea8fc640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f95d0075150 0x7f95d00775f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f95e98fa640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95d006c2b0 0x7f95d006e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95e40c4130 tx=0x7f95d4007330 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.524+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61b0007e80 con 0x7f61980747d0 2024-09-19T05:45:22.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f95ea8fc640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f95d0075150 0x7f95d00775f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabf400ca00 con 0x7fabf00746d0 2024-09-19T05:45:22.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f6773fff640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f676006c2b0 0x7f676006e770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6764007fd0 tx=0x7f6764007dc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a0c0d1da0 con 0x7f79fc003b70 2024-09-19T05:45:22.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8d3809e440 con 0x7f8d3004f170 2024-09-19T05:45:22.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7a04033440 con 0x7f79fc003b70 2024-09-19T05:45:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f6778eef640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6760074650 0x7f6760076ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== osd.5 v2:172.21.15.123:6800/2989037926 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95d0077cc0 con 0x7f95d0075150 2024-09-19T05:45:22.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f8bd9c1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bcc0dcf30 0x7f8bcc1815c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.525+0000 7f8bd9c1e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0e5fa0 0x7f8bcc049760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd9c1e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bcc02c0d0 0x7f8bcc0e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd9c1e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc044e60 con 0x7f8bcc0dcf30 2024-09-19T05:45:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8d2e7fc640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d1c06c380 0x7f8d1c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd9c1e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc04c450 con 0x7f8bcc02c0d0 2024-09-19T05:45:22.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd9c1e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc045140 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f7a02ffd640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f406c2e0 0x7f79f406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f7a040ab5e0 con 0x7f79fc003b70 2024-09-19T05:45:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f8d38107c40 con 0x7f8d3004f170 2024-09-19T05:45:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f7a02ffd640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f79f4074510 0x7f79f4076970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79f4077040 con 0x7f79f4074510 2024-09-19T05:45:22.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f7a040b2050 con 0x7f79fc003b70 2024-09-19T05:45:22.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd2ffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0e5fa0 0x7f8bcc049760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd2ffd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0e5fa0 0x7f8bcc049760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40822/0 (socket says 172.21.15.2:40822) 2024-09-19T05:45:22.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd2ffd640 1 -- 172.21.15.2:0/173207194 learned_addr learned my addr 172.21.15.2:0/173207194 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8d2e7fc640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1c074720 0x7f8d1c076b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8d3d89f640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d1c06c380 0x7f8d1c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd37fe640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bcc0dcf30 0x7f8bcc1815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd2ffd640 1 -- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bcc02c0d0 msgr2=0x7f8bcc0e1730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:45:22.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd2ffd640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bcc02c0d0 0x7f8bcc0e1730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd2ffd640 1 -- 172.21.15.2:0/173207194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bcc0dcf30 msgr2=0x7f8bcc1815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f8bd2ffd640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8bcc0dcf30 0x7f8bcc1815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8d3e0a0640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1c074720 0x7f8d1c076b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8bd2ffd640 1 -- 172.21.15.2:0/173207194 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0e0390 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.526+0000 7f7a12d9d640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f79f4074510 0x7f79f4076970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f6778eef640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6760074650 0x7f6760076ab0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8bd2ffd640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0e5fa0 0x7f8bcc049760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000c4d0 tx=0x7f8bc000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d38071e50 con 0x7f8d1c074720 2024-09-19T05:45:22.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f8d38108030 con 0x7f8d3004f170 2024-09-19T05:45:22.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0017070 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8d3e0a0640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1c074720 0x7f8d1c076b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc0e0670 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc0005ce0 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.527+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc0e0c30 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.528+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f676800af30 con 0x7f6760074650 2024-09-19T05:45:22.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.528+0000 7f7a11d9b640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f406c2e0 0x7f79f406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.528+0000 7f7a12d9d640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f79f4074510 0x7f79f4076970 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.528+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0007270 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.528+0000 7f8d3d89f640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d1c06c380 0x7f8d1c06e840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d28005880 tx=0x7f8d28009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.529+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d38071e50 con 0x7f8d1c074720 2024-09-19T05:45:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.530+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8bb4001030 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.530+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f79f4077040 con 0x7f79f4074510 2024-09-19T05:45:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.530+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8bc001a070 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.531+0000 7f8bbffff640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8bb806c300 0x7f8bb806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.531+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f8bc008a620 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.531+0000 7f8bd37fe640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8bb806c300 0x7f8bb806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.532+0000 7f8bd37fe640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8bb806c300 0x7f8bb806e7c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc04d7c0 tx=0x7f8bd406e030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.532+0000 7f8bbffff640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8bb80746a0 0x7f8bb8076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.532+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bb80771d0 con 0x7f8bb80746a0 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.532+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f8bc0091050 con 0x7f8bcc0e5fa0 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.532+0000 7f8bd3fff640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8bb80746a0 0x7f8bb8076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.533+0000 7f8bd3fff640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8bb80746a0 0x7f8bb8076b00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.533+0000 7f7a11d9b640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f406c2e0 0x7f79f406e7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0ccbc0 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.534+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8bb80771d0 con 0x7f8bb80746a0 2024-09-19T05:45:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.555+0000 7fe11679a640 1 -- 172.21.15.2:0/2159175595 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0f0002dd0 con 0x7fe0fc074720 2024-09-19T05:45:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.555+0000 7fe10cff9640 1 -- 172.21.15.2:0/2159175595 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe0f0002dd0 con 0x7fe0fc074720 2024-09-19T05:45:22.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.556+0000 7fe0f67fc640 1 -- 172.21.15.2:0/2159175595 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe0fc074720 msgr2=0x7fe0fc076b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.556+0000 7fe0f67fc640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fe0fc074720 0x7fe0fc076b80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.556+0000 7f61aeffd640 1 -- 172.21.15.2:0/1097240011 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6190002dd0 con 0x7f61980747d0 2024-09-19T05:45:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.556+0000 7fe0f67fc640 1 -- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe0fc06c380 msgr2=0x7fe0fc06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.556+0000 7fe0f67fc640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe0fc06c380 0x7fe0fc06e840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe108001460 tx=0x7fe108001820 comp rx=0 tx=0).stop 2024-09-19T05:45:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.556+0000 7fac09a3c640 1 -- 172.21.15.2:0/2435911212 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabe4002dd0 con 0x7fabf00746d0 2024-09-19T05:45:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.556+0000 7f61b57fa640 1 -- 172.21.15.2:0/1097240011 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6190002dd0 con 0x7f61980747d0 2024-09-19T05:45:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fe0f67fc640 1 -- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1100c5420 msgr2=0x7fe1100c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fe0f67fc640 1 --2- 172.21.15.2:0/2159175595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1100c5420 0x7fe1100c8340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c760 tx=0x7fe10400cc20 comp rx=0 tx=0).stop 2024-09-19T05:45:22.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fe114d37640 1 -- 172.21.15.2:0/2159175595 reap_dead start 2024-09-19T05:45:22.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fabebfff640 1 -- 172.21.15.2:0/2435911212 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fabe4002dd0 con 0x7fabf00746d0 2024-09-19T05:45:22.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fe0f67fc640 1 -- 172.21.15.2:0/2159175595 shutdown_connections 2024-09-19T05:45:22.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fe0f67fc640 1 -- 172.21.15.2:0/2159175595 >> 172.21.15.2:0/2159175595 conn(0x7fe110070fc0 msgr2=0x7fe1100ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f61980747d0 msgr2=0x7f6198076c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f61beb9f640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7f61980747d0 0x7f6198076c30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f619806c4b0 msgr2=0x7f619806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6754002dd0 con 0x7f6760074650 2024-09-19T05:45:22.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f61beb9f640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f619806c4b0 0x7f619806e970 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f61b815e6f0 tx=0x7f61a8001830 comp rx=0 tx=0).stop 2024-09-19T05:45:22.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61b80b30e0 msgr2=0x7f61b80c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f61beb9f640 1 --2- 172.21.15.2:0/1097240011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f61b80b30e0 0x7f61b80c8b80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f61b002a720 tx=0x7f61b00046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7f61bd13c640 1 -- 172.21.15.2:0/1097240011 reap_dead start 2024-09-19T05:45:22.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fe0f67fc640 1 -- 172.21.15.2:0/2159175595 shutdown_connections 2024-09-19T05:45:22.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.557+0000 7fe0f67fc640 1 -- 172.21.15.2:0/2159175595 wait complete. 2024-09-19T05:45:22.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95e406cf40 con 0x7f95d0075150 2024-09-19T05:45:22.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 shutdown_connections 2024-09-19T05:45:22.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 -- 172.21.15.2:0/2435911212 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fabf00746d0 msgr2=0x7fabf0076b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7fabf00746d0 0x7fabf0076b30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 >> 172.21.15.2:0/1097240011 conn(0x7f61b806d0e0 msgr2=0x7f61b815d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 shutdown_connections 2024-09-19T05:45:22.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f61beb9f640 1 -- 172.21.15.2:0/1097240011 wait complete. 2024-09-19T05:45:22.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f6770ff9640 1 -- 172.21.15.2:0/121416060 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6754002dd0 con 0x7f6760074650 2024-09-19T05:45:22.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 -- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fabf006c330 msgr2=0x7fabf006e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fabf006c330 0x7fabf006e7f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fabf8006fd0 tx=0x7fabf8008040 comp rx=0 tx=0).stop 2024-09-19T05:45:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 -- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 msgr2=0x7fac04159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 --2- 172.21.15.2:0/2435911212 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac04159320 0x7fac04159920 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fabf400e9b0 tx=0x7fabf400ee70 comp rx=0 tx=0).stop 2024-09-19T05:45:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fac037fe640 1 -- 172.21.15.2:0/2435911212 reap_dead start 2024-09-19T05:45:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 -- 172.21.15.2:0/2435911212 shutdown_connections 2024-09-19T05:45:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 -- 172.21.15.2:0/2435911212 >> 172.21.15.2:0/2435911212 conn(0x7fac0406bc10 msgr2=0x7fac04160ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f95e2ffd640 1 -- 172.21.15.2:0/559477579 <== osd.5 v2:172.21.15.123:6800/2989037926 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f95e406cf40 con 0x7f95d0075150 2024-09-19T05:45:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6760074650 msgr2=0x7f6760076ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f677a952640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f6760074650 0x7f6760076ab0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7fabe9ffb640 1 -- 172.21.15.2:0/2435911212 shutdown_connections 2024-09-19T05:45:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.558+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f676006c2b0 msgr2=0x7f676006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f677a952640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f676006c2b0 0x7f676006e770 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6764007fd0 tx=0x7f6764007dc0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7fabe9ffb640 1 -- 172.21.15.2:0/2435911212 wait complete. 2024-09-19T05:45:22.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b5ce0 msgr2=0x7f6774148a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f677a952640 1 --2- 172.21.15.2:0/121416060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f67740b5ce0 0x7f6774148a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6768031000 tx=0x7f67680046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f6778eef640 1 -- 172.21.15.2:0/121416060 reap_dead start 2024-09-19T05:45:22.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f95d0075150 msgr2=0x7f95d00775f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f95e0ff9640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7f95d0075150 0x7f95d00775f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 shutdown_connections 2024-09-19T05:45:22.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 >> 172.21.15.2:0/121416060 conn(0x7f6774067950 msgr2=0x7f67740ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95d006c2b0 msgr2=0x7f95d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f95e0ff9640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f95d006c2b0 0x7f95d006e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95e40c4130 tx=0x7f95d4007330 comp rx=0 tx=0).stop 2024-09-19T05:45:22.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e40c6ef0 msgr2=0x7f95e41597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f95e0ff9640 1 --2- 172.21.15.2:0/559477579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f95e40c6ef0 0x7f95e41597f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000c00 tx=0x7f95d80046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.559+0000 7f95ea8fc640 1 -- 172.21.15.2:0/559477579 reap_dead start 2024-09-19T05:45:22.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 shutdown_connections 2024-09-19T05:45:22.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f677a952640 1 -- 172.21.15.2:0/121416060 wait complete. 2024-09-19T05:45:22.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8d3004e460 con 0x7f8d1c074720 2024-09-19T05:45:22.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 shutdown_connections 2024-09-19T05:45:22.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 >> 172.21.15.2:0/559477579 conn(0x7f95e40ccc60 msgr2=0x7f95e40cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.621 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 shutdown_connections 2024-09-19T05:45:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f8d2e7fc640 1 -- 172.21.15.2:0/109025167 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8d3004e460 con 0x7f8d1c074720 2024-09-19T05:45:22.622 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.560+0000 7f95e0ff9640 1 -- 172.21.15.2:0/559477579 wait complete. 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.561+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1c074720 msgr2=0x7f8d1c076b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.561+0000 7f8d3e8a1640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7f8d1c074720 0x7f8d1c076b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.561+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d1c06c380 msgr2=0x7f8d1c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.561+0000 7f8d3e8a1640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8d1c06c380 0x7f8d1c06e840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d28005880 tx=0x7f8d28009040 comp rx=0 tx=0).stop 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.561+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d3004f170 msgr2=0x7f8d300396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.561+0000 7f8d3e8a1640 1 --2- 172.21.15.2:0/109025167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d3004f170 0x7f8d300396a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d380504d0 tx=0x7f8d38069890 comp rx=0 tx=0).stop 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.561+0000 7f8d3e0a0640 1 -- 172.21.15.2:0/109025167 reap_dead start 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.562+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 shutdown_connections 2024-09-19T05:45:22.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.562+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 >> 172.21.15.2:0/109025167 conn(0x7f8d30054060 msgr2=0x7f8d300d4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.562+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a0c052fa0 con 0x7f79f4074510 2024-09-19T05:45:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.562+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 shutdown_connections 2024-09-19T05:45:22.624 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.562+0000 7f8d3e8a1640 1 -- 172.21.15.2:0/109025167 wait complete. 2024-09-19T05:45:22.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.562+0000 7f7a02ffd640 1 -- 172.21.15.2:0/1731446614 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7a0c052fa0 con 0x7f79f4074510 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.563+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f79f4074510 msgr2=0x7f79f4076970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.563+0000 7f7a18b31640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f79f4074510 0x7f79f4076970 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.563+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f406c2e0 msgr2=0x7f79f406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.564+0000 7f7a18b31640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f79f406c2e0 0x7f79f406e7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0ccbc0 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.564+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 msgr2=0x7f7a0c0093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.564+0000 7f7a18b31640 1 --2- 172.21.15.2:0/1731446614 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f79fc003b70 0x7f7a0c0093f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400c2b0 tx=0x7f7a04001790 comp rx=0 tx=0).stop 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.564+0000 7f7a12d9d640 1 -- 172.21.15.2:0/1731446614 reap_dead start 2024-09-19T05:45:22.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.564+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8bb4002dd0 con 0x7f8bb80746a0 2024-09-19T05:45:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.564+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 shutdown_connections 2024-09-19T05:45:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.564+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 >> 172.21.15.2:0/1731446614 conn(0x7f7a0c051e50 msgr2=0x7f7a0c04d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.565+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 shutdown_connections 2024-09-19T05:45:22.627 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.565+0000 7f8bbffff640 1 -- 172.21.15.2:0/173207194 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8bb4002dd0 con 0x7f8bb80746a0 2024-09-19T05:45:22.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.565+0000 7f7a18b31640 1 -- 172.21.15.2:0/1731446614 wait complete. 2024-09-19T05:45:22.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.565+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8bb80746a0 msgr2=0x7f8bb8076b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.566+0000 7f8bd9c1e640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f8bb80746a0 0x7f8bb8076b00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.566+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8bb806c300 msgr2=0x7f8bb806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.566+0000 7f8bd9c1e640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8bb806c300 0x7f8bb806e7c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc04d7c0 tx=0x7f8bd406e030 comp rx=0 tx=0).stop 2024-09-19T05:45:22.629 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.566+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0e5fa0 msgr2=0x7f8bcc049760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.566+0000 7f8bd9c1e640 1 --2- 172.21.15.2:0/173207194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8bcc0e5fa0 0x7f8bcc049760 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000c4d0 tx=0x7f8bc000c990 comp rx=0 tx=0).stop 2024-09-19T05:45:22.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.566+0000 7f8bd3fff640 1 -- 172.21.15.2:0/173207194 reap_dead start 2024-09-19T05:45:22.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.567+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 shutdown_connections 2024-09-19T05:45:22.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.567+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 >> 172.21.15.2:0/173207194 conn(0x7f8bcc04fc40 msgr2=0x7f8bcc0e5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.567+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 shutdown_connections 2024-09-19T05:45:22.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.567+0000 7f8bd9c1e640 1 -- 172.21.15.2:0/173207194 wait complete. 2024-09-19T05:45:22.631 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:22.631 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T05:45:22.632 INFO:teuthology.orchestra.run.smithi002.stdout:64424509480 2024-09-19T05:45:22.632 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T05:45:22.633 INFO:teuthology.orchestra.run.smithi002.stdout:60129542184 2024-09-19T05:45:22.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T05:45:22.634 INFO:teuthology.orchestra.run.smithi002.stdout:60129542184 2024-09-19T05:45:22.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T05:45:22.635 INFO:teuthology.orchestra.run.smithi002.stdout:335007449096 2024-09-19T05:45:22.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T05:45:22.636 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:22.637 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T05:45:22.637 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:22.637 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T05:45:22.638 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:22.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T05:45:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.843+0000 7f023e67b640 1 Processor -- start 2024-09-19T05:45:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.843+0000 7f023e67b640 1 -- start start 2024-09-19T05:45:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.845+0000 7f023e67b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0238072650 0x7f0238071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.845+0000 7f023e67b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023815a130 con 0x7f0238150d20 2024-09-19T05:45:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.845+0000 7f023e67b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f023815a3a0 con 0x7f0238072090 2024-09-19T05:45:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.845+0000 7f023e67b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f02380c6990 con 0x7f0238072650 2024-09-19T05:45:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.845+0000 7f02377fe640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0238072650 0x7f0238071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.846+0000 7f023cc18640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0238072090 0x7f023806ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:45016/0 (socket says 172.21.15.2:45016) 2024-09-19T05:45:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.846+0000 7f023cc18640 1 -- 172.21.15.2:0/2763716719 learned_addr learned my addr 172.21.15.2:0/2763716719 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533253877 0 0) 0x7f023815a3a0 con 0x7f0238072090 2024-09-19T05:45:22.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0220003520 con 0x7f0238072090 2024-09-19T05:45:22.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.845+0000 7f585759e640 1 Processor -- start 2024-09-19T05:45:22.849 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.845+0000 7f585759e640 1 -- start start 2024-09-19T05:45:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 660871280 0 0) 0x7f0220003520 con 0x7f0238072090 2024-09-19T05:45:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 >> v2:172.21.15.123:3300/0 conn(0x7f0238072650 msgr2=0x7f0238071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 --2- 172.21.15.2:0/2763716719 >> v2:172.21.15.123:3300/0 conn(0x7f0238072650 0x7f0238071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f585759e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5850006870 0x7f58500453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f585759e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5850035d50 0x7f5850006c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f585759e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f5850046f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f585759e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f5850046760 con 0x7f5850035d50 2024-09-19T05:45:22.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f585759e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5850045ff0 con 0x7f5850006870 2024-09-19T05:45:22.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f585759e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f58500462d0 con 0x7f5850046b30 2024-09-19T05:45:22.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 >> v1:172.21.15.2:6789/0 conn(0x7f0238150d20 legacy=0x7f02380c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f264dae6640 1 Processor -- start 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023815a680 con 0x7f0238072090 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f022c001650 con 0x7f0238072090 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 >> v1:172.21.15.123:6789/0 conn(0x7f0238072090 legacy=0x7f023806ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0220003a10 0x7f0220003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220004390 0x7f0220004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f022000ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f02380c6990 con 0x7f0220004390 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0220010540 con 0x7f0220003a10 2024-09-19T05:45:22.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0220010740 con 0x7f0220004fe0 2024-09-19T05:45:22.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f023cc18640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f022000ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.847+0000 7f264dae6640 1 -- start start 2024-09-19T05:45:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f023cc18640 1 -- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0220003a10 msgr2=0x7f0220003e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T05:45:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f023cc18640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0220003a10 0x7f0220003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f023cc18640 1 -- 172.21.15.2:0/2763716719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220004390 msgr2=0x7f0220004800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:45:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f023cc18640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0220004390 0x7f0220004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f023cc18640 1 -- 172.21.15.2:0/2763716719 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023815a680 con 0x7f0220004fe0 2024-09-19T05:45:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f5855d9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5850035d50 0x7f5850006c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f5855d9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5850035d50 0x7f5850006c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54502/0 (socket says 172.21.15.2:54502) 2024-09-19T05:45:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f5855d9b640 1 -- 172.21.15.2:0/4100015420 learned_addr learned my addr 172.21.15.2:0/4100015420 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f585659c640 1 --2- 172.21.15.2:0/4100015420 >> v2:172.21.15.123:3300/0 conn(0x7f5850006870 0x7f58500453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.848+0000 7f5856d9d640 1 --2- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f5850046f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f023cc18640 1 -- 172.21.15.2:0/2763716719 reap_dead start 2024-09-19T05:45:22.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f585659c640 1 -- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 msgr2=0x7f5850046f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f264dae6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f264806d490 0x7f26480c9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f264dae6640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26480cb070 con 0x7f26480c91a0 2024-09-19T05:45:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f264dae6640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26480ca0a0 con 0x7f2648159120 2024-09-19T05:45:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f264dae6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f264806cd20 con 0x7f264806d490 2024-09-19T05:45:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f26477fe640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2648159120 0x7f264815ef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.2:45032/0 (socket says 172.21.15.2:45032) 2024-09-19T05:45:22.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f023cc18640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f022000ffd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f022c00c260 tx=0x7f0220005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecff79f03d42d49c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f26477fe640 1 -- 172.21.15.2:0/2053840697 learned_addr learned my addr 172.21.15.2:0/2053840697 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f2647fff640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f264806d490 0x7f26480c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c036070 con 0x7f0220004fe0 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022c005b10 con 0x7f0220004fe0 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f0236ffd640 1 -- 172.21.15.2:0/2763716719 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c007d50 con 0x7f0220004fe0 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f585659c640 1 --2- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f5850046f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f585659c640 1 -- 172.21.15.2:0/4100015420 >> v2:172.21.15.2:3300/0 conn(0x7f5850035d50 msgr2=0x7f5850006c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f585659c640 1 --2- 172.21.15.2:0/4100015420 >> v2:172.21.15.2:3300/0 conn(0x7f5850035d50 0x7f5850006c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f585659c640 1 -- 172.21.15.2:0/4100015420 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5850036700 con 0x7f5850006870 2024-09-19T05:45:22.858 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.849+0000 7f5856d9d640 1 --2- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f5850046f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1399489978 0 0) 0x7f26480ca0a0 con 0x7f2648159120 2024-09-19T05:45:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2630003520 con 0x7f2648159120 2024-09-19T05:45:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3119532906 0 0) 0x7f26480cb070 con 0x7f26480c91a0 2024-09-19T05:45:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26480ca0a0 con 0x7f26480c91a0 2024-09-19T05:45:22.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f5855d9b640 1 --2- 172.21.15.2:0/4100015420 >> v2:172.21.15.2:3300/0 conn(0x7f5850035d50 0x7f5850006c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2103986653 0 0) 0x7f2630003520 con 0x7f2648159120 2024-09-19T05:45:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f264806d490 msgr2=0x7f26480c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f264806d490 0x7f26480c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 >> v1:172.21.15.2:6789/0 conn(0x7f26480c91a0 legacy=0x7f26480cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26480718a0 con 0x7f2648159120 2024-09-19T05:45:22.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f585659c640 1 --2- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850006870 0x7f58500453c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f585000da90 tx=0x7f584802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e33b1776ec1a26f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f263c001690 con 0x7f2648159120 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 >> v1:172.21.15.123:6789/0 conn(0x7f2648159120 legacy=0x7f264815ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2630003b70 0x7f2630003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2630004800 0x7f2630004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26300051c0 0x7f263000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f264806cd20 con 0x7f2630003b70 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f263000bfb0 con 0x7f2630004800 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.850+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f263000c180 con 0x7f26300051c0 2024-09-19T05:45:22.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f2647fff640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f264806d490 0x7f26480c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f023e67b640 1 -- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 msgr2=0x7f022000ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f023e67b640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f022000ffd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f022c00c260 tx=0x7f0220005830 comp rx=0 tx=0).stop 2024-09-19T05:45:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f023e67b640 1 -- 172.21.15.2:0/2763716719 shutdown_connections 2024-09-19T05:45:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f023e67b640 1 --2- 172.21.15.2:0/2763716719 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f022000ffd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f023e67b640 1 -- 172.21.15.2:0/2763716719 >> 172.21.15.2:0/2763716719 conn(0x7f023806bc10 msgr2=0x7f023806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f26477fe640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2630003b70 0x7f2630003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f2647fff640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26300051c0 0x7f263000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f2646ffd640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2630004800 0x7f2630004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f585559a640 1 -- 172.21.15.2:0/4100015420 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848037070 con 0x7f5850006870 2024-09-19T05:45:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f585559a640 1 -- 172.21.15.2:0/4100015420 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5848032440 con 0x7f5850006870 2024-09-19T05:45:22.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.851+0000 7f585559a640 1 -- 172.21.15.2:0/4100015420 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848032810 con 0x7f5850006870 2024-09-19T05:45:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f26477fe640 1 -- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26300051c0 msgr2=0x7f263000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f26477fe640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f26300051c0 0x7f263000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f26477fe640 1 -- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2630004800 msgr2=0x7f2630004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f26477fe640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2630004800 0x7f2630004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f26477fe640 1 -- 172.21.15.2:0/2053840697 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2630003520 con 0x7f2630003b70 2024-09-19T05:45:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f023e67b640 1 -- 172.21.15.2:0/2763716719 shutdown_connections 2024-09-19T05:45:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f96fffff640 1 Processor -- start 2024-09-19T05:45:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f06a98dd640 1 Processor -- start 2024-09-19T05:45:22.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f06630f8640 1 Processor -- start 2024-09-19T05:45:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f06a98dd640 1 -- start start 2024-09-19T05:45:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f4988b2a640 1 Processor -- start 2024-09-19T05:45:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f023e67b640 1 -- 172.21.15.2:0/2763716719 wait complete. 2024-09-19T05:45:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f585759e640 1 -- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850006870 msgr2=0x7f58500453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f585759e640 1 --2- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850006870 0x7f58500453c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f585000da90 tx=0x7f584802a1c0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f26477fe640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2630003b70 0x7f2630003f50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c0e0 tx=0x7f2630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7d1b62e1315855 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c007c40 con 0x7f2630003b70 2024-09-19T05:45:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263c005ad0 con 0x7f2630003b70 2024-09-19T05:45:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f26467fc640 1 -- 172.21.15.2:0/2053840697 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c007070 con 0x7f2630003b70 2024-09-19T05:45:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f585759e640 1 -- 172.21.15.2:0/4100015420 shutdown_connections 2024-09-19T05:45:22.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f585759e640 1 --2- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f5850046f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f585759e640 1 --2- 172.21.15.2:0/4100015420 >> v2:172.21.15.2:3300/0 conn(0x7f5850035d50 0x7f5850006c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f585759e640 1 --2- 172.21.15.2:0/4100015420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850006870 0x7f58500453c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f585759e640 1 -- 172.21.15.2:0/4100015420 >> 172.21.15.2:0/4100015420 conn(0x7f58500504e0 msgr2=0x7f58500508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f585759e640 1 -- 172.21.15.2:0/4100015420 shutdown_connections 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f96fffff640 1 -- start start 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f96fffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f96f8038e40 0x7f96f804f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f96fffff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f800e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96fffff640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96f80e61f0 con 0x7f96f8039220 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96fffff640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f96f80e64d0 con 0x7f96f8038e40 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96fffff640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96f80e67b0 con 0x7f96f800e3b0 2024-09-19T05:45:22.868 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96feffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f96f8038e40 0x7f96f804f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f800e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96feffd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f96f8038e40 0x7f96f804f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54526/0 (socket says 172.21.15.2:54526) 2024-09-19T05:45:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f800e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40888/0 (socket says 172.21.15.2:40888) 2024-09-19T05:45:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f96feffd640 1 -- 172.21.15.2:0/741977145 learned_addr learned my addr 172.21.15.2:0/741977145 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f023e67b640 1 Processor -- start 2024-09-19T05:45:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.852+0000 7f4988b2a640 1 -- start start 2024-09-19T05:45:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f4988b2a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f49840cb150 0x7f4984162810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f498259c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f49840b2c80 0x7f4984156e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54100/0 (socket says 172.21.15.2:54100) 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f4988b2a640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f498406e5e0 con 0x7f49840b2c80 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f4988b2a640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f498406e880 con 0x7f49840cc650 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f4988b2a640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4984157590 con 0x7f49840cb150 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f06a98dd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c045570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f06a98dd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f069c006220 0x7f069c045ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f06a98dd640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069c0df620 con 0x7f069c0d8b40 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f06a98dd640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f069c04bff0 con 0x7f069c006220 2024-09-19T05:45:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f06a98dd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f069c04c2d0 con 0x7f069c0df150 2024-09-19T05:45:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f06a2ffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f069c0d8b40 0x7f069c005b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54094/0 (socket says 172.21.15.2:54094) 2024-09-19T05:45:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f06a2ffd640 1 -- 172.21.15.2:0/1497181473 learned_addr learned my addr 172.21.15.2:0/1497181473 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823329499 0 0) 0x7f069c0df620 con 0x7f069c0d8b40 2024-09-19T05:45:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f068c003520 con 0x7f069c0d8b40 2024-09-19T05:45:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f06a27fc640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c045570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f023e67b640 1 -- start start 2024-09-19T05:45:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f06a37fe640 1 --2- 172.21.15.2:0/1497181473 >> v2:172.21.15.2:3300/0 conn(0x7f069c006220 0x7f069c045ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f498259c640 1 -- 172.21.15.2:0/1394608827 learned_addr learned my addr 172.21.15.2:0/1394608827 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f4982d9d640 1 --2- 172.21.15.2:0/1394608827 >> v2:172.21.15.123:3300/0 conn(0x7f49840cb150 0x7f4984162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f4982d9d640 1 -- 172.21.15.2:0/1394608827 >> v1:172.21.15.123:6789/0 conn(0x7f49840cc650 legacy=0x7f498406cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f4982d9d640 1 -- 172.21.15.2:0/1394608827 >> v1:172.21.15.2:6789/0 conn(0x7f49840b2c80 legacy=0x7f4984156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f4982d9d640 1 -- 172.21.15.2:0/1394608827 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498415a710 con 0x7f49840cb150 2024-09-19T05:45:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f585759e640 1 -- 172.21.15.2:0/4100015420 wait complete. 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f264dae6640 1 -- 172.21.15.2:0/2053840697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2630003b70 msgr2=0x7f2630003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f264dae6640 1 --2- 172.21.15.2:0/2053840697 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2630003b70 0x7f2630003f50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c0e0 tx=0x7f2630005830 comp rx=0 tx=0).stop 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.853+0000 7f06630f8640 1 -- start start 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.854+0000 7f06630f8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c159470 0x7f065c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f06630f8640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f065c06f690 0x7f065c0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f06630f8640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f065c071e10 con 0x7f065c159090 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f06630f8640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f065c06fb40 con 0x7f065c06f690 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f06630f8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f065c15f160 con 0x7f065c159470 2024-09-19T05:45:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0653fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c159470 0x7f065c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0653fff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c159470 0x7f065c06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40892/0 (socket says 172.21.15.2:40892) 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0653fff640 1 -- 172.21.15.2:0/3468230610 learned_addr learned my addr 172.21.15.2:0/3468230610 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0661695640 1 --2- 172.21.15.2:0/3468230610 >> v2:172.21.15.2:3300/0 conn(0x7f065c06f690 0x7f065c0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f585759e640 1 Processor -- start 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0653fff640 1 -- 172.21.15.2:0/3468230610 >> v2:172.21.15.2:3300/0 conn(0x7f065c06f690 msgr2=0x7f065c0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0653fff640 1 --2- 172.21.15.2:0/3468230610 >> v2:172.21.15.2:3300/0 conn(0x7f065c06f690 0x7f065c0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0653fff640 1 -- 172.21.15.2:0/3468230610 >> v1:172.21.15.123:6789/0 conn(0x7f065c159090 legacy=0x7f065c0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f0653fff640 1 -- 172.21.15.2:0/3468230610 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c15ff80 con 0x7f065c159470 2024-09-19T05:45:22.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f585759e640 1 -- start start 2024-09-19T05:45:22.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f2647fff640 1 -- 172.21.15.2:0/2053840697 reap_dead start 2024-09-19T05:45:22.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f96fdffb640 1 -- 172.21.15.2:0/741977145 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768793688 0 0) 0x7f96f80e61f0 con 0x7f96f8039220 2024-09-19T05:45:22.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f96fdffb640 1 -- 172.21.15.2:0/741977145 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96e8003520 con 0x7f96f8039220 2024-09-19T05:45:22.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f96ff7fe640 1 -- 172.21.15.2:0/741977145 >> v2:172.21.15.2:3300/0 conn(0x7f96f8038e40 msgr2=0x7f96f804f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f96ff7fe640 1 --2- 172.21.15.2:0/741977145 >> v2:172.21.15.2:3300/0 conn(0x7f96f8038e40 0x7f96f804f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f96ff7fe640 1 -- 172.21.15.2:0/741977145 >> v1:172.21.15.123:6789/0 conn(0x7f96f8039220 legacy=0x7f96f80501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.855+0000 7f96ff7fe640 1 -- 172.21.15.2:0/741977145 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f80e6a90 con 0x7f96f800e3b0 2024-09-19T05:45:22.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f264dae6640 1 -- 172.21.15.2:0/2053840697 shutdown_connections 2024-09-19T05:45:22.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f264dae6640 1 -- 172.21.15.2:0/2053840697 >> 172.21.15.2:0/2053840697 conn(0x7f264806bdf0 msgr2=0x7f264806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f023e67b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f02380cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f264dae6640 1 -- 172.21.15.2:0/2053840697 shutdown_connections 2024-09-19T05:45:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f264dae6640 1 -- 172.21.15.2:0/2053840697 wait complete. 2024-09-19T05:45:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f023e67b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0238158960 0x7f0238144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f023e67b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02380cc360 0x7f02380cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f4982d9d640 1 --2- 172.21.15.2:0/1394608827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840cb150 0x7f4984162810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49840cc2b0 tx=0x7f497c02ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f065dbfb9d0711ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f023e67b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f02380c8cb0 con 0x7f02380cc360 2024-09-19T05:45:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f498159a640 1 -- 172.21.15.2:0/1394608827 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c02f5c0 con 0x7f49840cb150 2024-09-19T05:45:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f585759e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5850006870 0x7f58500e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f023e67b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f023815ac40 con 0x7f0238158960 2024-09-19T05:45:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f498159a640 1 -- 172.21.15.2:0/1394608827 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c02f760 con 0x7f49840cb150 2024-09-19T05:45:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f498159a640 1 -- 172.21.15.2:0/1394608827 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c02fb30 con 0x7f49840cb150 2024-09-19T05:45:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f02377fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0238158960 0x7f0238144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f023e67b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f02380c80a0 con 0x7f0220004fe0 2024-09-19T05:45:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.856+0000 7f02377fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0238158960 0x7f0238144c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37112/0 (socket says 172.21.15.2:37112) 2024-09-19T05:45:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585759e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850035d50 0x7f58500e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f0237fff640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f02380cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5850006870 0x7f58500e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585659c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5850006870 0x7f58500e3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54568/0 (socket says 172.21.15.2:54568) 2024-09-19T05:45:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585759e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f58501748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f02377fe640 1 -- 172.21.15.2:0/2963411070 learned_addr learned my addr 172.21.15.2:0/2963411070 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f5855d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850035d50 0x7f58500e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f5855d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850035d50 0x7f58500e46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37126/0 (socket says 172.21.15.2:37126) 2024-09-19T05:45:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2156565921 0 0) 0x7f068c003520 con 0x7f069c0d8b40 2024-09-19T05:45:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f96ff7fe640 1 --2- 172.21.15.2:0/741977145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f800e840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97000504d0 tx=0x7f97000a6ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab93b93db080c176 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585759e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f58500e3890 con 0x7f5850006870 2024-09-19T05:45:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585759e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58500e35b0 con 0x7f5850035d50 2024-09-19T05:45:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 msgr2=0x7f069c045570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585759e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5850045a50 con 0x7f5850046b30 2024-09-19T05:45:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f0653fff640 1 --2- 172.21.15.2:0/3468230610 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c159470 0x7f065c06f150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1cc118cb21f98b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06a1ffb640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c045570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 >> v2:172.21.15.2:3300/0 conn(0x7f069c006220 msgr2=0x7f069c045ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06a1ffb640 1 --2- 172.21.15.2:0/1497181473 >> v2:172.21.15.2:3300/0 conn(0x7f069c006220 0x7f069c045ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f023cc18640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02380cc360 0x7f02380cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f585659c640 1 -- 172.21.15.2:0/358390353 learned_addr learned my addr 172.21.15.2:0/358390353 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f96fdffb640 1 -- 172.21.15.2:0/741977145 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000a89c0 con 0x7f96f800e3b0 2024-09-19T05:45:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f96fdffb640 1 -- 172.21.15.2:0/741977145 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97000a8b60 con 0x7f96f800e3b0 2024-09-19T05:45:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f4988b2a640 1 -- 172.21.15.2:0/1394608827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840cb150 msgr2=0x7f4984162810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1394608827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840cb150 0x7f4984162810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f49840cc2b0 tx=0x7f497c02ecc0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f5855d9b640 1 -- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 msgr2=0x7f58501748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f5855d9b640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f58501748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c04b5b0 con 0x7f069c0d8b40 2024-09-19T05:45:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f5855d9b640 1 -- 172.21.15.2:0/358390353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5850006870 msgr2=0x7f58500e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f5855d9b640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5850006870 0x7f58500e3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f5855d9b640 1 -- 172.21.15.2:0/358390353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58500e4c10 con 0x7f5850035d50 2024-09-19T05:45:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06537fe640 1 -- 172.21.15.2:0/3468230610 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658037070 con 0x7f065c159470 2024-09-19T05:45:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06537fe640 1 -- 172.21.15.2:0/3468230610 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0658032640 con 0x7f065c159470 2024-09-19T05:45:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.857+0000 7f06537fe640 1 -- 172.21.15.2:0/3468230610 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065802ae60 con 0x7f065c159470 2024-09-19T05:45:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f264dae6640 1 Processor -- start 2024-09-19T05:45:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f0694001670 con 0x7f069c0d8b40 2024-09-19T05:45:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a27fc640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c045570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a37fe640 1 --2- 172.21.15.2:0/1497181473 >> v2:172.21.15.2:3300/0 conn(0x7f069c006220 0x7f069c045ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f02377fe640 1 -- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 msgr2=0x7f02380cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f4988b2a640 1 -- 172.21.15.2:0/1394608827 shutdown_connections 2024-09-19T05:45:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1394608827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840cb150 0x7f4984162810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f4988b2a640 1 -- 172.21.15.2:0/1394608827 >> 172.21.15.2:0/1394608827 conn(0x7f498406fed0 msgr2=0x7f49840702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f02377fe640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f02380cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f02377fe640 1 -- 172.21.15.2:0/2963411070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02380cc360 msgr2=0x7f02380cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f02377fe640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02380cc360 0x7f02380cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.891 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f02377fe640 1 -- 172.21.15.2:0/2963411070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02381f5910 con 0x7f0238158960 2024-09-19T05:45:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f264dae6640 1 -- start start 2024-09-19T05:45:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f0237fff640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f02380cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f023cc18640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02380cc360 0x7f02380cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.892 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 >> v1:172.21.15.2:6789/0 conn(0x7f069c0d8b40 legacy=0x7f069c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f96fdffb640 1 -- 172.21.15.2:0/741977145 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000b0420 con 0x7f96f800e3b0 2024-09-19T05:45:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f585659c640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5850006870 0x7f58500e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a1ffb640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f068c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.893 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a1ffb640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068c003fa0 0x7f068c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f5855d9b640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850035d50 0x7f58500e46d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f584c00e890 tx=0x7f584c00ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06630f8640 1 -- 172.21.15.2:0/3468230610 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c159470 msgr2=0x7f065c06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06630f8640 1 --2- 172.21.15.2:0/3468230610 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c159470 0x7f065c06f150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0658009870 tx=0x7f065802a310 comp rx=0 tx=0).stop 2024-09-19T05:45:22.894 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a1ffb640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f068c004d40 0x7f068c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f4988b2a640 1 -- 172.21.15.2:0/1394608827 shutdown_connections 2024-09-19T05:45:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f4988b2a640 1 -- 172.21.15.2:0/1394608827 wait complete. 2024-09-19T05:45:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06630f8640 1 -- 172.21.15.2:0/3468230610 shutdown_connections 2024-09-19T05:45:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06630f8640 1 --2- 172.21.15.2:0/3468230610 >> v2:172.21.15.2:3300/0 conn(0x7f065c06f690 0x7f065c0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.895 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f06a27fc640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068c003fa0 0x7f068c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06630f8640 1 --2- 172.21.15.2:0/3468230610 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c159470 0x7f065c06f150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06630f8640 1 -- 172.21.15.2:0/3468230610 >> 172.21.15.2:0/3468230610 conn(0x7f065c06bc10 msgr2=0x7f065c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f06630f8640 1 -- 172.21.15.2:0/3468230610 shutdown_connections 2024-09-19T05:45:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f06630f8640 1 -- 172.21.15.2:0/3468230610 wait complete. 2024-09-19T05:45:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f069c04bff0 con 0x7f068c004d40 2024-09-19T05:45:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.858+0000 7f06a2ffd640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f068c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f264dae6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26480c7e10 0x7f26480c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f06a37fe640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f068c004d40 0x7f068c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f264dae6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26481f5940 0x7f26481585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f264dae6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2648158b10 0x7f2648161e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f264dae6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2648072100 con 0x7f26480c7e10 2024-09-19T05:45:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f264dae6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2648071e20 con 0x7f26481f5940 2024-09-19T05:45:22.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f264dae6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2648159500 con 0x7f2648158b10 2024-09-19T05:45:22.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f96fffff640 1 -- 172.21.15.2:0/741977145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 msgr2=0x7f96f800e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f96fffff640 1 --2- 172.21.15.2:0/741977145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f800e840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97000504d0 tx=0x7f97000a6ba0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.898 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f2646ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26481f5940 0x7f26481585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f2646ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26481f5940 0x7f26481585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37142/0 (socket says 172.21.15.2:37142) 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f2646ffd640 1 -- 172.21.15.2:0/2705564798 learned_addr learned my addr 172.21.15.2:0/2705564798 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f2647fff640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2648158b10 0x7f2648161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f26477fe640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26480c7e10 0x7f26480c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f96fffff640 1 -- 172.21.15.2:0/741977145 shutdown_connections 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f96fffff640 1 --2- 172.21.15.2:0/741977145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f800e840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f96fffff640 1 --2- 172.21.15.2:0/741977145 >> v2:172.21.15.2:3300/0 conn(0x7f96f8038e40 0x7f96f804f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f96fffff640 1 -- 172.21.15.2:0/741977145 >> 172.21.15.2:0/741977145 conn(0x7f96f8054450 msgr2=0x7f96f8054860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f069c04c2d0 con 0x7f068c003fa0 2024-09-19T05:45:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f068c00bb30 con 0x7f069c0df150 2024-09-19T05:45:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f84859fc640 1 Processor -- start 2024-09-19T05:45:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f96fffff640 1 -- 172.21.15.2:0/741977145 shutdown_connections 2024-09-19T05:45:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f2647fff640 1 -- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26481f5940 msgr2=0x7f26481585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f2647fff640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26481f5940 0x7f26481585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f2647fff640 1 -- 172.21.15.2:0/2705564798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26480c7e10 msgr2=0x7f26480c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f5846ffd640 1 -- 172.21.15.2:0/358390353 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c017070 con 0x7f5850035d50 2024-09-19T05:45:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f5846ffd640 1 -- 172.21.15.2:0/358390353 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584c00c8e0 con 0x7f5850035d50 2024-09-19T05:45:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f5846ffd640 1 -- 172.21.15.2:0/358390353 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c00cc90 con 0x7f5850035d50 2024-09-19T05:45:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f585759e640 1 -- 172.21.15.2:0/358390353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5850181300 con 0x7f5850035d50 2024-09-19T05:45:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f06a2ffd640 1 -- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068c003fa0 msgr2=0x7f068c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f06a2ffd640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068c003fa0 0x7f068c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f06a2ffd640 1 -- 172.21.15.2:0/1497181473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f068c004d40 msgr2=0x7f068c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f06a2ffd640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f068c004d40 0x7f068c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f585759e640 1 -- 172.21.15.2:0/358390353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5850184bd0 con 0x7f5850035d50 2024-09-19T05:45:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f06a2ffd640 1 -- 172.21.15.2:0/1497181473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c003520 con 0x7f069c0df150 2024-09-19T05:45:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.859+0000 7f84859fc640 1 -- start start 2024-09-19T05:45:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f96fffff640 1 -- 172.21.15.2:0/741977145 wait complete. 2024-09-19T05:45:22.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f06630f8640 1 Processor -- start 2024-09-19T05:45:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f4988b2a640 1 Processor -- start 2024-09-19T05:45:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f06630f8640 1 -- start start 2024-09-19T05:45:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f2647fff640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26480c7e10 0x7f26480c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f2647fff640 1 -- 172.21.15.2:0/2705564798 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26480cd6f0 con 0x7f2648158b10 2024-09-19T05:45:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.860+0000 7f26477fe640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26480c7e10 0x7f26480c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f2647fff640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2648158b10 0x7f2648161e60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2648072480 tx=0x7f2638011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84859fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f8478048fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f262ffff640 1 -- 172.21.15.2:0/2705564798 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263801c070 con 0x7f2648158b10 2024-09-19T05:45:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f262ffff640 1 -- 172.21.15.2:0/2705564798 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638004030 con 0x7f2648158b10 2024-09-19T05:45:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f262ffff640 1 -- 172.21.15.2:0/2705564798 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638004400 con 0x7f2648158b10 2024-09-19T05:45:22.905 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f264dae6640 1 -- 172.21.15.2:0/2705564798 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2648160a20 con 0x7f2648158b10 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84859fc640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84780073d0 0x7f847804ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84859fc640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f84780e1b50 0x7f84780e1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84859fc640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f8478050960 con 0x7f84780e1b50 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84859fc640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8478054d40 con 0x7f84780073d0 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84859fc640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8478006f40 con 0x7f84780d8870 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84849fa640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f8478048fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84851fb640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f84780e1b50 0x7f84780e1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.906 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f264dae6640 1 -- 172.21.15.2:0/2705564798 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2648160fe0 con 0x7f2648158b10 2024-09-19T05:45:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84849fa640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f8478048fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40934/0 (socket says 172.21.15.2:40934) 2024-09-19T05:45:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f84849fa640 1 -- 172.21.15.2:0/2977363095 learned_addr learned my addr 172.21.15.2:0/2977363095 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.907 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f96fffff640 1 Processor -- start 2024-09-19T05:45:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f847ffff640 1 --2- 172.21.15.2:0/2977363095 >> v2:172.21.15.123:3300/0 conn(0x7f84780073d0 0x7f847804ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f585759e640 1 -- 172.21.15.2:0/358390353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5850046f80 con 0x7f5850035d50 2024-09-19T05:45:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.861+0000 7f4988b2a640 1 -- start start 2024-09-19T05:45:22.908 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f02377fe640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0238158960 0x7f0238144c60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0230009f90 tx=0x7f023000e560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f84849fa640 1 -- 172.21.15.2:0/2977363095 >> v2:172.21.15.123:3300/0 conn(0x7f84780073d0 msgr2=0x7f847804ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f84849fa640 1 --2- 172.21.15.2:0/2977363095 >> v2:172.21.15.123:3300/0 conn(0x7f84780073d0 0x7f847804ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f84849fa640 1 -- 172.21.15.2:0/2977363095 >> v2:172.21.15.2:3300/0 conn(0x7f84780e1b50 msgr2=0x7f84780e1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:22.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06630f8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c06f690 0x7f065c14b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f0234ff9640 1 -- 172.21.15.2:0/2963411070 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230007cf0 con 0x7f0238158960 2024-09-19T05:45:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f84849fa640 1 --2- 172.21.15.2:0/2977363095 >> v2:172.21.15.2:3300/0 conn(0x7f84780e1b50 0x7f84780e1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f0234ff9640 1 -- 172.21.15.2:0/2963411070 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0230019070 con 0x7f0238158960 2024-09-19T05:45:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f4988b2a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840b2c80 0x7f49840c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f4988b2a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49840cb150 0x7f4984159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f4988b2a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49840cc650 0x7f4984159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f4988b2a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f498415acd0 con 0x7f49840cb150 2024-09-19T05:45:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f4988b2a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f498415b040 con 0x7f49840b2c80 2024-09-19T05:45:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f4988b2a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f498415a420 con 0x7f49840cc650 2024-09-19T05:45:22.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06630f8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c159090 0x7f065c14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06a37fe640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f068c004d40 0x7f068c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f0234ff9640 1 -- 172.21.15.2:0/2963411070 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0230009310 con 0x7f0238158960 2024-09-19T05:45:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06630f8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f065c159470 0x7f065c144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.912 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06a2ffd640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f068c0039b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0694005070 tx=0x7f068c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fc4e1860e517708 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f84849fa640 1 -- 172.21.15.2:0/2977363095 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8478049520 con 0x7f84780d8870 2024-09-19T05:45:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f0660e94640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c06f690 0x7f065c14b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f0660e94640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c06f690 0x7f065c14b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:40950/0 (socket says 172.21.15.2:40950) 2024-09-19T05:45:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f0660e94640 1 -- 172.21.15.2:0/2537567178 learned_addr learned my addr 172.21.15.2:0/2537567178 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f0653fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c159090 0x7f065c14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f498259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840b2c80 0x7f49840c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f498259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840b2c80 0x7f49840c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37158/0 (socket says 172.21.15.2:37158) 2024-09-19T05:45:22.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f498259c640 1 -- 172.21.15.2:0/1037015225 learned_addr learned my addr 172.21.15.2:0/1037015225 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f065c15f9c0 con 0x7f065c159470 2024-09-19T05:45:22.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f065c15f6e0 con 0x7f065c159090 2024-09-19T05:45:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f065c15fca0 con 0x7f065c06f690 2024-09-19T05:45:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.862+0000 7f4982d9d640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49840cc650 0x7f4984159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f0661695640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f065c159470 0x7f065c144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f5846ffd640 1 -- 172.21.15.2:0/358390353 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f584c01a070 con 0x7f5850035d50 2024-09-19T05:45:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f498259c640 1 -- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49840cc650 msgr2=0x7f4984159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f498259c640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49840cc650 0x7f4984159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f498259c640 1 -- 172.21.15.2:0/1037015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49840cb150 msgr2=0x7f4984159430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T05:45:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f96fffff640 1 -- start start 2024-09-19T05:45:22.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02381f5630 con 0x7f0238158960 2024-09-19T05:45:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f4981d9b640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49840cb150 0x7f4984159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f498259c640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49840cb150 0x7f4984159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f498259c640 1 -- 172.21.15.2:0/1037015225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49840c6650 con 0x7f49840b2c80 2024-09-19T05:45:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694004c40 con 0x7f069c0df150 2024-09-19T05:45:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069402f030 con 0x7f069c0df150 2024-09-19T05:45:22.917 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f06a1ffb640 1 -- 172.21.15.2:0/1497181473 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694005cf0 con 0x7f069c0df150 2024-09-19T05:45:22.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f0660e94640 1 -- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c159090 msgr2=0x7f065c14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f0660e94640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c159090 0x7f065c14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f0660e94640 1 -- 172.21.15.2:0/2537567178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f065c159470 msgr2=0x7f065c144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02380ca640 con 0x7f0238158960 2024-09-19T05:45:22.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f4981d9b640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49840cb150 0x7f4984159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f262ffff640 1 -- 172.21.15.2:0/2705564798 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f263801f070 con 0x7f2648158b10 2024-09-19T05:45:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f0660e94640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f065c159470 0x7f065c144860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f498259c640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840b2c80 0x7f49840c5e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f497400b430 tx=0x7f497400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.863+0000 7f0660e94640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c1480f0 con 0x7f065c06f690 2024-09-19T05:45:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f0661695640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f065c159470 0x7f065c144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f5846ffd640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f583806c3b0 0x7f583806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f84849fa640 1 --2- 172.21.15.2:0/2977363095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f8478048fe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8480000b60 tx=0x7f84800a6b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c114e60581e649d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f0660e94640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c06f690 0x7f065c14b670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c400 tx=0x7f064c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f264dae6640 1 -- 172.21.15.2:0/2705564798 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2628005290 con 0x7f2648158b10 2024-09-19T05:45:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f585659c640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f583806c3b0 0x7f583806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f5846ffd640 1 -- 172.21.15.2:0/358390353 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f584c08b1c0 con 0x7f5850035d50 2024-09-19T05:45:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f06517fa640 1 -- 172.21.15.2:0/2537567178 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c0052d0 con 0x7f065c06f690 2024-09-19T05:45:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c1437c0 con 0x7f065c06f690 2024-09-19T05:45:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f06517fa640 1 -- 172.21.15.2:0/2537567178 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064c005490 con 0x7f065c06f690 2024-09-19T05:45:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f06517fa640 1 -- 172.21.15.2:0/2537567178 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c007260 con 0x7f065c06f690 2024-09-19T05:45:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f96fffff640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f804dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c143d80 con 0x7f065c06f690 2024-09-19T05:45:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f96fffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8038e40 0x7f96f800a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f96fffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f8039220 0x7f96f80d88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f847f7fe640 1 -- 172.21.15.2:0/2977363095 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800a8990 con 0x7f84780d8870 2024-09-19T05:45:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f847f7fe640 1 -- 172.21.15.2:0/2977363095 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84800a8b30 con 0x7f84780d8870 2024-09-19T05:45:22.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f847f7fe640 1 -- 172.21.15.2:0/2977363095 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800b0420 con 0x7f84780d8870 2024-09-19T05:45:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f96fffff640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f96f80076b0 con 0x7f96f8038e40 2024-09-19T05:45:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f96fffff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8007530 con 0x7f96f8039220 2024-09-19T05:45:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.864+0000 7f262ffff640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f263406c380 0x7f263406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f262ffff640 1 -- 172.21.15.2:0/2705564798 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f263808ab70 con 0x7f2648158b10 2024-09-19T05:45:22.924 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f96fffff640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96f8005ea0 con 0x7f96f800e3b0 2024-09-19T05:45:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0238151100 con 0x7f0238158960 2024-09-19T05:45:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.865+0000 7f26477fe640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f263406c380 0x7f263406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.925 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f26477fe640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f263406c380 0x7f263406e840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f263c032000 tx=0x7f263c02d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f585659c640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f583806c3b0 0x7f583806e870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5848009560 tx=0x7f584802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96fe7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8038e40 0x7f96f800a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 -- 172.21.15.2:0/2977363095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 msgr2=0x7f8478048fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 --2- 172.21.15.2:0/2977363095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f8478048fe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8480000b60 tx=0x7f84800a6b70 comp rx=0 tx=0).stop 2024-09-19T05:45:22.926 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f8039220 0x7f96f80d88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c005290 con 0x7f065c06f690 2024-09-19T05:45:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96fe7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8038e40 0x7f96f800a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54638/0 (socket says 172.21.15.2:54638) 2024-09-19T05:45:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96fe7fc640 1 -- 172.21.15.2:0/2476097326 learned_addr learned my addr 172.21.15.2:0/2476097326 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96ff7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f8039220 0x7f96f80d88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37176/0 (socket says 172.21.15.2:37176) 2024-09-19T05:45:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96feffd640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f804dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 -- 172.21.15.2:0/2977363095 shutdown_connections 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 --2- 172.21.15.2:0/2977363095 >> v2:172.21.15.2:3300/0 conn(0x7f84780e1b50 0x7f84780e1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 --2- 172.21.15.2:0/2977363095 >> v2:172.21.15.123:3300/0 conn(0x7f84780073d0 0x7f847804ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 --2- 172.21.15.2:0/2977363095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f8478048fe0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 -- 172.21.15.2:0/2977363095 >> 172.21.15.2:0/2977363095 conn(0x7f8478054060 msgr2=0x7f8478054470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 -- 172.21.15.2:0/2977363095 shutdown_connections 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06a98dd640 1 -- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 msgr2=0x7f068c0039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.928 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06a98dd640 1 --2- 172.21.15.2:0/1497181473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f068c0039b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0694005070 tx=0x7f068c005830 comp rx=0 tx=0).stop 2024-09-19T05:45:22.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f84859fc640 1 -- 172.21.15.2:0/2977363095 wait complete. 2024-09-19T05:45:22.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06a37fe640 1 -- 172.21.15.2:0/1497181473 reap_dead start 2024-09-19T05:45:22.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06a98dd640 1 -- 172.21.15.2:0/1497181473 shutdown_connections 2024-09-19T05:45:22.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06a98dd640 1 -- 172.21.15.2:0/1497181473 >> 172.21.15.2:0/1497181473 conn(0x7f069c04fc80 msgr2=0x7f069c050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:22.929 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06a98dd640 1 -- 172.21.15.2:0/1497181473 shutdown_connections 2024-09-19T05:45:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f06a98dd640 1 -- 172.21.15.2:0/1497181473 wait complete. 2024-09-19T05:45:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96feffd640 1 -- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f8039220 msgr2=0x7f96f80d88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96feffd640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f8039220 0x7f96f80d88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96feffd640 1 -- 172.21.15.2:0/2476097326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8038e40 msgr2=0x7f96f800a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96feffd640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8038e40 0x7f96f800a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96feffd640 1 -- 172.21.15.2:0/2476097326 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f80d6450 con 0x7f96f800e3b0 2024-09-19T05:45:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96ff7fe640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f8039220 0x7f96f80d88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96fe7fc640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8038e40 0x7f96f800a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.866+0000 7f96feffd640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f804dd20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e880 tx=0x7f96f000ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.867+0000 7f84859fc640 1 Processor -- start 2024-09-19T05:45:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.867+0000 7f84859fc640 1 -- start start 2024-09-19T05:45:22.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.867+0000 7f06a98dd640 1 Processor -- start 2024-09-19T05:45:22.932 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f06a98dd640 1 -- start start 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84859fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84780073d0 0x7f8478051e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84859fc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f84780e1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84859fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84780e1b50 0x7f84780cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84859fc640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8478006930 con 0x7f84780073d0 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84859fc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8478050100 con 0x7f84780e1b50 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84859fc640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f84780503a0 con 0x7f84780d8870 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84849fa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84780073d0 0x7f8478051e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84849fa640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84780073d0 0x7f8478051e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54648/0 (socket says 172.21.15.2:54648) 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84849fa640 1 -- 172.21.15.2:0/1335332092 learned_addr learned my addr 172.21.15.2:0/1335332092 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f84851fb640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84780e1b50 0x7f84780cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.933 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f847ffff640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f84780e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f06a98dd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c168270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f06a98dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c16b310 0x7f069c16b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f06a98dd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f069c175eb0 0x7f069c176330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f06a98dd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f069c045570 con 0x7f069c175eb0 2024-09-19T05:45:22.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f06a98dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f069c04a8c0 con 0x7f069c16b310 2024-09-19T05:45:22.934 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.868+0000 7f06a98dd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f069c0456f0 con 0x7f069c0df150 2024-09-19T05:45:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84849fa640 1 -- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 msgr2=0x7f84780e1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84849fa640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f84780e1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84849fa640 1 -- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84780e1b50 msgr2=0x7f84780cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84849fa640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84780e1b50 0x7f84780cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.935 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84849fa640 1 -- 172.21.15.2:0/1335332092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8478174580 con 0x7f84780073d0 2024-09-19T05:45:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f069c175eb0 0x7f069c176330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f069c175eb0 0x7f069c176330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54650/0 (socket says 172.21.15.2:54650) 2024-09-19T05:45:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 -- 172.21.15.2:0/1577426006 learned_addr learned my addr 172.21.15.2:0/1577426006 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a2ffd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c168270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a27fc640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c16b310 0x7f069c16b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.936 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84849fa640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84780073d0 0x7f8478051e40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f848002dd50 tx=0x7f8480069b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f5846ffd640 1 -- 172.21.15.2:0/358390353 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f584c059940 con 0x7f5850035d50 2024-09-19T05:45:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f4972ffd640 1 -- 172.21.15.2:0/1037015225 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974015020 con 0x7f49840b2c80 2024-09-19T05:45:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f4972ffd640 1 -- 172.21.15.2:0/1037015225 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974011830 con 0x7f49840b2c80 2024-09-19T05:45:22.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f4972ffd640 1 -- 172.21.15.2:0/1037015225 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974011c20 con 0x7f49840b2c80 2024-09-19T05:45:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 -- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 msgr2=0x7f069c168270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c168270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 -- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c16b310 msgr2=0x7f069c16b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c16b310 0x7f069c16b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f06a37fe640 1 -- 172.21.15.2:0/1577426006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c1687b0 con 0x7f069c175eb0 2024-09-19T05:45:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4984154040 con 0x7f49840b2c80 2024-09-19T05:45:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f847d7fa640 1 -- 172.21.15.2:0/1335332092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800a8790 con 0x7f84780073d0 2024-09-19T05:45:22.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49840c6bf0 con 0x7f49840b2c80 2024-09-19T05:45:22.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f847d7fa640 1 -- 172.21.15.2:0/1335332092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84800a8930 con 0x7f84780073d0 2024-09-19T05:45:22.939 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f847d7fa640 1 -- 172.21.15.2:0/1335332092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800bb640 con 0x7f84780073d0 2024-09-19T05:45:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.870+0000 7f262ffff640 1 -- 172.21.15.2:0/2705564798 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2638058c30 con 0x7f2648158b10 2024-09-19T05:45:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.870+0000 7f96e7fff640 1 -- 172.21.15.2:0/2476097326 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0017070 con 0x7f96f800e3b0 2024-09-19T05:45:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.871+0000 7f0234ff9640 1 -- 172.21.15.2:0/2963411070 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0230005d10 con 0x7f0238158960 2024-09-19T05:45:22.940 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.871+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4984157790 con 0x7f49840b2c80 2024-09-19T05:45:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84780dfdf0 con 0x7f84780073d0 2024-09-19T05:45:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.869+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84780dfb10 con 0x7f84780073d0 2024-09-19T05:45:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.871+0000 7f06a37fe640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f069c175eb0 0x7f069c176330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f06a4000b60 tx=0x7f06a4069890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.941 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.871+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84780d3de0 con 0x7f84780073d0 2024-09-19T05:45:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.870+0000 7f96e7fff640 1 -- 172.21.15.2:0/2476097326 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f000c940 con 0x7f96f800e3b0 2024-09-19T05:45:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.870+0000 7f96e7fff640 1 -- 172.21.15.2:0/2476097326 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000cd10 con 0x7f96f800e3b0 2024-09-19T05:45:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.870+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f804c550 con 0x7f96f800e3b0 2024-09-19T05:45:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.870+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f8053df0 con 0x7f96f800e3b0 2024-09-19T05:45:22.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.871+0000 7f068bfff640 1 -- 172.21.15.2:0/1577426006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4072070 con 0x7f069c175eb0 2024-09-19T05:45:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f068bfff640 1 -- 172.21.15.2:0/1577426006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a4063240 con 0x7f069c175eb0 2024-09-19T05:45:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f0234ff9640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f022406c3b0 0x7f022406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f06517fa640 1 -- 172.21.15.2:0/2537567178 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f064c007400 con 0x7f065c06f690 2024-09-19T05:45:22.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f847d7fa640 1 -- 172.21.15.2:0/1335332092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f84800c4020 con 0x7f84780073d0 2024-09-19T05:45:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f0234ff9640 1 -- 172.21.15.2:0/2963411070 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f023008c730 con 0x7f0238158960 2024-09-19T05:45:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f06517fa640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f064806c2b0 0x7f064806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.873+0000 7f06517fa640 1 -- 172.21.15.2:0/2537567178 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f064c08ad30 con 0x7f065c06f690 2024-09-19T05:45:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c176870 con 0x7f069c175eb0 2024-09-19T05:45:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f068bfff640 1 -- 172.21.15.2:0/1577426006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a4072070 con 0x7f069c175eb0 2024-09-19T05:45:22.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c047400 con 0x7f069c175eb0 2024-09-19T05:45:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.872+0000 7f847d7fa640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f846406c380 0x7f846406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.873+0000 7f847d7fa640 1 -- 172.21.15.2:0/1335332092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f84801281f0 con 0x7f84780073d0 2024-09-19T05:45:22.945 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.873+0000 7f0653fff640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f064806c2b0 0x7f064806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.873+0000 7f847ffff640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f846406c380 0x7f846406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.873+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e0005290 con 0x7f96f800e3b0 2024-09-19T05:45:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.873+0000 7f0237fff640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f022406c3b0 0x7f022406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.946 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.874+0000 7f847ffff640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f846406c380 0x7f846406e840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8478171fa0 tx=0x7f846c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.874+0000 7f0653fff640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f064806c2b0 0x7f064806e770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0658034000 tx=0x7f065802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.874+0000 7f0237fff640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f022406c3b0 0x7f022406e870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0228001bc0 tx=0x7f0228009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.947 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.874+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069c005b10 con 0x7f069c175eb0 2024-09-19T05:45:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.874+0000 7f0234ff9640 1 -- 172.21.15.2:0/2963411070 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f0230023020 con 0x7f0238158960 2024-09-19T05:45:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.876+0000 7f06517fa640 1 -- 172.21.15.2:0/2537567178 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f064c058e60 con 0x7f065c06f690 2024-09-19T05:45:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.876+0000 7f4972ffd640 1 -- 172.21.15.2:0/1037015225 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4974018020 con 0x7f49840b2c80 2024-09-19T05:45:22.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.876+0000 7f96e7fff640 1 -- 172.21.15.2:0/2476097326 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f96f001a070 con 0x7f96f800e3b0 2024-09-19T05:45:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.877+0000 7f96e7fff640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96ec06c3b0 0x7f96ec06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.877+0000 7f4972ffd640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f496406c300 0x7f496406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.877+0000 7f4972ffd640 1 -- 172.21.15.2:0/1037015225 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f497408b910 con 0x7f49840b2c80 2024-09-19T05:45:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.877+0000 7f96e7fff640 1 -- 172.21.15.2:0/2476097326 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f96f008bac0 con 0x7f96f800e3b0 2024-09-19T05:45:22.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.877+0000 7f96fe7fc640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96ec06c3b0 0x7f96ec06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.877+0000 7f4981d9b640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f496406c300 0x7f496406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.878+0000 7f96fe7fc640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96ec06c3b0 0x7f96ec06e870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96f80053c0 tx=0x7f96f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.878+0000 7f4981d9b640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f496406c300 0x7f496406e7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49840c63d0 tx=0x7f4978009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.950 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.879+0000 7f4972ffd640 1 -- 172.21.15.2:0/1037015225 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f497401f020 con 0x7f49840b2c80 2024-09-19T05:45:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.881+0000 7f96e7fff640 1 -- 172.21.15.2:0/2476097326 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f96f0059bf0 con 0x7f96f800e3b0 2024-09-19T05:45:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.881+0000 7f068bfff640 1 -- 172.21.15.2:0/1577426006 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f06a40633f0 con 0x7f069c175eb0 2024-09-19T05:45:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.881+0000 7f068bfff640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f069006c6b0 0x7f069006eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.882+0000 7f068bfff640 1 -- 172.21.15.2:0/1577426006 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f06a4050ea0 con 0x7f069c175eb0 2024-09-19T05:45:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.882+0000 7f06a2ffd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f069006c6b0 0x7f069006eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:22.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.879+0000 7f847d7fa640 1 -- 172.21.15.2:0/1335332092 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f84780d3de0 con 0x7f84780073d0 2024-09-19T05:45:22.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.883+0000 7f06a2ffd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f069006c6b0 0x7f069006eb70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0694005040 tx=0x7f0694007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:22.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:22.889+0000 7f068bfff640 1 -- 172.21.15.2:0/1577426006 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f06a40a4080 con 0x7f069c175eb0 2024-09-19T05:45:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.271+0000 7f264dae6640 1 -- 172.21.15.2:0/2705564798 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2628005580 con 0x7f2648158b10 2024-09-19T05:45:23.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.271+0000 7f262ffff640 1 -- 172.21.15.2:0/2705564798 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f263805cf30 con 0x7f2648158b10 2024-09-19T05:45:23.274 INFO:teuthology.orchestra.run.smithi002.stdout:60129542182 2024-09-19T05:45:23.277 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 -- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f263406c380 msgr2=0x7f263406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f263406c380 0x7f263406e840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f263c032000 tx=0x7f263c02d000 comp rx=0 tx=0).stop 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 -- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2648158b10 msgr2=0x7f2648161e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2648158b10 0x7f2648161e60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2648072480 tx=0x7f2638011d70 comp rx=0 tx=0).stop 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 -- 172.21.15.2:0/2705564798 shutdown_connections 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f263406c380 0x7f263406e840 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26481f5940 0x7f26481585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.275+0000 7f262dffb640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2648158b10 0x7f2648161e60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.276+0000 7f262dffb640 1 --2- 172.21.15.2:0/2705564798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26480c7e10 0x7f26480c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.276+0000 7f262dffb640 1 -- 172.21.15.2:0/2705564798 >> 172.21.15.2:0/2705564798 conn(0x7f264806bdf0 msgr2=0x7f26480cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.276+0000 7f262dffb640 1 -- 172.21.15.2:0/2705564798 shutdown_connections 2024-09-19T05:45:23.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.276+0000 7f262dffb640 1 -- 172.21.15.2:0/2705564798 wait complete. 2024-09-19T05:45:23.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.282+0000 7f585759e640 1 -- 172.21.15.2:0/358390353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5850045dd0 con 0x7f5850035d50 2024-09-19T05:45:23.284 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.282+0000 7f5846ffd640 1 -- 172.21.15.2:0/358390353 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f584c059240 con 0x7f5850035d50 2024-09-19T05:45:23.284 INFO:teuthology.orchestra.run.smithi002.stdout:64424509478 2024-09-19T05:45:23.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 -- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f583806c3b0 msgr2=0x7f583806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f583806c3b0 0x7f583806e870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5848009560 tx=0x7f584802e000 comp rx=0 tx=0).stop 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 -- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850035d50 msgr2=0x7f58500e46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850035d50 0x7f58500e46d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f584c00e890 tx=0x7f584c00ed50 comp rx=0 tx=0).stop 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 -- 172.21.15.2:0/358390353 shutdown_connections 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f583806c3b0 0x7f583806e870 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5850035d50 0x7f58500e46d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5850046b30 0x7f58501748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.286+0000 7f5844ff9640 1 --2- 172.21.15.2:0/358390353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5850006870 0x7f58500e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.287+0000 7f5844ff9640 1 -- 172.21.15.2:0/358390353 >> 172.21.15.2:0/358390353 conn(0x7f58500504e0 msgr2=0x7f585004fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.287+0000 7f5844ff9640 1 -- 172.21.15.2:0/358390353 shutdown_connections 2024-09-19T05:45:23.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.287+0000 7f5844ff9640 1 -- 172.21.15.2:0/358390353 wait complete. 2024-09-19T05:45:23.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.290+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f02380c4600 con 0x7f0238158960 2024-09-19T05:45:23.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.291+0000 7f0234ff9640 1 -- 172.21.15.2:0/2963411070 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0230011070 con 0x7f0238158960 2024-09-19T05:45:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:23.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.292+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f49840cbcb0 con 0x7f49840b2c80 2024-09-19T05:45:23.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.292+0000 7f4972ffd640 1 -- 172.21.15.2:0/1037015225 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f497405a100 con 0x7f49840b2c80 2024-09-19T05:45:23.294 INFO:teuthology.orchestra.run.smithi002.stdout:335007449095 2024-09-19T05:45:23.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542182 for osd.1 2024-09-19T05:45:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8478174370 con 0x7f84780073d0 2024-09-19T05:45:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f022406c3b0 msgr2=0x7f022406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f022406c3b0 0x7f022406e870 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0228001bc0 tx=0x7f0228009040 comp rx=0 tx=0).stop 2024-09-19T05:45:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0238158960 msgr2=0x7f0238144c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0238158960 0x7f0238144c60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0230009f90 tx=0x7f023000e560 comp rx=0 tx=0).stop 2024-09-19T05:45:23.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f063c005850 con 0x7f065c06f690 2024-09-19T05:45:23.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 shutdown_connections 2024-09-19T05:45:23.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f022406c3b0 0x7f022406e870 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0238158960 0x7f0238144c60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02380cc360 0x7f02380cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 --2- 172.21.15.2:0/2963411070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0220004fe0 0x7f02380cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.298 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 >> 172.21.15.2:0/2963411070 conn(0x7f023806bc10 msgr2=0x7f023806c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f847d7fa640 1 -- 172.21.15.2:0/1335332092 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f84800f6200 con 0x7f84780073d0 2024-09-19T05:45:23.299 INFO:teuthology.orchestra.run.smithi002.stdout:60129542182 2024-09-19T05:45:23.299 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 shutdown_connections 2024-09-19T05:45:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.295+0000 7f06517fa640 1 -- 172.21.15.2:0/2537567178 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f064c05d160 con 0x7f065c06f690 2024-09-19T05:45:23.300 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f023e67b640 1 -- 172.21.15.2:0/2963411070 wait complete. 2024-09-19T05:45:23.301 INFO:teuthology.orchestra.run.smithi002.stdout:60129542182 2024-09-19T05:45:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f496406c300 msgr2=0x7f496406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f496406c300 0x7f496406e7c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49840c63d0 tx=0x7f4978009040 comp rx=0 tx=0).stop 2024-09-19T05:45:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840b2c80 msgr2=0x7f49840c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840b2c80 0x7f49840c5e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f497400b430 tx=0x7f497400b8f0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 shutdown_connections 2024-09-19T05:45:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f496406c300 0x7f496406e7c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f49840cc650 0x7f4984159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49840cb150 0x7f4984159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 --2- 172.21.15.2:0/1037015225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49840b2c80 0x7f49840c5e90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.296+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 >> 172.21.15.2:0/1037015225 conn(0x7f498406fed0 msgr2=0x7f4984072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.297+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 shutdown_connections 2024-09-19T05:45:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.297+0000 7f4988b2a640 1 -- 172.21.15.2:0/1037015225 wait complete. 2024-09-19T05:45:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.297+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f069c04b1f0 con 0x7f069c175eb0 2024-09-19T05:45:23.303 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.297+0000 7f068bfff640 1 -- 172.21.15.2:0/1577426006 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06a40a4080 con 0x7f069c175eb0 2024-09-19T05:45:23.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.298+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f96e0005850 con 0x7f96f800e3b0 2024-09-19T05:45:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:60129542182 2024-09-19T05:45:23.304 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.298+0000 7f96e7fff640 1 -- 172.21.15.2:0/2476097326 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96f005def0 con 0x7f96f800e3b0 2024-09-19T05:45:23.305 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f846406c380 msgr2=0x7f846406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f846406c380 0x7f846406e840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8478171fa0 tx=0x7f846c008040 comp rx=0 tx=0).stop 2024-09-19T05:45:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84780073d0 msgr2=0x7f8478051e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.305 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84780073d0 0x7f8478051e40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f848002dd50 tx=0x7f8480069b30 comp rx=0 tx=0).stop 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 shutdown_connections 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f846406c380 0x7f846406e840 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84780e1b50 0x7f84780cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f84780d8870 0x7f84780e1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 --2- 172.21.15.2:0/1335332092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84780073d0 0x7f8478051e40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 >> 172.21.15.2:0/1335332092 conn(0x7f8478054060 msgr2=0x7f84780d7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f064806c2b0 msgr2=0x7f064806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.306 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f06630f8640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f064806c2b0 0x7f064806e770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0658034000 tx=0x7f065802e000 comp rx=0 tx=0).stop 2024-09-19T05:45:23.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 shutdown_connections 2024-09-19T05:45:23.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c06f690 msgr2=0x7f065c14b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.307 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f84859fc640 1 -- 172.21.15.2:0/1335332092 wait complete. 2024-09-19T05:45:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.299+0000 7f06630f8640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c06f690 0x7f065c14b670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c400 tx=0x7f064c00c8c0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 shutdown_connections 2024-09-19T05:45:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f064806c2b0 0x7f064806e770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f065c159090 0x7f065c14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f065c159470 0x7f065c144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 --2- 172.21.15.2:0/2537567178 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f065c06f690 0x7f065c14b670 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 >> 172.21.15.2:0/2537567178 conn(0x7f065c06bc10 msgr2=0x7f065c0c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f069006c6b0 msgr2=0x7f069006eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 shutdown_connections 2024-09-19T05:45:23.309 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06630f8640 1 -- 172.21.15.2:0/2537567178 wait complete. 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f069006c6b0 0x7f069006eb70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0694005040 tx=0x7f0694007000 comp rx=0 tx=0).stop 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f069c175eb0 msgr2=0x7f069c176330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f069c175eb0 0x7f069c176330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f06a4000b60 tx=0x7f06a4069890 comp rx=0 tx=0).stop 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 shutdown_connections 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f069006c6b0 0x7f069006eb70 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0694005040 tx=0x7f0694007000 comp rx=0 tx=0).stop 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f069c16b310 0x7f069c16b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f069c175eb0 0x7f069c176330 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 --2- 172.21.15.2:0/1577426006 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f069c0df150 0x7f069c168270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 >> 172.21.15.2:0/1577426006 conn(0x7f069c04fc80 msgr2=0x7f069c0d6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.301+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 shutdown_connections 2024-09-19T05:45:23.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f06a98dd640 1 -- 172.21.15.2:0/1577426006 wait complete. 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96ec06c3b0 msgr2=0x7f96ec06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96ec06c3b0 0x7f96ec06e870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f96f80053c0 tx=0x7f96f4009040 comp rx=0 tx=0).stop 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 msgr2=0x7f96f804dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f804dd20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e880 tx=0x7f96f000ed40 comp rx=0 tx=0).stop 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 shutdown_connections 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f96ec06c3b0 0x7f96ec06e870 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96f8039220 0x7f96f80d88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8038e40 0x7f96f800a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 --2- 172.21.15.2:0/2476097326 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96f800e3b0 0x7f96f804dd20 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f96f000e880 tx=0x7f96f000ed40 comp rx=0 tx=0).stop 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 >> 172.21.15.2:0/2476097326 conn(0x7f96f8054450 msgr2=0x7f96f8052330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:23.312 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.302+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 shutdown_connections 2024-09-19T05:45:23.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:23.303+0000 7f96fffff640 1 -- 172.21.15.2:0/2476097326 wait complete. 2024-09-19T05:45:23.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509478 for osd.2 2024-09-19T05:45:23.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542184 got 60129542183 for osd.6 2024-09-19T05:45:23.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542182 for osd.3 2024-09-19T05:45:23.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 335007449096 got 335007449095 for osd.5 2024-09-19T05:45:23.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542182 for osd.7 2024-09-19T05:45:23.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542184 got 60129542183 for osd.0 2024-09-19T05:45:23.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542182 for osd.4 2024-09-19T05:45:24.298 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T05:45:24.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T05:45:24.318 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T05:45:24.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T05:45:24.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T05:45:24.325 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T05:45:24.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T05:45:24.327 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T05:45:24.469 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.467+0000 7fd5f0e27640 1 Processor -- start 2024-09-19T05:45:24.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.467+0000 7fd5f0e27640 1 -- start start 2024-09-19T05:45:24.471 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.469+0000 7fd5f0e27640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd5ec072650 0x7fd5ec071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.469+0000 7fd5f0e27640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd5ec072090 0x7fd5ec06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.469+0000 7fd5f0e27640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ec15a130 con 0x7fd5ec150d20 2024-09-19T05:45:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.469+0000 7fd5f0e27640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ec0c6990 con 0x7fd5ec072090 2024-09-19T05:45:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.469+0000 7fd5f0e27640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ec15a3a0 con 0x7fd5ec072650 2024-09-19T05:45:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.469+0000 7fd5ea59c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd5ec150d20 0x7fd5ec0c77a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54112/0 (socket says 172.21.15.2:54112) 2024-09-19T05:45:24.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.469+0000 7fd5ea59c640 1 -- 172.21.15.2:0/2678525747 learned_addr learned my addr 172.21.15.2:0/2678525747 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.470+0000 7fd5ead9d640 1 --2- 172.21.15.2:0/2678525747 >> v2:172.21.15.2:3300/0 conn(0x7fd5ec072090 0x7fd5ec06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.470+0000 7fd5e9d9b640 1 --2- 172.21.15.2:0/2678525747 >> v2:172.21.15.123:3300/0 conn(0x7fd5ec072650 0x7fd5ec071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.471+0000 7fd5ead9d640 1 -- 172.21.15.2:0/2678525747 >> v2:172.21.15.123:3300/0 conn(0x7fd5ec072650 msgr2=0x7fd5ec071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.471+0000 7fd5ead9d640 1 --2- 172.21.15.2:0/2678525747 >> v2:172.21.15.123:3300/0 conn(0x7fd5ec072650 0x7fd5ec071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.471+0000 7fd5ead9d640 1 -- 172.21.15.2:0/2678525747 >> v1:172.21.15.2:6789/0 conn(0x7fd5ec150d20 legacy=0x7fd5ec0c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.471+0000 7fd5ead9d640 1 -- 172.21.15.2:0/2678525747 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec15a680 con 0x7fd5ec072090 2024-09-19T05:45:24.473 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.471+0000 7fd5e9d9b640 1 --2- 172.21.15.2:0/2678525747 >> v2:172.21.15.123:3300/0 conn(0x7fd5ec072650 0x7fd5ec071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:24.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.472+0000 7fd5ead9d640 1 --2- 172.21.15.2:0/2678525747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec06ee70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0706b0 tx=0x7fd5e002c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1966673abe6d63c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.473+0000 7fd5e959a640 1 -- 172.21.15.2:0/2678525747 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0039070 con 0x7fd5ec072090 2024-09-19T05:45:24.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.473+0000 7fd5e959a640 1 -- 172.21.15.2:0/2678525747 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e0034440 con 0x7fd5ec072090 2024-09-19T05:45:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.474+0000 7fd5f0e27640 1 -- 172.21.15.2:0/2678525747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 msgr2=0x7fd5ec06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.474+0000 7fd5f0e27640 1 --2- 172.21.15.2:0/2678525747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec06ee70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0706b0 tx=0x7fd5e002c220 comp rx=0 tx=0).stop 2024-09-19T05:45:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.475+0000 7fd5f0e27640 1 -- 172.21.15.2:0/2678525747 shutdown_connections 2024-09-19T05:45:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.475+0000 7fd5f0e27640 1 --2- 172.21.15.2:0/2678525747 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec06ee70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.477 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.475+0000 7fd5f0e27640 1 --2- 172.21.15.2:0/2678525747 >> v2:172.21.15.123:3300/0 conn(0x7fd5ec072650 0x7fd5ec071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.475+0000 7fd5f0e27640 1 -- 172.21.15.2:0/2678525747 >> 172.21.15.2:0/2678525747 conn(0x7fd5ec06bc10 msgr2=0x7fd5ec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.478 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.475+0000 7fd5f0e27640 1 -- 172.21.15.2:0/2678525747 shutdown_connections 2024-09-19T05:45:24.479 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.475+0000 7fd5f0e27640 1 -- 172.21.15.2:0/2678525747 wait complete. 2024-09-19T05:45:24.480 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.477+0000 7fd5f0e27640 1 Processor -- start 2024-09-19T05:45:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.478+0000 7fd5f0e27640 1 -- start start 2024-09-19T05:45:24.481 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.479+0000 7fd5f0e27640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec1e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5ea59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec1e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5ea59c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec1e0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54664/0 (socket says 172.21.15.2:54664) 2024-09-19T05:45:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5ea59c640 1 -- 172.21.15.2:0/1627724995 learned_addr learned my addr 172.21.15.2:0/1627724995 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5f0e27640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5ec072650 0x7fd5ec1fd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.482 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5f0e27640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5ec150d20 0x7fd5ec1e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5f0e27640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec0c8cb0 con 0x7fd5ec072090 2024-09-19T05:45:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5e9d9b640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5ec072650 0x7fd5ec1fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.483 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.480+0000 7fd5ead9d640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5ec150d20 0x7fd5ec1e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.481+0000 7fd5f0e27640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec15ac40 con 0x7fd5ec150d20 2024-09-19T05:45:24.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.481+0000 7fd5f0e27640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ec0c80a0 con 0x7fd5ec072650 2024-09-19T05:45:24.484 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.481+0000 7fd5ea59c640 1 -- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5ec072650 msgr2=0x7fd5ec1fd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.481+0000 7fd5ea59c640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5ec072650 0x7fd5ec1fd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.481+0000 7fd5ea59c640 1 -- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5ec150d20 msgr2=0x7fd5ec1e4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.481+0000 7fd5ea59c640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5ec150d20 0x7fd5ec1e4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.485 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.482+0000 7fd5ea59c640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec159d30 con 0x7fd5ec072090 2024-09-19T05:45:24.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.482+0000 7fd5ea59c640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec1e0e90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400cb60 tx=0x7fd5e400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.483+0000 7fd5daffd640 1 -- 172.21.15.2:0/1627724995 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4018070 con 0x7fd5ec072090 2024-09-19T05:45:24.486 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.483+0000 7fd5f0e27640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ec14cb20 con 0x7fd5ec072090 2024-09-19T05:45:24.487 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.483+0000 7fd5f0e27640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ec0c9af0 con 0x7fd5ec072090 2024-09-19T05:45:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.486+0000 7fd5daffd640 1 -- 172.21.15.2:0/1627724995 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e4004b60 con 0x7fd5ec072090 2024-09-19T05:45:24.488 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.486+0000 7fd5daffd640 1 -- 172.21.15.2:0/1627724995 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400ab40 con 0x7fd5ec072090 2024-09-19T05:45:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.487+0000 7fd5daffd640 1 -- 172.21.15.2:0/1627724995 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd5e401b070 con 0x7fd5ec072090 2024-09-19T05:45:24.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.487+0000 7fd5daffd640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd5cc06c3b0 0x7fd5cc06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.488+0000 7fd5e9d9b640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd5cc06c3b0 0x7fd5cc06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.488+0000 7fd5daffd640 1 -- 172.21.15.2:0/1627724995 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fd5e408ad80 con 0x7fd5ec072090 2024-09-19T05:45:24.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.488+0000 7fd5e9d9b640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd5cc06c3b0 0x7fd5cc06e870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0c9660 tx=0x7fd5dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.493+0000 7fd5f0e27640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d0005290 con 0x7fd5ec072090 2024-09-19T05:45:24.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.503+0000 7fd5daffd640 1 -- 172.21.15.2:0/1627724995 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fd5e4058e20 con 0x7fd5ec072090 2024-09-19T05:45:24.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.521+0000 7ff744f9b640 1 Processor -- start 2024-09-19T05:45:24.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.521+0000 7ff744f9b640 1 -- start start 2024-09-19T05:45:24.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7ff744f9b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7400c6fb0 0x7ff74015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7ff744f9b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff74015af90 con 0x7ff7401577f0 2024-09-19T05:45:24.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7ff744f9b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400c8740 con 0x7ff74006c700 2024-09-19T05:45:24.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7ff744f9b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff740156a50 con 0x7ff7400c6fb0 2024-09-19T05:45:24.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7ff73f7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7400c6fb0 0x7ff74015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7ff73effd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff7401577f0 0x7ff7400719f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54122/0 (socket says 172.21.15.2:54122) 2024-09-19T05:45:24.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7ff73effd640 1 -- 172.21.15.2:0/1762952903 learned_addr learned my addr 172.21.15.2:0/1762952903 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:24.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.523+0000 7f2f38bfc640 1 Processor -- start 2024-09-19T05:45:24.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.524+0000 7fa83f835640 1 Processor -- start 2024-09-19T05:45:24.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.524+0000 7f2f38bfc640 1 -- start start 2024-09-19T05:45:24.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.524+0000 7ff73f7fe640 1 -- 172.21.15.2:0/1762952903 >> v1:172.21.15.123:6789/0 conn(0x7ff74006c700 legacy=0x7ff7400c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.524+0000 7ff73f7fe640 1 -- 172.21.15.2:0/1762952903 >> v1:172.21.15.2:6789/0 conn(0x7ff7401577f0 legacy=0x7ff7400719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.524+0000 7ff73f7fe640 1 -- 172.21.15.2:0/1762952903 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400c77e0 con 0x7ff7400c6fb0 2024-09-19T05:45:24.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f55e48af640 1 Processor -- start 2024-09-19T05:45:24.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f38bfc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f340c6af0 0x7f2f340d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f38bfc640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2f340c7750 0x7f2f340d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f38bfc640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2f340c7ec0 0x7f2f340ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f38bfc640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f2f3415ab10 con 0x7f2f340c7750 2024-09-19T05:45:24.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f38bfc640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2f3415adb0 con 0x7f2f340c7ec0 2024-09-19T05:45:24.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f38bfc640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3415b090 con 0x7f2f340c6af0 2024-09-19T05:45:24.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7ff73f7fe640 1 --2- 172.21.15.2:0/1762952903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7400c6fb0 0x7ff74015aa50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff738009dc0 tx=0x7ff73802a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544c580c6f21362e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7fa83f835640 1 -- start start 2024-09-19T05:45:24.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f55e48af640 1 -- start start 2024-09-19T05:45:24.532 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f33fff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2f340c7ec0 0x7f2f340ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f33fff640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2f340c7ec0 0x7f2f340ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37214/0 (socket says 172.21.15.2:37214) 2024-09-19T05:45:24.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f33fff640 1 -- 172.21.15.2:0/2785411005 learned_addr learned my addr 172.21.15.2:0/2785411005 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7ff73e7fc640 1 -- 172.21.15.2:0/1762952903 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738035020 con 0x7ff7400c6fb0 2024-09-19T05:45:24.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83f835640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa838159090 0x7fa838159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83f835640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8380c8ee0 0x7fa8380c9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83f835640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa838071e10 con 0x7fa83806f150 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83f835640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa83815f160 con 0x7fa838159090 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83f835640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa83815ff80 con 0x7fa8380c8ee0 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83ddd2640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa83806f150 0x7fa8380c72c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54128/0 (socket says 172.21.15.2:54128) 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83ddd2640 1 -- 172.21.15.2:0/2386419949 learned_addr learned my addr 172.21.15.2:0/2386419949 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7fa83cdd0640 1 --2- 172.21.15.2:0/2386419949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8380c8ee0 0x7fa8380c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7f2f337fe640 1 --2- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f340c6af0 0x7f2f340d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f2f32ffd640 1 --2- 172.21.15.2:0/2785411005 >> v2:172.21.15.2:3300/0 conn(0x7f2f340c7750 0x7f2f340d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f2f33fff640 1 -- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f340c6af0 msgr2=0x7f2f340d02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f2f33fff640 1 --2- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f340c6af0 0x7f2f340d02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f2f33fff640 1 -- 172.21.15.2:0/2785411005 >> v2:172.21.15.2:3300/0 conn(0x7f2f340c7750 msgr2=0x7f2f340d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.525+0000 7ff73e7fc640 1 -- 172.21.15.2:0/1762952903 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738007740 con 0x7ff7400c6fb0 2024-09-19T05:45:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7ff744f9b640 1 -- 172.21.15.2:0/1762952903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7400c6fb0 msgr2=0x7ff74015aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7ff744f9b640 1 --2- 172.21.15.2:0/1762952903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7400c6fb0 0x7ff74015aa50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff738009dc0 tx=0x7ff73802a500 comp rx=0 tx=0).stop 2024-09-19T05:45:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f43684dd640 1 Processor -- start 2024-09-19T05:45:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f43684dd640 1 -- start start 2024-09-19T05:45:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7fa82ffff640 1 -- 172.21.15.2:0/2386419949 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4087257006 0 0) 0x7fa838071e10 con 0x7fa83806f150 2024-09-19T05:45:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f2f33fff640 1 --2- 172.21.15.2:0/2785411005 >> v2:172.21.15.2:3300/0 conn(0x7f2f340c7750 0x7f2f340d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.536 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.526+0000 7f2f33fff640 1 -- 172.21.15.2:0/2785411005 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3415b370 con 0x7f2f340c7ec0 2024-09-19T05:45:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7ff744f9b640 1 -- 172.21.15.2:0/1762952903 shutdown_connections 2024-09-19T05:45:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7ff744f9b640 1 --2- 172.21.15.2:0/1762952903 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7400c6fb0 0x7ff74015aa50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7ff744f9b640 1 -- 172.21.15.2:0/1762952903 >> 172.21.15.2:0/1762952903 conn(0x7ff74006bc10 msgr2=0x7ff74006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7fa82ffff640 1 -- 172.21.15.2:0/2386419949 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa824003520 con 0x7fa83806f150 2024-09-19T05:45:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f2f32ffd640 1 --2- 172.21.15.2:0/2785411005 >> v2:172.21.15.2:3300/0 conn(0x7f2f340c7750 0x7f2f340d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f2f33fff640 1 --2- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c7ec0 0x7f2f340ccb00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f3408df30 tx=0x7f2f2c02a740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fa148d2ace57547 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7fa83cdd0640 1 -- 172.21.15.2:0/2386419949 >> v2:172.21.15.123:3300/0 conn(0x7fa838159090 msgr2=0x7fa838159470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:45:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7fa83cdd0640 1 --2- 172.21.15.2:0/2386419949 >> v2:172.21.15.123:3300/0 conn(0x7fa838159090 0x7fa838159470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7fa83cdd0640 1 -- 172.21.15.2:0/2386419949 >> v1:172.21.15.2:6789/0 conn(0x7fa83806f150 legacy=0x7fa8380c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7fa83cdd0640 1 -- 172.21.15.2:0/2386419949 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83815f400 con 0x7fa8380c8ee0 2024-09-19T05:45:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f2f327fc640 1 -- 172.21.15.2:0/2785411005 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c036070 con 0x7f2f340c7ec0 2024-09-19T05:45:24.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f2f327fc640 1 -- 172.21.15.2:0/2785411005 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2c031600 con 0x7f2f340c7ec0 2024-09-19T05:45:24.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f43684dd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9130 0x7f4360071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f55e48af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00cd220 0x7f55e00b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f55e48af640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f55e00b4630 0x7f55e015f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f55e48af640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e01627c0 con 0x7f55e00b30e0 2024-09-19T05:45:24.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f55e48af640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f55e00d0450 con 0x7f55e00b4630 2024-09-19T05:45:24.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.527+0000 7f55e48af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f55e0162aa0 con 0x7f55e00cd220 2024-09-19T05:45:24.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f55de59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00cd220 0x7f55e00b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f55de59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00cd220 0x7f55e00b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:41036/0 (socket says 172.21.15.2:41036) 2024-09-19T05:45:24.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f55de59c640 1 -- 172.21.15.2:0/3026450088 learned_addr learned my addr 172.21.15.2:0/3026450088 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f43684dd640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f43600c9510 0x7f436006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7fa83cdd0640 1 --2- 172.21.15.2:0/2386419949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8380c8ee0 0x7fa8380c9300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097a0 tx=0x7fa83402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7d8968ed9e5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f4366279640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9130 0x7f4360071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f4366279640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9130 0x7f4360071fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:41042/0 (socket says 172.21.15.2:41042) 2024-09-19T05:45:24.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f4366279640 1 -- 172.21.15.2:0/2052946486 learned_addr learned my addr 172.21.15.2:0/2052946486 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f55de59c640 1 -- 172.21.15.2:0/3026450088 >> v2:172.21.15.123:3300/0 conn(0x7f55e00b4630 msgr2=0x7f55e015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f55de59c640 1 --2- 172.21.15.2:0/3026450088 >> v2:172.21.15.123:3300/0 conn(0x7f55e00b4630 0x7f55e015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f55de59c640 1 -- 172.21.15.2:0/3026450088 >> v1:172.21.15.123:6789/0 conn(0x7f55e00b30e0 legacy=0x7f55e00d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f43684dd640 1 --2- 172.21.15.2:0/2052946486 >> v2:172.21.15.2:3300/0 conn(0x7f4360071b50 0x7f4360156b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f55de59c640 1 -- 172.21.15.2:0/3026450088 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e0162d80 con 0x7f55e00cd220 2024-09-19T05:45:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c7ec0 msgr2=0x7f2f340ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c7ec0 0x7f2f340ccb00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2f3408df30 tx=0x7f2f2c02a740 comp rx=0 tx=0).stop 2024-09-19T05:45:24.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f4366a7a640 1 --2- 172.21.15.2:0/2052946486 >> v2:172.21.15.2:3300/0 conn(0x7f4360071b50 0x7f4360156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f436006ee70 con 0x7f4360071b50 2024-09-19T05:45:24.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4360157650 con 0x7f43600c9510 2024-09-19T05:45:24.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4360157930 con 0x7f43600c9130 2024-09-19T05:45:24.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2785411005 shutdown_connections 2024-09-19T05:45:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c7ec0 0x7f2f340ccb00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2785411005 >> v2:172.21.15.2:3300/0 conn(0x7f2f340c7750 0x7f2f340d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2785411005 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f340c6af0 0x7f2f340d02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2785411005 >> 172.21.15.2:0/2785411005 conn(0x7f2f3406cb70 msgr2=0x7f2f3406e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fa82ffff640 1 -- 172.21.15.2:0/2386419949 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834037070 con 0x7fa8380c8ee0 2024-09-19T05:45:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7ff744f9b640 1 -- 172.21.15.2:0/1762952903 shutdown_connections 2024-09-19T05:45:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fa82ffff640 1 -- 172.21.15.2:0/2386419949 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa834032440 con 0x7fa8380c8ee0 2024-09-19T05:45:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fa82ffff640 1 -- 172.21.15.2:0/2386419949 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834032810 con 0x7fa8380c8ee0 2024-09-19T05:45:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7fba1c63c640 1 Processor -- start 2024-09-19T05:45:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.528+0000 7fba1c63c640 1 -- start start 2024-09-19T05:45:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f4365a78640 1 --2- 172.21.15.2:0/2052946486 >> v2:172.21.15.123:3300/0 conn(0x7f43600c9510 0x7f436006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f4366a7a640 1 -- 172.21.15.2:0/2052946486 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9130 msgr2=0x7f4360071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f4366a7a640 1 --2- 172.21.15.2:0/2052946486 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9130 0x7f4360071fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f4366a7a640 1 -- 172.21.15.2:0/2052946486 >> v2:172.21.15.123:3300/0 conn(0x7f43600c9510 msgr2=0x7f436006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f4366a7a640 1 --2- 172.21.15.2:0/2052946486 >> v2:172.21.15.123:3300/0 conn(0x7f43600c9510 0x7f436006fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7f4366a7a640 1 -- 172.21.15.2:0/2052946486 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436006f0c0 con 0x7f4360071b50 2024-09-19T05:45:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fba1c63c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba141679f0 0x7fba140c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fba1c63c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fba140c9420 0x7fba140c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fba1c63c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fba140c88b0 0x7fba1406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fba1c63c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fba1406f240 con 0x7fba140c88b0 2024-09-19T05:45:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fba1c63c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fba14167dd0 con 0x7fba140c9420 2024-09-19T05:45:24.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7fba1c63c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fba14168030 con 0x7fba141679f0 2024-09-19T05:45:24.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.529+0000 7ff744f9b640 1 -- 172.21.15.2:0/1762952903 wait complete. 2024-09-19T05:45:24.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba141679f0 0x7fba140c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.550 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7f2a83d59640 1 Processor -- start 2024-09-19T05:45:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba141679f0 0x7fba140c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:41048/0 (socket says 172.21.15.2:41048) 2024-09-19T05:45:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 -- 172.21.15.2:0/1626214431 learned_addr learned my addr 172.21.15.2:0/1626214431 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1abd9640 1 --2- 172.21.15.2:0/1626214431 >> v2:172.21.15.2:3300/0 conn(0x7fba140c88b0 0x7fba1406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fa83f835640 1 -- 172.21.15.2:0/2386419949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8380c8ee0 msgr2=0x7fa8380c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fa83f835640 1 --2- 172.21.15.2:0/2386419949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8380c8ee0 0x7fa8380c9300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097a0 tx=0x7fa83402a230 comp rx=0 tx=0).stop 2024-09-19T05:45:24.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2785411005 shutdown_connections 2024-09-19T05:45:24.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7f2a83d59640 1 -- start start 2024-09-19T05:45:24.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7f55de59c640 1 --2- 172.21.15.2:0/3026450088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00cd220 0x7f55e00b5a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009870 tx=0x7f55d002a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=310332a2cdd8cef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 -- 172.21.15.2:0/1626214431 >> v2:172.21.15.123:3300/0 conn(0x7fba140c9420 msgr2=0x7fba140c8370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T05:45:24.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 --2- 172.21.15.2:0/1626214431 >> v2:172.21.15.123:3300/0 conn(0x7fba140c9420 0x7fba140c8370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 -- 172.21.15.2:0/1626214431 >> v2:172.21.15.2:3300/0 conn(0x7fba140c88b0 msgr2=0x7fba1406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 --2- 172.21.15.2:0/1626214431 >> v2:172.21.15.2:3300/0 conn(0x7fba140c88b0 0x7fba1406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fba1a3d8640 1 -- 172.21.15.2:0/1626214431 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14168310 con 0x7fba141679f0 2024-09-19T05:45:24.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fa83f835640 1 -- 172.21.15.2:0/2386419949 shutdown_connections 2024-09-19T05:45:24.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7ff744f9b640 1 Processor -- start 2024-09-19T05:45:24.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7ff744f9b640 1 -- start start 2024-09-19T05:45:24.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f4366a7a640 1 --2- 172.21.15.2:0/2052946486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4360071b50 0x7f4360156b30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f43600c4210 tx=0x7f435802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2641ff3854ae18ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f55dd59a640 1 -- 172.21.15.2:0/3026450088 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0037070 con 0x7f55e00cd220 2024-09-19T05:45:24.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f55dd59a640 1 -- 172.21.15.2:0/3026450088 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d0032640 con 0x7f55e00cd220 2024-09-19T05:45:24.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fa83f835640 1 --2- 172.21.15.2:0/2386419949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa8380c8ee0 0x7fa8380c9300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fa83f835640 1 --2- 172.21.15.2:0/2386419949 >> v2:172.21.15.123:3300/0 conn(0x7fa838159090 0x7fa838159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7fa83f835640 1 -- 172.21.15.2:0/2386419949 >> 172.21.15.2:0/2386419949 conn(0x7fa83806bc10 msgr2=0x7fa83806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7fa83f835640 1 -- 172.21.15.2:0/2386419949 shutdown_connections 2024-09-19T05:45:24.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f4365277640 1 -- 172.21.15.2:0/2052946486 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358031600 con 0x7f4360071b50 2024-09-19T05:45:24.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f2a83d59640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a7404b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f2a83d59640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2a740507a0 0x7f2a74044e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.556 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f4365277640 1 -- 172.21.15.2:0/2052946486 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43580317a0 con 0x7f4360071b50 2024-09-19T05:45:24.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f2a83d59640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a74045860 con 0x7f2a740453d0 2024-09-19T05:45:24.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f2a83d59640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f2a74006490 con 0x7f2a740507a0 2024-09-19T05:45:24.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f2a83d59640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7404bbc0 con 0x7f2a740ddec0 2024-09-19T05:45:24.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.531+0000 7f2a812f4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2a740507a0 0x7f2a74044e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2a812f4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2a740507a0 0x7f2a74044e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54692/0 (socket says 172.21.15.2:54692) 2024-09-19T05:45:24.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2a812f4640 1 -- 172.21.15.2:0/1860446884 learned_addr learned my addr 172.21.15.2:0/1860446884 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.557 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2a81af5640 1 --2- 172.21.15.2:0/1860446884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a7404b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.530+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2785411005 wait complete. 2024-09-19T05:45:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2a812f4640 1 -- 172.21.15.2:0/1860446884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 msgr2=0x7f2a7404b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2a812f4640 1 --2- 172.21.15.2:0/1860446884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a7404b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2a812f4640 1 -- 172.21.15.2:0/1860446884 >> v1:172.21.15.2:6789/0 conn(0x7f2a740453d0 legacy=0x7f2a740d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:24.558 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2a812f4640 1 -- 172.21.15.2:0/1860446884 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7404b250 con 0x7f2a740507a0 2024-09-19T05:45:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f55e48af640 1 -- 172.21.15.2:0/3026450088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00cd220 msgr2=0x7f55e00b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f55e48af640 1 --2- 172.21.15.2:0/3026450088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00cd220 0x7f55e00b5a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009870 tx=0x7f55d002a320 comp rx=0 tx=0).stop 2024-09-19T05:45:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2f38bfc640 1 Processor -- start 2024-09-19T05:45:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f4365277640 1 -- 172.21.15.2:0/2052946486 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358031b70 con 0x7f4360071b50 2024-09-19T05:45:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7fa83f835640 1 -- 172.21.15.2:0/2386419949 wait complete. 2024-09-19T05:45:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f55e48af640 1 -- 172.21.15.2:0/3026450088 shutdown_connections 2024-09-19T05:45:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f55e48af640 1 --2- 172.21.15.2:0/3026450088 >> v2:172.21.15.123:3300/0 conn(0x7f55e00b4630 0x7f55e015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f55e48af640 1 --2- 172.21.15.2:0/3026450088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00cd220 0x7f55e00b5a50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.560 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f55e48af640 1 -- 172.21.15.2:0/3026450088 >> 172.21.15.2:0/3026450088 conn(0x7f55e006d0e0 msgr2=0x7f55e006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7ff744f9b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff74006c700 0x7ff740149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7fba1abd9640 1 --2- 172.21.15.2:0/1626214431 >> v2:172.21.15.2:3300/0 conn(0x7fba140c88b0 0x7fba1406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:24.561 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.532+0000 7f2f38bfc640 1 -- start start 2024-09-19T05:45:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff744f9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7400c6fb0 0x7ff74014af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff744f9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7401577f0 0x7ff74014b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff744f9b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff740156ff0 con 0x7ff7401577f0 2024-09-19T05:45:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff744f9b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff740156d10 con 0x7ff7400c6fb0 2024-09-19T05:45:24.562 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7fba1a3d8640 1 --2- 172.21.15.2:0/1626214431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba141679f0 0x7fba140c8ee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba14074190 tx=0x7fba1002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f18c81eab931e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff744f9b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff7401572d0 con 0x7ff74006c700 2024-09-19T05:45:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f55e48af640 1 -- 172.21.15.2:0/3026450088 shutdown_connections 2024-09-19T05:45:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4360071b50 msgr2=0x7f4360156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.563 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f43684dd640 1 --2- 172.21.15.2:0/2052946486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4360071b50 0x7f4360156b30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f43600c4210 tx=0x7f435802a5d0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff73effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7400c6fb0 0x7ff74014af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff73f7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff74006c700 0x7ff740149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff73effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7400c6fb0 0x7ff74014af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37246/0 (socket says 172.21.15.2:37246) 2024-09-19T05:45:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff73effd640 1 -- 172.21.15.2:0/3714998410 learned_addr learned my addr 172.21.15.2:0/3714998410 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.564 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7ff73ffff640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7401577f0 0x7ff74014b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f2a812f4640 1 --2- 172.21.15.2:0/1860446884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a74044e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009870 tx=0x7f2a6c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f92ee8b10b9441 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.565 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f55e48af640 1 -- 172.21.15.2:0/3026450088 wait complete. 2024-09-19T05:45:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f2f38bfc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c6af0 0x7f2f34072770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f2f38bfc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f340c7ec0 0x7f2f341f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.533+0000 7f2f38bfc640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f34072d20 0x7f2f342004c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c6af0 0x7f2f34072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff73effd640 1 -- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff74006c700 msgr2=0x7ff740149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff73effd640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff74006c700 0x7ff740149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.566 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff73effd640 1 -- 172.21.15.2:0/3714998410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7401577f0 msgr2=0x7ff74014b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff73effd640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7401577f0 0x7ff74014b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2a80af3640 1 -- 172.21.15.2:0/1860446884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c037070 con 0x7f2a740507a0 2024-09-19T05:45:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2a80af3640 1 -- 172.21.15.2:0/1860446884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c032440 con 0x7f2a740507a0 2024-09-19T05:45:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2a80af3640 1 -- 172.21.15.2:0/1860446884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c032810 con 0x7f2a740507a0 2024-09-19T05:45:24.567 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff73effd640 1 -- 172.21.15.2:0/3714998410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff740073660 con 0x7ff7400c6fb0 2024-09-19T05:45:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7fba193d6640 1 -- 172.21.15.2:0/1626214431 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10037070 con 0x7fba141679f0 2024-09-19T05:45:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7fba193d6640 1 -- 172.21.15.2:0/1626214431 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1002ae50 con 0x7fba141679f0 2024-09-19T05:45:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f32ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f340c7ec0 0x7f2f341f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f32ffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f340c7ec0 0x7f2f341f0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54704/0 (socket says 172.21.15.2:54704) 2024-09-19T05:45:24.568 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c6af0 0x7f2f34072770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37262/0 (socket says 172.21.15.2:37262) 2024-09-19T05:45:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff73effd640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7400c6fb0 0x7ff74014af30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c780 tx=0x7ff73000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7fa83f835640 1 Processor -- start 2024-09-19T05:45:24.569 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7fa83f835640 1 -- start start 2024-09-19T05:45:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f32ffd640 1 -- 172.21.15.2:0/2840182609 learned_addr learned my addr 172.21.15.2:0/2840182609 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 shutdown_connections 2024-09-19T05:45:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f43684dd640 1 --2- 172.21.15.2:0/2052946486 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4360071b50 0x7f4360156b30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f43684dd640 1 --2- 172.21.15.2:0/2052946486 >> v2:172.21.15.123:3300/0 conn(0x7f43600c9510 0x7f436006fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f43684dd640 1 --2- 172.21.15.2:0/2052946486 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9130 0x7f4360071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2f340cb860 con 0x7f2f340c7ec0 2024-09-19T05:45:24.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2f340cb6e0 con 0x7f2f340c6af0 2024-09-19T05:45:24.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2f340cb260 con 0x7f2f34072d20 2024-09-19T05:45:24.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 >> 172.21.15.2:0/2052946486 conn(0x7f436006bc10 msgr2=0x7f436006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.571 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f2f33fff640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f34072d20 0x7f2f342004c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff727fff640 1 -- 172.21.15.2:0/3714998410 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730018070 con 0x7ff7400c6fb0 2024-09-19T05:45:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7ff727fff640 1 -- 172.21.15.2:0/3714998410 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7300072d0 con 0x7ff7400c6fb0 2024-09-19T05:45:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7400ccec0 con 0x7ff7400c6fb0 2024-09-19T05:45:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7400732f0 con 0x7ff7400c6fb0 2024-09-19T05:45:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 shutdown_connections 2024-09-19T05:45:24.572 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.534+0000 7f43684dd640 1 -- 172.21.15.2:0/2052946486 wait complete. 2024-09-19T05:45:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2f337fe640 1 -- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f34072d20 msgr2=0x7f2f342004c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2f337fe640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f34072d20 0x7f2f342004c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2f337fe640 1 -- 172.21.15.2:0/2840182609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f340c7ec0 msgr2=0x7f2f341f0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.573 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2a83d59640 1 -- 172.21.15.2:0/1860446884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 msgr2=0x7f2a74044e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1860446884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a74044e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009870 tx=0x7f2a6c02a180 comp rx=0 tx=0).stop 2024-09-19T05:45:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2f337fe640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f340c7ec0 0x7f2f341f0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2f337fe640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3406efd0 con 0x7f2f340c6af0 2024-09-19T05:45:24.574 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7ff727fff640 1 -- 172.21.15.2:0/3714998410 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300047e0 con 0x7ff7400c6fb0 2024-09-19T05:45:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2f337fe640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c6af0 0x7f2f34072770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e870 tx=0x7f2f2800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2a83d59640 1 -- 172.21.15.2:0/1860446884 shutdown_connections 2024-09-19T05:45:24.575 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f55e48af640 1 Processor -- start 2024-09-19T05:45:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fa83f835640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa83806f150 0x7fa838160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2f32ffd640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f340c7ec0 0x7f2f341f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:24.576 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1860446884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a74044e90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fa83f835640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8380c8ee0 0x7fa838162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1860446884 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a7404b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.577 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7f2a83d59640 1 -- 172.21.15.2:0/1860446884 >> 172.21.15.2:0/1860446884 conn(0x7f2a7404fc80 msgr2=0x7f2a74050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fa83f835640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa838159090 0x7fa8381541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fa83f835640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815fca0 con 0x7fa8380c8ee0 2024-09-19T05:45:24.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fa83f835640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815f9c0 con 0x7fa838159090 2024-09-19T05:45:24.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fa83f835640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa83806f600 con 0x7fa83806f150 2024-09-19T05:45:24.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 -- 172.21.15.2:0/1626214431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba141679f0 msgr2=0x7fba140c8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 --2- 172.21.15.2:0/1626214431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba141679f0 0x7fba140c8ee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fba14074190 tx=0x7fba1002a100 comp rx=0 tx=0).stop 2024-09-19T05:45:24.578 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 -- 172.21.15.2:0/1626214431 shutdown_connections 2024-09-19T05:45:24.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 --2- 172.21.15.2:0/1626214431 >> v2:172.21.15.2:3300/0 conn(0x7fba140c88b0 0x7fba1406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 --2- 172.21.15.2:0/1626214431 >> v2:172.21.15.123:3300/0 conn(0x7fba140c9420 0x7fba140c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 --2- 172.21.15.2:0/1626214431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba141679f0 0x7fba140c8ee0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 -- 172.21.15.2:0/1626214431 >> 172.21.15.2:0/1626214431 conn(0x7fba1406bc10 msgr2=0x7fba1406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.535+0000 7fba1c63c640 1 -- 172.21.15.2:0/1626214431 shutdown_connections 2024-09-19T05:45:24.579 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7fa83ddd2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa838159090 0x7fa8381541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7f2f1bfff640 1 -- 172.21.15.2:0/2840182609 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28017070 con 0x7f2f340c6af0 2024-09-19T05:45:24.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7f2f1bfff640 1 -- 172.21.15.2:0/2840182609 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f2800c940 con 0x7f2f340c6af0 2024-09-19T05:45:24.580 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7fa83ddd2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa838159090 0x7fa8381541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37276/0 (socket says 172.21.15.2:37276) 2024-09-19T05:45:24.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7fa83ddd2640 1 -- 172.21.15.2:0/4178120825 learned_addr learned my addr 172.21.15.2:0/4178120825 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7fa83d5d1640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa83806f150 0x7fa838160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7f2f1bfff640 1 -- 172.21.15.2:0/2840182609 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800ccf0 con 0x7f2f340c6af0 2024-09-19T05:45:24.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3406f2b0 con 0x7f2f340c6af0 2024-09-19T05:45:24.581 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3415e7e0 con 0x7f2f340c6af0 2024-09-19T05:45:24.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f2a83d59640 1 -- 172.21.15.2:0/1860446884 shutdown_connections 2024-09-19T05:45:24.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f43684dd640 1 Processor -- start 2024-09-19T05:45:24.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f43684dd640 1 -- start start 2024-09-19T05:45:24.582 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f2a83d59640 1 -- 172.21.15.2:0/1860446884 wait complete. 2024-09-19T05:45:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7f55e48af640 1 -- start start 2024-09-19T05:45:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f55e48af640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00b30e0 0x7f55e0159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f55e48af640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55e00b4630 0x7f55e01f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.536+0000 7fba1c63c640 1 -- 172.21.15.2:0/1626214431 wait complete. 2024-09-19T05:45:24.583 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff720005290 con 0x7ff7400c6fb0 2024-09-19T05:45:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f340c3c60 con 0x7f2f340c6af0 2024-09-19T05:45:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7fa83ddd2640 1 -- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa83806f150 msgr2=0x7fa838160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7fa83ddd2640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa83806f150 0x7fa838160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7fa83ddd2640 1 -- 172.21.15.2:0/4178120825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8380c8ee0 msgr2=0x7fa838162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.584 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7fa83ddd2640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8380c8ee0 0x7fa838162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7fa83ddd2640 1 -- 172.21.15.2:0/4178120825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83815dcb0 con 0x7fa838159090 2024-09-19T05:45:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7fa83ddd2640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa838159090 0x7fa8381541d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa83000cb40 tx=0x7fa83000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7fba1c63c640 1 Processor -- start 2024-09-19T05:45:24.585 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7fba1c63c640 1 -- start start 2024-09-19T05:45:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f43684dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4360071b50 0x7f4360158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f43684dd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43600c9130 0x7f43600cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f43684dd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9510 0x7f43601f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f43684dd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f43600c8920 con 0x7f43600c9130 2024-09-19T05:45:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f43684dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43600c87a0 con 0x7f4360071b50 2024-09-19T05:45:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f43684dd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f43600c8aa0 con 0x7f43600c9510 2024-09-19T05:45:24.586 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f2f1bfff640 1 -- 172.21.15.2:0/2840182609 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f2801a070 con 0x7f2f340c6af0 2024-09-19T05:45:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4360071b50 0x7f4360158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f55de59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00b30e0 0x7f55e0159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.537+0000 7f55de59c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00b30e0 0x7f55e0159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:41108/0 (socket says 172.21.15.2:41108) 2024-09-19T05:45:24.587 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4360071b50 0x7f4360158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37304/0 (socket says 172.21.15.2:37304) 2024-09-19T05:45:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 -- 172.21.15.2:0/2235102966 learned_addr learned my addr 172.21.15.2:0/2235102966 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366a7a640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9510 0x7f43601f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4365a78640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43600c9130 0x7f43600cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f2a83d59640 1 Processor -- start 2024-09-19T05:45:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55e48af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e00cd220 0x7f55e00cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55e48af640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f55e00b5f90 con 0x7f55e00b4630 2024-09-19T05:45:24.588 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55e48af640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55e0163340 con 0x7f55e00cd220 2024-09-19T05:45:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f2a83d59640 1 -- start start 2024-09-19T05:45:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55e48af640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f55e00cd600 con 0x7f55e00b30e0 2024-09-19T05:45:24.589 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f2f1bfff640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2f2006c380 0x7f2f2006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 -- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9510 msgr2=0x7f43601f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9510 0x7f43601f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 -- 172.21.15.2:0/2235102966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43600c9130 msgr2=0x7f43600cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43600c9130 0x7f43600cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366279640 1 -- 172.21.15.2:0/2235102966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436015ac70 con 0x7f4360071b50 2024-09-19T05:45:24.590 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f4366a7a640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9510 0x7f43601f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f2f1bfff640 1 -- 172.21.15.2:0/2840182609 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f2f2808b740 con 0x7f2f340c6af0 2024-09-19T05:45:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f2f32ffd640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2f2006c380 0x7f2f2006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55de59c640 1 -- 172.21.15.2:0/1156995735 learned_addr learned my addr 172.21.15.2:0/1156995735 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55ddd9b640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55e00b4630 0x7f55e01f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.591 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55ded9d640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e00cd220 0x7f55e00cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55de59c640 1 -- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e00cd220 msgr2=0x7f55e00cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55de59c640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e00cd220 0x7f55e00cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55de59c640 1 -- 172.21.15.2:0/1156995735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55e00b4630 msgr2=0x7f55e01f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55de59c640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55e00b4630 0x7f55e01f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.538+0000 7f55de59c640 1 -- 172.21.15.2:0/1156995735 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e015a600 con 0x7f55e00b30e0 2024-09-19T05:45:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.539+0000 7f55ddd9b640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55e00b4630 0x7f55e01f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:24.592 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba1c63c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba140c88b0 0x7fba14163720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7f4366279640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4360071b50 0x7f4360158700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f435c00c7c0 tx=0x7f435c00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fa82dffb640 1 -- 172.21.15.2:0/4178120825 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830017070 con 0x7fa838159090 2024-09-19T05:45:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba1a3d8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba140c88b0 0x7fba14163720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.593 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba1a3d8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba140c88b0 0x7fba14163720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54748/0 (socket says 172.21.15.2:54748) 2024-09-19T05:45:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba1c63c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba140c9420 0x7fba14163c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7f2f32ffd640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2f2006c380 0x7f2f2006e840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f3415a650 tx=0x7f2f24008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba1a3d8640 1 -- 172.21.15.2:0/3779330282 learned_addr learned my addr 172.21.15.2:0/3779330282 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7f55de59c640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00b30e0 0x7f55e0159cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009870 tx=0x7f55d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba1c63c640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba14166a10 0x7fba141621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7f2a83d59640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a740453d0 0x7f2a74046f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fa82dffb640 1 -- 172.21.15.2:0/4178120825 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83000aca0 con 0x7fa838159090 2024-09-19T05:45:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83815df90 con 0x7fa838159090 2024-09-19T05:45:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba19bd7640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba140c9420 0x7fba14163c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83815e550 con 0x7fa838159090 2024-09-19T05:45:24.596 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7f2a83d59640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a740d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7fba1abd9640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba14166a10 0x7fba141621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7fba1c63c640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fba1406fc10 con 0x7fba140c88b0 2024-09-19T05:45:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a740453d0 0x7f2a74046f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a740453d0 0x7f2a74046f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.2:37320/0 (socket says 172.21.15.2:37320) 2024-09-19T05:45:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 -- 172.21.15.2:0/1504000837 learned_addr learned my addr 172.21.15.2:0/1504000837 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a812f4640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a740d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a740d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7fba1c63c640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba141688d0 con 0x7fba140c9420 2024-09-19T05:45:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f55ceffd640 1 -- 172.21.15.2:0/1156995735 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0052020 con 0x7f55e00b30e0 2024-09-19T05:45:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7ff727fff640 1 -- 172.21.15.2:0/3714998410 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff73001b070 con 0x7ff7400c6fb0 2024-09-19T05:45:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.540+0000 7ff727fff640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff72c06c3b0 0x7ff72c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7ff727fff640 1 -- 172.21.15.2:0/3714998410 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7ff73008aa40 con 0x7ff7400c6fb0 2024-09-19T05:45:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7fba1c63c640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fba1406fd90 con 0x7fba14166a10 2024-09-19T05:45:24.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e01f9110 con 0x7f55e00b30e0 2024-09-19T05:45:24.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7fa82dffb640 1 -- 172.21.15.2:0/4178120825 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830017070 con 0x7fa838159090 2024-09-19T05:45:24.600 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7ff73f7fe640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff72c06c3b0 0x7ff72c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e015a320 con 0x7f55e00b30e0 2024-09-19T05:45:24.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74006ca0 con 0x7f2a740507a0 2024-09-19T05:45:24.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a740069c0 con 0x7f2a740453d0 2024-09-19T05:45:24.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2a74006f80 con 0x7f2a740ddec0 2024-09-19T05:45:24.601 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a822f6640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a740d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f55ceffd640 1 -- 172.21.15.2:0/1156995735 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d004e630 con 0x7f55e00b30e0 2024-09-19T05:45:24.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 -- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 msgr2=0x7f2a740d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a740d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 -- 172.21.15.2:0/1504000837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 msgr2=0x7f2a740d96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.602 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a740d96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a81af5640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a74049990 con 0x7f2a740453d0 2024-09-19T05:45:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7f2a812f4640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a740d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7fba1a3d8640 1 -- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba14166a10 msgr2=0x7fba141621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7fba1a3d8640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba14166a10 0x7fba141621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7fba1a3d8640 1 -- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba140c9420 msgr2=0x7fba14163c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7fba1a3d8640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba140c9420 0x7fba14163c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.603 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7fba1a3d8640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14165080 con 0x7fba140c88b0 2024-09-19T05:45:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7f2a81af5640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a740453d0 0x7f2a74046f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d860 tx=0x7f2a7800dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7fba1a3d8640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba140c88b0 0x7fba14163720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fba100097c0 tx=0x7fba100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.604 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7f55ceffd640 1 -- 172.21.15.2:0/1156995735 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0037040 con 0x7f55e00b30e0 2024-09-19T05:45:24.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.541+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838158790 con 0x7fa838159090 2024-09-19T05:45:24.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7fba0affd640 1 -- 172.21.15.2:0/3779330282 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10041020 con 0x7fba140c88b0 2024-09-19T05:45:24.605 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.543+0000 7fba0affd640 1 -- 172.21.15.2:0/3779330282 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1003d620 con 0x7fba140c88b0 2024-09-19T05:45:24.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.542+0000 7ff73f7fe640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff72c06c3b0 0x7ff72c06e870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff738033000 tx=0x7ff73802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.543+0000 7fba1c63c640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba14165360 con 0x7fba140c88b0 2024-09-19T05:45:24.606 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.543+0000 7fba0affd640 1 -- 172.21.15.2:0/3779330282 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10037040 con 0x7fba140c88b0 2024-09-19T05:45:24.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.543+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c4005290 con 0x7f55e00b30e0 2024-09-19T05:45:24.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.543+0000 7f55ceffd640 1 -- 172.21.15.2:0/1156995735 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f55d0055020 con 0x7f55e00b30e0 2024-09-19T05:45:24.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7fba1c63c640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba141793f0 con 0x7fba140c88b0 2024-09-19T05:45:24.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7f2a727fc640 1 -- 172.21.15.2:0/1504000837 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800de50 con 0x7f2a740453d0 2024-09-19T05:45:24.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7f4352ffd640 1 -- 172.21.15.2:0/2235102966 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c01a070 con 0x7f4360071b50 2024-09-19T05:45:24.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a740db170 con 0x7f2a740453d0 2024-09-19T05:45:24.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a740d8bf0 con 0x7f2a740453d0 2024-09-19T05:45:24.608 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7f2f1bfff640 1 -- 172.21.15.2:0/2840182609 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2f340c3c60 con 0x7f2f340c6af0 2024-09-19T05:45:24.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7f55ceffd640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f55c006c300 0x7f55c006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.545+0000 7f55ddd9b640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f55c006c300 0x7f55c006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.544+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4360156760 con 0x7f4360071b50 2024-09-19T05:45:24.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.545+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4360156a00 con 0x7f4360071b50 2024-09-19T05:45:24.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.545+0000 7fba0affd640 1 -- 172.21.15.2:0/3779330282 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fba10044020 con 0x7fba140c88b0 2024-09-19T05:45:24.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.545+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7404ae30 con 0x7f2a740453d0 2024-09-19T05:45:24.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.545+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f0005290 con 0x7fba140c88b0 2024-09-19T05:45:24.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.545+0000 7f55ceffd640 1 -- 172.21.15.2:0/1156995735 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f55d00bd4b0 con 0x7f55e00b30e0 2024-09-19T05:45:24.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.546+0000 7f55ddd9b640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f55c006c300 0x7f55c006e7c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f55d8007fd0 tx=0x7f55d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.546+0000 7fba0affd640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb9f806c2b0 0x7fb9f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.546+0000 7fba0affd640 1 -- 172.21.15.2:0/3779330282 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fba100b0060 con 0x7fba140c88b0 2024-09-19T05:45:24.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.546+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4360071fa0 con 0x7f4360071b50 2024-09-19T05:45:24.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.546+0000 7fba19bd7640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb9f806c2b0 0x7fb9f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.547+0000 7fba19bd7640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb9f806c2b0 0x7fb9f806e770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fba04006fd0 tx=0x7fba04008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.548+0000 7f4352ffd640 1 -- 172.21.15.2:0/2235102966 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c005990 con 0x7f4360071b50 2024-09-19T05:45:24.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.548+0000 7ff727fff640 1 -- 172.21.15.2:0/3714998410 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff73008ce20 con 0x7ff7400c6fb0 2024-09-19T05:45:24.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.548+0000 7f4352ffd640 1 -- 172.21.15.2:0/2235102966 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c005d40 con 0x7f4360071b50 2024-09-19T05:45:24.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.549+0000 7f55ceffd640 1 -- 172.21.15.2:0/1156995735 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f55d005c030 con 0x7f55e00b30e0 2024-09-19T05:45:24.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.549+0000 7f2a727fc640 1 -- 172.21.15.2:0/1504000837 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a78019070 con 0x7f2a740453d0 2024-09-19T05:45:24.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.549+0000 7f2a727fc640 1 -- 172.21.15.2:0/1504000837 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800de50 con 0x7f2a740453d0 2024-09-19T05:45:24.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.550+0000 7fa82dffb640 1 -- 172.21.15.2:0/4178120825 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa83001a070 con 0x7fa838159090 2024-09-19T05:45:24.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.551+0000 7fa82dffb640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa81c06c4e0 0x7fa81c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.551+0000 7fa82dffb640 1 -- 172.21.15.2:0/4178120825 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fa8300574c0 con 0x7fa838159090 2024-09-19T05:45:24.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.551+0000 7f2a727fc640 1 -- 172.21.15.2:0/1504000837 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2a7800b8d0 con 0x7f2a740453d0 2024-09-19T05:45:24.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.551+0000 7f2a727fc640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a6006c4b0 0x7f2a6006e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.551+0000 7f4352ffd640 1 -- 172.21.15.2:0/2235102966 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f435c01d070 con 0x7f4360071b50 2024-09-19T05:45:24.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.552+0000 7fa83d5d1640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa81c06c4e0 0x7fa81c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.552+0000 7f4352ffd640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f434406c2b0 0x7f434406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:24.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.552+0000 7f2a812f4640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a6006c4b0 0x7f2a6006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.552+0000 7f4365a78640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f434406c2b0 0x7f434406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:24.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.553+0000 7f4365a78640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f434406c2b0 0x7f434406e770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f43600ccd60 tx=0x7f4354009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.552+0000 7f2a812f4640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a6006c4b0 0x7f2a6006e970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c034000 tx=0x7f2a6c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.552+0000 7fa83d5d1640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa81c06c4e0 0x7fa81c06e9a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa828009930 tx=0x7fa828009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:24.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.553+0000 7fba0affd640 1 -- 172.21.15.2:0/3779330282 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fba10079b70 con 0x7fba140c88b0 2024-09-19T05:45:24.618 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.554+0000 7f4352ffd640 1 -- 172.21.15.2:0/2235102966 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f435c056c20 con 0x7f4360071b50 2024-09-19T05:45:24.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.554+0000 7fa82dffb640 1 -- 172.21.15.2:0/4178120825 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa838158790 con 0x7fa838159090 2024-09-19T05:45:24.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.554+0000 7f2a727fc640 1 -- 172.21.15.2:0/1504000837 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f2a78057fe0 con 0x7f2a740453d0 2024-09-19T05:45:24.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.558+0000 7f4352ffd640 1 -- 172.21.15.2:0/2235102966 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f435c024080 con 0x7f4360071b50 2024-09-19T05:45:24.620 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.563+0000 7f2a727fc640 1 -- 172.21.15.2:0/1504000837 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2a740db170 con 0x7f2a740453d0 2024-09-19T05:45:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.901+0000 7fd5f0e27640 1 -- 172.21.15.2:0/1627724995 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd5d0005580 con 0x7fd5ec072090 2024-09-19T05:45:24.904 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.902+0000 7fd5daffd640 1 -- 172.21.15.2:0/1627724995 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5e405d120 con 0x7fd5ec072090 2024-09-19T05:45:24.904 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.906+0000 7fd5d8ff9640 1 -- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd5cc06c3b0 msgr2=0x7fd5cc06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.906+0000 7fd5d8ff9640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd5cc06c3b0 0x7fd5cc06e870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec0c9660 tx=0x7fd5dc008040 comp rx=0 tx=0).stop 2024-09-19T05:45:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.906+0000 7fd5d8ff9640 1 -- 172.21.15.2:0/1627724995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 msgr2=0x7fd5ec1e0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.906+0000 7fd5d8ff9640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec1e0e90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400cb60 tx=0x7fd5e400a430 comp rx=0 tx=0).stop 2024-09-19T05:45:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.906+0000 7fd5d8ff9640 1 -- 172.21.15.2:0/1627724995 shutdown_connections 2024-09-19T05:45:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.907+0000 7fd5d8ff9640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd5cc06c3b0 0x7fd5cc06e870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.907+0000 7fd5d8ff9640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5ec150d20 0x7fd5ec1e4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.907+0000 7fd5d8ff9640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd5ec072650 0x7fd5ec1fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.907+0000 7fd5d8ff9640 1 --2- 172.21.15.2:0/1627724995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd5ec072090 0x7fd5ec1e0e90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.907+0000 7fd5d8ff9640 1 -- 172.21.15.2:0/1627724995 >> 172.21.15.2:0/1627724995 conn(0x7fd5ec06bc10 msgr2=0x7fd5ec06ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.907+0000 7fd5d8ff9640 1 -- 172.21.15.2:0/1627724995 shutdown_connections 2024-09-19T05:45:24.910 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.907+0000 7fd5d8ff9640 1 -- 172.21.15.2:0/1627724995 wait complete. 2024-09-19T05:45:24.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542183 for osd.1 2024-09-19T05:45:24.929 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.937 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.936+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f55c4005580 con 0x7f55e00b30e0 2024-09-19T05:45:24.938 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.936+0000 7f55ceffd640 1 -- 172.21.15.2:0/1156995735 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55d008bcb0 con 0x7f55e00b30e0 2024-09-19T05:45:24.938 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.938+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f55c006c300 msgr2=0x7f55c006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f55c006c300 0x7f55c006e7c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f55d8007fd0 tx=0x7f55d8009040 comp rx=0 tx=0).stop 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00b30e0 msgr2=0x7f55e0159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00b30e0 0x7f55e0159cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009870 tx=0x7f55d00046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 shutdown_connections 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f55c006c300 0x7f55c006e7c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55e00cd220 0x7f55e00cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55e00b4630 0x7f55e01f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 --2- 172.21.15.2:0/1156995735 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55e00b30e0 0x7f55e0159cd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 >> 172.21.15.2:0/1156995735 conn(0x7f55e006d0e0 msgr2=0x7f55e0154ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 shutdown_connections 2024-09-19T05:45:24.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.939+0000 7f55e48af640 1 -- 172.21.15.2:0/1156995735 wait complete. 2024-09-19T05:45:24.943 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.941+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2f340d02e0 con 0x7f2f340c6af0 2024-09-19T05:45:24.944 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.942+0000 7f2f1bfff640 1 -- 172.21.15.2:0/2840182609 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f28059750 con 0x7f2f340c6af0 2024-09-19T05:45:24.944 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:24.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.945+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2f2006c380 msgr2=0x7f2f2006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.945+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2f2006c380 0x7f2f2006e840 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2f3415a650 tx=0x7f2f24008040 comp rx=0 tx=0).stop 2024-09-19T05:45:24.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.945+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c6af0 msgr2=0x7f2f34072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.945+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c6af0 0x7f2f34072770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e870 tx=0x7f2f2800ed30 comp rx=0 tx=0).stop 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.945+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 shutdown_connections 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.946+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2f2006c380 0x7f2f2006e840 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.946+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2f34072d20 0x7f2f342004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.946+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f340c7ec0 0x7f2f341f0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.946+0000 7f2f38bfc640 1 --2- 172.21.15.2:0/2840182609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f340c6af0 0x7f2f34072770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.946+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 >> 172.21.15.2:0/2840182609 conn(0x7f2f3406cb70 msgr2=0x7f2f340c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.946+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 shutdown_connections 2024-09-19T05:45:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.946+0000 7f2f38bfc640 1 -- 172.21.15.2:0/2840182609 wait complete. 2024-09-19T05:45:24.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542183 for osd.4 2024-09-19T05:45:24.955 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542183 for osd.3 2024-09-19T05:45:24.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.962+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff720005580 con 0x7ff7400c6fb0 2024-09-19T05:45:24.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.963+0000 7ff727fff640 1 -- 172.21.15.2:0/3714998410 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff730059130 con 0x7ff7400c6fb0 2024-09-19T05:45:24.965 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.965 INFO:teuthology.orchestra.run.smithi002.stdout:335007449096 2024-09-19T05:45:24.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.965+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2a7404d1d0 con 0x7f2a740453d0 2024-09-19T05:45:24.967 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.965+0000 7f2a727fc640 1 -- 172.21.15.2:0/1504000837 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2a7805a640 con 0x7f2a740453d0 2024-09-19T05:45:24.968 INFO:teuthology.orchestra.run.smithi002.stdout:60129542184 2024-09-19T05:45:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff72c06c3b0 msgr2=0x7ff72c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff72c06c3b0 0x7ff72c06e870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff738033000 tx=0x7ff73802f000 comp rx=0 tx=0).stop 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7400c6fb0 msgr2=0x7ff74014af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7400c6fb0 0x7ff74014af30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff73000c780 tx=0x7ff73000cc40 comp rx=0 tx=0).stop 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 shutdown_connections 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff72c06c3b0 0x7ff72c06e870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7400c6fb0 0x7ff74014af30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7401577f0 0x7ff74014b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 --2- 172.21.15.2:0/3714998410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff74006c700 0x7ff740149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 >> 172.21.15.2:0/3714998410 conn(0x7ff74006bc10 msgr2=0x7ff740072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 shutdown_connections 2024-09-19T05:45:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.966+0000 7ff744f9b640 1 -- 172.21.15.2:0/3714998410 wait complete. 2024-09-19T05:45:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a6006c4b0 msgr2=0x7f2a6006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a6006c4b0 0x7f2a6006e970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c034000 tx=0x7f2a6c02e000 comp rx=0 tx=0).stop 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a740453d0 msgr2=0x7f2a74046f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a740453d0 0x7f2a74046f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d860 tx=0x7f2a7800dd20 comp rx=0 tx=0).stop 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 shutdown_connections 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f2a6006c4b0 0x7f2a6006e970 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2a740ddec0 0x7f2a740d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a740507a0 0x7f2a740d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 --2- 172.21.15.2:0/1504000837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a740453d0 0x7f2a74046f80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 >> 172.21.15.2:0/1504000837 conn(0x7f2a7404fc80 msgr2=0x7f2a7404c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 shutdown_connections 2024-09-19T05:45:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7f2a83d59640 1 -- 172.21.15.2:0/1504000837 wait complete. 2024-09-19T05:45:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.968+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb9f0005850 con 0x7fba140c88b0 2024-09-19T05:45:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.969+0000 7fba0affd640 1 -- 172.21.15.2:0/3779330282 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba10079470 con 0x7fba140c88b0 2024-09-19T05:45:24.972 INFO:teuthology.orchestra.run.smithi002.stdout:60129542184 2024-09-19T05:45:24.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.970+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa83806f9d0 con 0x7fa838159090 2024-09-19T05:45:24.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.971+0000 7fa82dffb640 1 -- 172.21.15.2:0/4178120825 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa838158790 con 0x7fa838159090 2024-09-19T05:45:24.973 INFO:teuthology.orchestra.run.smithi002.stdout:64424509480 2024-09-19T05:45:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb9f806c2b0 msgr2=0x7fb9f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb9f806c2b0 0x7fb9f806e770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fba04006fd0 tx=0x7fba04008040 comp rx=0 tx=0).stop 2024-09-19T05:45:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba140c88b0 msgr2=0x7fba14163720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba140c88b0 0x7fba14163720 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fba100097c0 tx=0x7fba100046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 shutdown_connections 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fb9f806c2b0 0x7fb9f806e770 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba140c9420 0x7fba14163c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba14166a10 0x7fba141621d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 --2- 172.21.15.2:0/3779330282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fba140c88b0 0x7fba14163720 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 >> 172.21.15.2:0/3779330282 conn(0x7fba1406bc10 msgr2=0x7fba140cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 shutdown_connections 2024-09-19T05:45:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.972+0000 7fba08ff9640 1 -- 172.21.15.2:0/3779330282 wait complete. 2024-09-19T05:45:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f43600c4e60 con 0x7f4360071b50 2024-09-19T05:45:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa81c06c4e0 msgr2=0x7fa81c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa81c06c4e0 0x7fa81c06e9a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa828009930 tx=0x7fa828009060 comp rx=0 tx=0).stop 2024-09-19T05:45:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa838159090 msgr2=0x7fa8381541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa838159090 0x7fa8381541d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa83000cb40 tx=0x7fa83000a060 comp rx=0 tx=0).stop 2024-09-19T05:45:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 shutdown_connections 2024-09-19T05:45:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa81c06c4e0 0x7fa81c06e9a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa838159090 0x7fa8381541d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8380c8ee0 0x7fa838162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 --2- 172.21.15.2:0/4178120825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa83806f150 0x7fa838160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.974+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 >> 172.21.15.2:0/4178120825 conn(0x7fa83806bc10 msgr2=0x7fa838158080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.975+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 shutdown_connections 2024-09-19T05:45:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.975+0000 7fa83f835640 1 -- 172.21.15.2:0/4178120825 wait complete. 2024-09-19T05:45:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.975+0000 7f4352ffd640 1 -- 172.21.15.2:0/2235102966 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f435c058b80 con 0x7f4360071b50 2024-09-19T05:45:24.979 INFO:teuthology.orchestra.run.smithi002.stdout:60129542183 2024-09-19T05:45:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.977+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f434406c2b0 msgr2=0x7f434406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.977+0000 7f43684dd640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f434406c2b0 0x7f434406e770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f43600ccd60 tx=0x7f4354009190 comp rx=0 tx=0).stop 2024-09-19T05:45:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.977+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4360071b50 msgr2=0x7f4360158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.977+0000 7f43684dd640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4360071b50 0x7f4360158700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f435c00c7c0 tx=0x7f435c00cc80 comp rx=0 tx=0).stop 2024-09-19T05:45:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 shutdown_connections 2024-09-19T05:45:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f434406c2b0 0x7f434406e770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43600c9510 0x7f43601f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43600c9130 0x7f43600cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 --2- 172.21.15.2:0/2235102966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4360071b50 0x7f4360158700 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 >> 172.21.15.2:0/2235102966 conn(0x7f436006bc10 msgr2=0x7f436014e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 shutdown_connections 2024-09-19T05:45:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:24.978+0000 7f43684dd640 1 -- 172.21.15.2:0/2235102966 wait complete. 2024-09-19T05:45:24.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 335007449096 got 335007449096 for osd.5 2024-09-19T05:45:24.985 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542184 got 60129542184 for osd.0 2024-09-19T05:45:24.986 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542183 got 60129542183 for osd.7 2024-09-19T05:45:24.994 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542184 got 60129542184 for osd.6 2024-09-19T05:45:24.995 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509480 got 64424509480 for osd.2 2024-09-19T05:45:24.996 DEBUG:teuthology.parallel:result is None 2024-09-19T05:45:24.997 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-19T05:45:24.997 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.138+0000 7fdf07f59640 1 Processor -- start 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.138+0000 7fdf07f59640 1 -- start start 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf07f59640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdf000cce60 0x7fdf0015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf07f59640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf0015afe0 0x7fdf000cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf07f59640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf000cd780 con 0x7fdf00154e70 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf07f59640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fdf000cd260 con 0x7fdf000cce60 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf07f59640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdf00155250 con 0x7fdf0015afe0 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf054f4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdf000cce60 0x7fdf0015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.142 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf054f4640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdf000cce60 0x7fdf0015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54766/0 (socket says 172.21.15.2:54766) 2024-09-19T05:45:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf054f4640 1 -- 172.21.15.2:0/1766360279 learned_addr learned my addr 172.21.15.2:0/1766360279 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf064f6640 1 --2- 172.21.15.2:0/1766360279 >> v2:172.21.15.123:3300/0 conn(0x7fdf0015afe0 0x7fdf000cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf054f4640 1 -- 172.21.15.2:0/1766360279 >> v2:172.21.15.123:3300/0 conn(0x7fdf0015afe0 msgr2=0x7fdf000cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf054f4640 1 --2- 172.21.15.2:0/1766360279 >> v2:172.21.15.123:3300/0 conn(0x7fdf0015afe0 0x7fdf000cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf054f4640 1 -- 172.21.15.2:0/1766360279 >> v1:172.21.15.123:6789/0 conn(0x7fdf00154e70 legacy=0x7fdf000cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.139+0000 7fdf054f4640 1 -- 172.21.15.2:0/1766360279 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf000cd4a0 con 0x7fdf000cce60 2024-09-19T05:45:25.143 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.140+0000 7fdf054f4640 1 --2- 172.21.15.2:0/1766360279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf000cce60 0x7fdf0015aaa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf000cdf90 tx=0x7fdef002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b188817babba637e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.140+0000 7fdf04cf3640 1 -- 172.21.15.2:0/1766360279 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0037070 con 0x7fdf000cce60 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.140+0000 7fdf04cf3640 1 -- 172.21.15.2:0/1766360279 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef0032440 con 0x7fdf000cce60 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.140+0000 7fdf04cf3640 1 -- 172.21.15.2:0/1766360279 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef0032810 con 0x7fdf000cce60 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 -- 172.21.15.2:0/1766360279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf000cce60 msgr2=0x7fdf0015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1766360279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf000cce60 0x7fdf0015aaa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf000cdf90 tx=0x7fdef002a2a0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 -- 172.21.15.2:0/1766360279 shutdown_connections 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1766360279 >> v2:172.21.15.123:3300/0 conn(0x7fdf0015afe0 0x7fdf000cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1766360279 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf000cce60 0x7fdf0015aaa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 -- 172.21.15.2:0/1766360279 >> 172.21.15.2:0/1766360279 conn(0x7fdf0006da40 msgr2=0x7fdf00073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 -- 172.21.15.2:0/1766360279 shutdown_connections 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.141+0000 7fdf07f59640 1 -- 172.21.15.2:0/1766360279 wait complete. 2024-09-19T05:45:25.147 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.142+0000 7fdf07f59640 1 Processor -- start 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.142+0000 7fdf07f59640 1 -- start start 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf07f59640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf000cce60 0x7fdf0014ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf07f59640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf00154e70 0x7fdf0014b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf07f59640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf0015afe0 0x7fdf000c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf07f59640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0015e670 con 0x7fdf0015afe0 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf07f59640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0015e390 con 0x7fdf000cce60 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf07f59640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdf0015e950 con 0x7fdf00154e70 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf0015afe0 0x7fdf000c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf0015afe0 0x7fdf000c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54778/0 (socket says 172.21.15.2:54778) 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 -- 172.21.15.2:0/1114794742 learned_addr learned my addr 172.21.15.2:0/1114794742 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf054f4640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf00154e70 0x7fdf0014b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf05cf5640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf000cce60 0x7fdf0014ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.148 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 -- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf00154e70 msgr2=0x7fdf0014b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf00154e70 0x7fdf0014b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 -- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf000cce60 msgr2=0x7fdf0014ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf000cce60 0x7fdf0014ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 -- 172.21.15.2:0/1114794742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf00073480 con 0x7fdf0015afe0 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdf064f6640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf0015afe0 0x7fdf000c84e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdef8007d80 tx=0x7fdef800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdef67fc640 1 -- 172.21.15.2:0/1114794742 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8014420 con 0x7fdf0015afe0 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.143+0000 7fdef67fc640 1 -- 172.21.15.2:0/1114794742 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdef8019070 con 0x7fdf0015afe0 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.144+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0015abc0 con 0x7fdf0015afe0 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.144+0000 7fdef67fc640 1 -- 172.21.15.2:0/1114794742 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8014800 con 0x7fdf0015afe0 2024-09-19T05:45:25.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.144+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0014d560 con 0x7fdf0015afe0 2024-09-19T05:45:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.144+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf000ca790 con 0x7fdf0015afe0 2024-09-19T05:45:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.145+0000 7fdef67fc640 1 -- 172.21.15.2:0/1114794742 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdef80149a0 con 0x7fdf0015afe0 2024-09-19T05:45:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.146+0000 7fdef67fc640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdee406c2b0 0x7fdee406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.146+0000 7fdef67fc640 1 -- 172.21.15.2:0/1114794742 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fdef808ba90 con 0x7fdf0015afe0 2024-09-19T05:45:25.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.146+0000 7fdf05cf5640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdee406c2b0 0x7fdee406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.151 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.147+0000 7fdf05cf5640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdee406c2b0 0x7fdee406e770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdefc001780 tx=0x7fdefc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.152 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.150+0000 7fdef67fc640 1 -- 172.21.15.2:0/1114794742 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdf000ca790 con 0x7fdf0015afe0 2024-09-19T05:45:25.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.323+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf0014d3a0 con 0x7fdee406c2b0 2024-09-19T05:45:25.332 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.330+0000 7fdef67fc640 1 -- 172.21.15.2:0/1114794742 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253668 (secure 0 0 0) 0x7fdefc001420 con 0x7fdee406c2b0 2024-09-19T05:45:25.336 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:25.338 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:45:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdee406c2b0 msgr2=0x7fdee406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdee406c2b0 0x7fdee406e770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdefc001780 tx=0x7fdefc009040 comp rx=0 tx=0).stop 2024-09-19T05:45:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf0015afe0 msgr2=0x7fdf000c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.340 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf0015afe0 0x7fdf000c84e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdef8007d80 tx=0x7fdef800e4b0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 shutdown_connections 2024-09-19T05:45:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdee406c2b0 0x7fdee406e770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf0015afe0 0x7fdf000c84e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdf00154e70 0x7fdf0014b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 --2- 172.21.15.2:0/1114794742 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf000cce60 0x7fdf0014ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 >> 172.21.15.2:0/1114794742 conn(0x7fdf0006da40 msgr2=0x7fdf00153270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:25.341 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 shutdown_connections 2024-09-19T05:45:25.342 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.333+0000 7fdf07f59640 1 -- 172.21.15.2:0/1114794742 wait complete. 2024-09-19T05:45:25.347 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-19T05:45:23.854635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560741,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13281,"num_write_kb":25179740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14765,"ondisk_log_size":14765,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342248,"kb_used_data":21916,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632472,"statfs":{"total":773094113280,"available":772743651328,"internally_reserved":0,"allocated":22441984,"data_stored":21573735,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.189314"},"pg_stats":[{"pgid":"3.3f","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687024+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:45:21.687024+0000","last_peered":"2024-09-19T05:45:21.687024+0000","last_clean":"2024-09-19T05:45:21.687024+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:21.687024+0000","last_undegraded":"2024-09-19T05:45:21.687024+0000","last_fullsized":"2024-09-19T05:45:21.687024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'148","reported_seq":301,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688407+0000","last_change":"2024-09-19T05:44:54.646789+0000","last_active":"2024-09-19T05:45:21.688407+0000","last_peered":"2024-09-19T05:45:21.688407+0000","last_clean":"2024-09-19T05:45:21.688407+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:21.688407+0000","last_undegraded":"2024-09-19T05:45:21.688407+0000","last_fullsized":"2024-09-19T05:45:21.688407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:11.761535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"84'171","reported_seq":325,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688172+0000","last_change":"2024-09-19T05:44:53.475804+0000","last_active":"2024-09-19T05:45:21.688172+0000","last_peered":"2024-09-19T05:45:21.688172+0000","last_clean":"2024-09-19T05:45:21.688172+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:21.688172+0000","last_undegraded":"2024-09-19T05:45:21.688172+0000","last_fullsized":"2024-09-19T05:45:21.688172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:57.009039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"84'185","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935211+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:45:21.935211+0000","last_peered":"2024-09-19T05:45:21.935211+0000","last_clean":"2024-09-19T05:45:21.935211+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:21.935211+0000","last_undegraded":"2024-09-19T05:45:21.935211+0000","last_fullsized":"2024-09-19T05:45:21.935211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"84'170","reported_seq":438,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687949+0000","last_change":"2024-09-19T05:44:53.953498+0000","last_active":"2024-09-19T05:45:21.687949+0000","last_peered":"2024-09-19T05:45:21.687949+0000","last_clean":"2024-09-19T05:45:21.687949+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:21.687949+0000","last_undegraded":"2024-09-19T05:45:21.687949+0000","last_fullsized":"2024-09-19T05:45:21.687949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:39.367204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"84'156","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935150+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:45:21.935150+0000","last_peered":"2024-09-19T05:45:21.935150+0000","last_clean":"2024-09-19T05:45:21.935150+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:21.935150+0000","last_undegraded":"2024-09-19T05:45:21.935150+0000","last_fullsized":"2024-09-19T05:45:21.935150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"84'183","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686741+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:45:21.686741+0000","last_peered":"2024-09-19T05:45:21.686741+0000","last_clean":"2024-09-19T05:45:21.686741+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:21.686741+0000","last_undegraded":"2024-09-19T05:45:21.686741+0000","last_fullsized":"2024-09-19T05:45:21.686741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"84'195","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687356+0000","last_change":"2024-09-19T05:44:56.567156+0000","last_active":"2024-09-19T05:45:21.687356+0000","last_peered":"2024-09-19T05:45:21.687356+0000","last_clean":"2024-09-19T05:45:21.687356+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:21.687356+0000","last_undegraded":"2024-09-19T05:45:21.687356+0000","last_fullsized":"2024-09-19T05:45:21.687356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:17.811359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"84'167","reported_seq":280,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190149+0000","last_change":"2024-09-19T05:45:10.627715+0000","last_active":"2024-09-19T05:45:22.190149+0000","last_peered":"2024-09-19T05:45:22.190149+0000","last_clean":"2024-09-19T05:45:22.190149+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:22.190149+0000","last_undegraded":"2024-09-19T05:45:22.190149+0000","last_fullsized":"2024-09-19T05:45:22.190149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:38.174880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686962+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:45:21.686962+0000","last_peered":"2024-09-19T05:45:21.686962+0000","last_clean":"2024-09-19T05:45:21.686962+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:21.686962+0000","last_undegraded":"2024-09-19T05:45:21.686962+0000","last_fullsized":"2024-09-19T05:45:21.686962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"84'208","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686760+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:45:21.686760+0000","last_peered":"2024-09-19T05:45:21.686760+0000","last_clean":"2024-09-19T05:45:21.686760+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:21.686760+0000","last_undegraded":"2024-09-19T05:45:21.686760+0000","last_fullsized":"2024-09-19T05:45:21.686760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"84'159","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688440+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:45:21.688440+0000","last_peered":"2024-09-19T05:45:21.688440+0000","last_clean":"2024-09-19T05:45:21.688440+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:21.688440+0000","last_undegraded":"2024-09-19T05:45:21.688440+0000","last_fullsized":"2024-09-19T05:45:21.688440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:54.111389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'167","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773293+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:45:21.773293+0000","last_peered":"2024-09-19T05:45:21.773293+0000","last_clean":"2024-09-19T05:45:21.773293+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:21.773293+0000","last_undegraded":"2024-09-19T05:45:21.773293+0000","last_fullsized":"2024-09-19T05:45:21.773293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773059+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:45:21.773059+0000","last_peered":"2024-09-19T05:45:21.773059+0000","last_clean":"2024-09-19T05:45:21.773059+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:21.773059+0000","last_undegraded":"2024-09-19T05:45:21.773059+0000","last_fullsized":"2024-09-19T05:45:21.773059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"84'170","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687615+0000","last_change":"2024-09-19T05:44:53.930699+0000","last_active":"2024-09-19T05:45:21.687615+0000","last_peered":"2024-09-19T05:45:21.687615+0000","last_clean":"2024-09-19T05:45:21.687615+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:21.687615+0000","last_undegraded":"2024-09-19T05:45:21.687615+0000","last_fullsized":"2024-09-19T05:45:21.687615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:02.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"84'153","reported_seq":324,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716095+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:45:21.716095+0000","last_peered":"2024-09-19T05:45:21.716095+0000","last_clean":"2024-09-19T05:45:21.716095+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:21.716095+0000","last_undegraded":"2024-09-19T05:45:21.716095+0000","last_fullsized":"2024-09-19T05:45:21.716095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"84'168","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715985+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:45:21.715985+0000","last_peered":"2024-09-19T05:45:21.715985+0000","last_clean":"2024-09-19T05:45:21.715985+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:21.715985+0000","last_undegraded":"2024-09-19T05:45:21.715985+0000","last_fullsized":"2024-09-19T05:45:21.715985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"84'169","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687671+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:45:21.687671+0000","last_peered":"2024-09-19T05:45:21.687671+0000","last_clean":"2024-09-19T05:45:21.687671+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:21.687671+0000","last_undegraded":"2024-09-19T05:45:21.687671+0000","last_fullsized":"2024-09-19T05:45:21.687671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"84'198","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716789+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:45:21.716789+0000","last_peered":"2024-09-19T05:45:21.716789+0000","last_clean":"2024-09-19T05:45:21.716789+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:21.716789+0000","last_undegraded":"2024-09-19T05:45:21.716789+0000","last_fullsized":"2024-09-19T05:45:21.716789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"84'171","reported_seq":353,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688653+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:45:21.688653+0000","last_peered":"2024-09-19T05:45:21.688653+0000","last_clean":"2024-09-19T05:45:21.688653+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:21.688653+0000","last_undegraded":"2024-09-19T05:45:21.688653+0000","last_fullsized":"2024-09-19T05:45:21.688653+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:17.766409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"84'143","reported_seq":315,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708984+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:45:21.708984+0000","last_peered":"2024-09-19T05:45:21.708984+0000","last_clean":"2024-09-19T05:45:21.708984+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:21.708984+0000","last_undegraded":"2024-09-19T05:45:21.708984+0000","last_fullsized":"2024-09-19T05:45:21.708984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"84'189","reported_seq":246,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687810+0000","last_change":"2024-09-19T05:44:55.559370+0000","last_active":"2024-09-19T05:45:21.687810+0000","last_peered":"2024-09-19T05:45:21.687810+0000","last_clean":"2024-09-19T05:45:21.687810+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:21.687810+0000","last_undegraded":"2024-09-19T05:45:21.687810+0000","last_fullsized":"2024-09-19T05:45:21.687810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:25.838393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'142","reported_seq":295,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687168+0000","last_change":"2024-09-19T05:44:56.134769+0000","last_active":"2024-09-19T05:45:21.687168+0000","last_peered":"2024-09-19T05:45:21.687168+0000","last_clean":"2024-09-19T05:45:21.687168+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:21.687168+0000","last_undegraded":"2024-09-19T05:45:21.687168+0000","last_fullsized":"2024-09-19T05:45:21.687168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:06.255089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"84'201","reported_seq":492,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716107+0000","last_change":"2024-09-19T05:44:56.565770+0000","last_active":"2024-09-19T05:45:21.716107+0000","last_peered":"2024-09-19T05:45:21.716107+0000","last_clean":"2024-09-19T05:45:21.716107+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:21.716107+0000","last_undegraded":"2024-09-19T05:45:21.716107+0000","last_fullsized":"2024-09-19T05:45:21.716107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:03.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"84'158","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688447+0000","last_change":"2024-09-19T05:44:56.582314+0000","last_active":"2024-09-19T05:45:21.688447+0000","last_peered":"2024-09-19T05:45:21.688447+0000","last_clean":"2024-09-19T05:45:21.688447+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:21.688447+0000","last_undegraded":"2024-09-19T05:45:21.688447+0000","last_fullsized":"2024-09-19T05:45:21.688447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:41.223871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"84'202","reported_seq":374,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935387+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:45:21.935387+0000","last_peered":"2024-09-19T05:45:21.935387+0000","last_clean":"2024-09-19T05:45:21.935387+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:21.935387+0000","last_undegraded":"2024-09-19T05:45:21.935387+0000","last_fullsized":"2024-09-19T05:45:21.935387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773382+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:45:21.773382+0000","last_peered":"2024-09-19T05:45:21.773382+0000","last_clean":"2024-09-19T05:45:21.773382+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:21.773382+0000","last_undegraded":"2024-09-19T05:45:21.773382+0000","last_fullsized":"2024-09-19T05:45:21.773382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"84'148","reported_seq":320,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773260+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:45:21.773260+0000","last_peered":"2024-09-19T05:45:21.773260+0000","last_clean":"2024-09-19T05:45:21.773260+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:21.773260+0000","last_undegraded":"2024-09-19T05:45:21.773260+0000","last_fullsized":"2024-09-19T05:45:21.773260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'127","reported_seq":281,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688046+0000","last_change":"2024-09-19T05:44:54.977580+0000","last_active":"2024-09-19T05:45:21.688046+0000","last_peered":"2024-09-19T05:45:21.688046+0000","last_clean":"2024-09-19T05:45:21.688046+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:21.688046+0000","last_undegraded":"2024-09-19T05:45:21.688046+0000","last_fullsized":"2024-09-19T05:45:21.688046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:12.165530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"84'145","reported_seq":408,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716323+0000","last_change":"2024-09-19T05:44:53.510921+0000","last_active":"2024-09-19T05:45:21.716323+0000","last_peered":"2024-09-19T05:45:21.716323+0000","last_clean":"2024-09-19T05:45:21.716323+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:21.716323+0000","last_undegraded":"2024-09-19T05:45:21.716323+0000","last_fullsized":"2024-09-19T05:45:21.716323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:39.530204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"84'146","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687436+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:45:21.687436+0000","last_peered":"2024-09-19T05:45:21.687436+0000","last_clean":"2024-09-19T05:45:21.687436+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:21.687436+0000","last_undegraded":"2024-09-19T05:45:21.687436+0000","last_fullsized":"2024-09-19T05:45:21.687436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'171","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935092+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:45:21.935092+0000","last_peered":"2024-09-19T05:45:21.935092+0000","last_clean":"2024-09-19T05:45:21.935092+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:21.935092+0000","last_undegraded":"2024-09-19T05:45:21.935092+0000","last_fullsized":"2024-09-19T05:45:21.935092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"84'142","reported_seq":240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189849+0000","last_change":"2024-09-19T05:45:10.624968+0000","last_active":"2024-09-19T05:45:22.189849+0000","last_peered":"2024-09-19T05:45:22.189849+0000","last_clean":"2024-09-19T05:45:22.189849+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:22.189849+0000","last_undegraded":"2024-09-19T05:45:22.189849+0000","last_fullsized":"2024-09-19T05:45:22.189849+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:59.539763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"94'1809","reported_seq":1885,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687685+0000","last_change":"2024-09-19T05:44:55.560257+0000","last_active":"2024-09-19T05:45:21.687685+0000","last_peered":"2024-09-19T05:45:21.687685+0000","last_clean":"2024-09-19T05:45:21.687685+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:21.687685+0000","last_undegraded":"2024-09-19T05:45:21.687685+0000","last_fullsized":"2024-09-19T05:45:21.687685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1809,"log_dups_size":0,"ondisk_log_size":1809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:25.647547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'154","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708831+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:45:21.708831+0000","last_peered":"2024-09-19T05:45:21.708831+0000","last_clean":"2024-09-19T05:45:21.708831+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:21.708831+0000","last_undegraded":"2024-09-19T05:45:21.708831+0000","last_fullsized":"2024-09-19T05:45:21.708831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716460+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:45:21.716460+0000","last_peered":"2024-09-19T05:45:21.716460+0000","last_clean":"2024-09-19T05:45:21.716460+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:21.716460+0000","last_undegraded":"2024-09-19T05:45:21.716460+0000","last_fullsized":"2024-09-19T05:45:21.716460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773054+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:45:21.773054+0000","last_peered":"2024-09-19T05:45:21.773054+0000","last_clean":"2024-09-19T05:45:21.773054+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:21.773054+0000","last_undegraded":"2024-09-19T05:45:21.773054+0000","last_fullsized":"2024-09-19T05:45:21.773054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'182","reported_seq":337,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687351+0000","last_change":"2024-09-19T05:44:59.989289+0000","last_active":"2024-09-19T05:45:21.687351+0000","last_peered":"2024-09-19T05:45:21.687351+0000","last_clean":"2024-09-19T05:45:21.687351+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:21.687351+0000","last_undegraded":"2024-09-19T05:45:21.687351+0000","last_fullsized":"2024-09-19T05:45:21.687351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:52.052457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716243+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:45:21.716243+0000","last_peered":"2024-09-19T05:45:21.716243+0000","last_clean":"2024-09-19T05:45:21.716243+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:21.716243+0000","last_undegraded":"2024-09-19T05:45:21.716243+0000","last_fullsized":"2024-09-19T05:45:21.716243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"84'177","reported_seq":359,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708983+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:45:21.708983+0000","last_peered":"2024-09-19T05:45:21.708983+0000","last_clean":"2024-09-19T05:45:21.708983+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:21.708983+0000","last_undegraded":"2024-09-19T05:45:21.708983+0000","last_fullsized":"2024-09-19T05:45:21.708983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190091+0000","last_change":"2024-09-19T05:45:10.622931+0000","last_active":"2024-09-19T05:45:22.190091+0000","last_peered":"2024-09-19T05:45:22.190091+0000","last_clean":"2024-09-19T05:45:22.190091+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:22.190091+0000","last_undegraded":"2024-09-19T05:45:22.190091+0000","last_fullsized":"2024-09-19T05:45:22.190091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:34:06.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"84'241","reported_seq":413,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935402+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:45:21.935402+0000","last_peered":"2024-09-19T05:45:21.935402+0000","last_clean":"2024-09-19T05:45:21.935402+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:21.935402+0000","last_undegraded":"2024-09-19T05:45:21.935402+0000","last_fullsized":"2024-09-19T05:45:21.935402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688495+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:45:21.688495+0000","last_peered":"2024-09-19T05:45:21.688495+0000","last_clean":"2024-09-19T05:45:21.688495+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:21.688495+0000","last_undegraded":"2024-09-19T05:45:21.688495+0000","last_fullsized":"2024-09-19T05:45:21.688495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:39.063921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"84'151","reported_seq":300,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688324+0000","last_change":"2024-09-19T05:44:59.517543+0000","last_active":"2024-09-19T05:45:21.688324+0000","last_peered":"2024-09-19T05:45:21.688324+0000","last_clean":"2024-09-19T05:45:21.688324+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:21.688324+0000","last_undegraded":"2024-09-19T05:45:21.688324+0000","last_fullsized":"2024-09-19T05:45:21.688324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.453343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"89'10","reported_seq":185,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687787+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:45:21.687787+0000","last_peered":"2024-09-19T05:45:21.687787+0000","last_clean":"2024-09-19T05:45:21.687787+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:21.687787+0000","last_undegraded":"2024-09-19T05:45:21.687787+0000","last_fullsized":"2024-09-19T05:45:21.687787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"84'157","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716300+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:45:21.716300+0000","last_peered":"2024-09-19T05:45:21.716300+0000","last_clean":"2024-09-19T05:45:21.716300+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:21.716300+0000","last_undegraded":"2024-09-19T05:45:21.716300+0000","last_fullsized":"2024-09-19T05:45:21.716300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"94'2017","reported_seq":2191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935286+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:45:21.935286+0000","last_peered":"2024-09-19T05:45:21.935286+0000","last_clean":"2024-09-19T05:45:21.935286+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:21.935286+0000","last_undegraded":"2024-09-19T05:45:21.935286+0000","last_fullsized":"2024-09-19T05:45:21.935286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688257+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:45:21.688257+0000","last_peered":"2024-09-19T05:45:21.688257+0000","last_clean":"2024-09-19T05:45:21.688257+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:21.688257+0000","last_undegraded":"2024-09-19T05:45:21.688257+0000","last_fullsized":"2024-09-19T05:45:21.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687838+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:45:21.687838+0000","last_peered":"2024-09-19T05:45:21.687838+0000","last_clean":"2024-09-19T05:45:21.687838+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:21.687838+0000","last_undegraded":"2024-09-19T05:45:21.687838+0000","last_fullsized":"2024-09-19T05:45:21.687838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"84'168","reported_seq":340,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773221+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:45:21.773221+0000","last_peered":"2024-09-19T05:45:21.773221+0000","last_clean":"2024-09-19T05:45:21.773221+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:21.773221+0000","last_undegraded":"2024-09-19T05:45:21.773221+0000","last_fullsized":"2024-09-19T05:45:21.773221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716531+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:45:21.716531+0000","last_peered":"2024-09-19T05:45:21.716531+0000","last_clean":"2024-09-19T05:45:21.716531+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:21.716531+0000","last_undegraded":"2024-09-19T05:45:21.716531+0000","last_fullsized":"2024-09-19T05:45:21.716531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"84'186","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715983+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:45:21.715983+0000","last_peered":"2024-09-19T05:45:21.715983+0000","last_clean":"2024-09-19T05:45:21.715983+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:21.715983+0000","last_undegraded":"2024-09-19T05:45:21.715983+0000","last_fullsized":"2024-09-19T05:45:21.715983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687854+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:45:21.687854+0000","last_peered":"2024-09-19T05:45:21.687854+0000","last_clean":"2024-09-19T05:45:21.687854+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:21.687854+0000","last_undegraded":"2024-09-19T05:45:21.687854+0000","last_fullsized":"2024-09-19T05:45:21.687854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"84'191","reported_seq":283,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189767+0000","last_change":"2024-09-19T05:45:10.627674+0000","last_active":"2024-09-19T05:45:22.189767+0000","last_peered":"2024-09-19T05:45:22.189767+0000","last_clean":"2024-09-19T05:45:22.189767+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:22.189767+0000","last_undegraded":"2024-09-19T05:45:22.189767+0000","last_fullsized":"2024-09-19T05:45:22.189767+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:21.685895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"89'9","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708963+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:45:21.708963+0000","last_peered":"2024-09-19T05:45:21.708963+0000","last_clean":"2024-09-19T05:45:21.708963+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:21.708963+0000","last_undegraded":"2024-09-19T05:45:21.708963+0000","last_fullsized":"2024-09-19T05:45:21.708963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"84'209","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716252+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:45:21.716252+0000","last_peered":"2024-09-19T05:45:21.716252+0000","last_clean":"2024-09-19T05:45:21.716252+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:21.716252+0000","last_undegraded":"2024-09-19T05:45:21.716252+0000","last_fullsized":"2024-09-19T05:45:21.716252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688782+0000","last_change":"2024-09-19T05:44:53.065559+0000","last_active":"2024-09-19T05:45:21.688782+0000","last_peered":"2024-09-19T05:45:21.688782+0000","last_clean":"2024-09-19T05:45:21.688782+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:21.688782+0000","last_undegraded":"2024-09-19T05:45:21.688782+0000","last_fullsized":"2024-09-19T05:45:21.688782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:13.889455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935000+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:45:21.935000+0000","last_peered":"2024-09-19T05:45:21.935000+0000","last_clean":"2024-09-19T05:45:21.935000+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:21.935000+0000","last_undegraded":"2024-09-19T05:45:21.935000+0000","last_fullsized":"2024-09-19T05:45:21.935000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709231+0000","last_change":"2024-09-19T05:44:53.059769+0000","last_active":"2024-09-19T05:45:21.709231+0000","last_peered":"2024-09-19T05:45:21.709231+0000","last_clean":"2024-09-19T05:45:21.709231+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:21.709231+0000","last_undegraded":"2024-09-19T05:45:21.709231+0000","last_fullsized":"2024-09-19T05:45:21.709231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:03.909805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"84'153","reported_seq":231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687781+0000","last_change":"2024-09-19T05:44:15.787415+0000","last_active":"2024-09-19T05:45:21.687781+0000","last_peered":"2024-09-19T05:45:21.687781+0000","last_clean":"2024-09-19T05:45:21.687781+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:21.687781+0000","last_undegraded":"2024-09-19T05:45:21.687781+0000","last_fullsized":"2024-09-19T05:45:21.687781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:10.095369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":125,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688186+0000","last_change":"2024-09-19T05:44:53.058979+0000","last_active":"2024-09-19T05:45:21.688186+0000","last_peered":"2024-09-19T05:45:21.688186+0000","last_clean":"2024-09-19T05:45:21.688186+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:21.688186+0000","last_undegraded":"2024-09-19T05:45:21.688186+0000","last_fullsized":"2024-09-19T05:45:21.688186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:04.530725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686740+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:45:21.686740+0000","last_peered":"2024-09-19T05:45:21.686740+0000","last_clean":"2024-09-19T05:45:21.686740+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:21.686740+0000","last_undegraded":"2024-09-19T05:45:21.686740+0000","last_fullsized":"2024-09-19T05:45:21.686740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773397+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:45:21.773397+0000","last_peered":"2024-09-19T05:45:21.773397+0000","last_clean":"2024-09-19T05:45:21.773397+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:21.773397+0000","last_undegraded":"2024-09-19T05:45:21.773397+0000","last_fullsized":"2024-09-19T05:45:21.773397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"84'168","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716718+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:45:21.716718+0000","last_peered":"2024-09-19T05:45:21.716718+0000","last_clean":"2024-09-19T05:45:21.716718+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:21.716718+0000","last_undegraded":"2024-09-19T05:45:21.716718+0000","last_fullsized":"2024-09-19T05:45:21.716718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":260,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189834+0000","last_change":"2024-09-19T05:45:10.627237+0000","last_active":"2024-09-19T05:45:22.189834+0000","last_peered":"2024-09-19T05:45:22.189834+0000","last_clean":"2024-09-19T05:45:22.189834+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:22.189834+0000","last_undegraded":"2024-09-19T05:45:22.189834+0000","last_fullsized":"2024-09-19T05:45:22.189834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:04.886546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716440+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:45:21.716440+0000","last_peered":"2024-09-19T05:45:21.716440+0000","last_clean":"2024-09-19T05:45:21.716440+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:21.716440+0000","last_undegraded":"2024-09-19T05:45:21.716440+0000","last_fullsized":"2024-09-19T05:45:21.716440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708974+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:45:21.708974+0000","last_peered":"2024-09-19T05:45:21.708974+0000","last_clean":"2024-09-19T05:45:21.708974+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:21.708974+0000","last_undegraded":"2024-09-19T05:45:21.708974+0000","last_fullsized":"2024-09-19T05:45:21.708974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716388+0000","last_change":"2024-09-19T05:44:53.069283+0000","last_active":"2024-09-19T05:45:21.716388+0000","last_peered":"2024-09-19T05:45:21.716388+0000","last_clean":"2024-09-19T05:45:21.716388+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:21.716388+0000","last_undegraded":"2024-09-19T05:45:21.716388+0000","last_fullsized":"2024-09-19T05:45:21.716388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:51.209720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773046+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:45:21.773046+0000","last_peered":"2024-09-19T05:45:21.773046+0000","last_clean":"2024-09-19T05:45:21.773046+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:21.773046+0000","last_undegraded":"2024-09-19T05:45:21.773046+0000","last_fullsized":"2024-09-19T05:45:21.773046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687539+0000","last_change":"2024-09-19T05:44:53.068415+0000","last_active":"2024-09-19T05:45:21.687539+0000","last_peered":"2024-09-19T05:45:21.687539+0000","last_clean":"2024-09-19T05:45:21.687539+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:21.687539+0000","last_undegraded":"2024-09-19T05:45:21.687539+0000","last_fullsized":"2024-09-19T05:45:21.687539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:16.609871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"84'157","reported_seq":310,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687083+0000","last_change":"2024-09-19T05:44:58.820818+0000","last_active":"2024-09-19T05:45:21.687083+0000","last_peered":"2024-09-19T05:45:21.687083+0000","last_clean":"2024-09-19T05:45:21.687083+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:21.687083+0000","last_undegraded":"2024-09-19T05:45:21.687083+0000","last_fullsized":"2024-09-19T05:45:21.687083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:32.427214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716567+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:45:21.716567+0000","last_peered":"2024-09-19T05:45:21.716567+0000","last_clean":"2024-09-19T05:45:21.716567+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:21.716567+0000","last_undegraded":"2024-09-19T05:45:21.716567+0000","last_fullsized":"2024-09-19T05:45:21.716567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686759+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:45:21.686759+0000","last_peered":"2024-09-19T05:45:21.686759+0000","last_clean":"2024-09-19T05:45:21.686759+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:21.686759+0000","last_undegraded":"2024-09-19T05:45:21.686759+0000","last_fullsized":"2024-09-19T05:45:21.686759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687970+0000","last_change":"2024-09-19T05:44:53.063483+0000","last_active":"2024-09-19T05:45:21.687970+0000","last_peered":"2024-09-19T05:45:21.687970+0000","last_clean":"2024-09-19T05:45:21.687970+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:21.687970+0000","last_undegraded":"2024-09-19T05:45:21.687970+0000","last_fullsized":"2024-09-19T05:45:21.687970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:40.306784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"84'155","reported_seq":327,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935176+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:45:21.935176+0000","last_peered":"2024-09-19T05:45:21.935176+0000","last_clean":"2024-09-19T05:45:21.935176+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:21.935176+0000","last_undegraded":"2024-09-19T05:45:21.935176+0000","last_fullsized":"2024-09-19T05:45:21.935176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716905+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:45:21.716905+0000","last_peered":"2024-09-19T05:45:21.716905+0000","last_clean":"2024-09-19T05:45:21.716905+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:21.716905+0000","last_undegraded":"2024-09-19T05:45:21.716905+0000","last_fullsized":"2024-09-19T05:45:21.716905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'153","reported_seq":424,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773336+0000","last_change":"2024-09-19T05:44:55.675882+0000","last_active":"2024-09-19T05:45:21.773336+0000","last_peered":"2024-09-19T05:45:21.773336+0000","last_clean":"2024-09-19T05:45:21.773336+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:21.773336+0000","last_undegraded":"2024-09-19T05:45:21.773336+0000","last_fullsized":"2024-09-19T05:45:21.773336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:04.593886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709321+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:45:21.709321+0000","last_peered":"2024-09-19T05:45:21.709321+0000","last_clean":"2024-09-19T05:45:21.709321+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:21.709321+0000","last_undegraded":"2024-09-19T05:45:21.709321+0000","last_fullsized":"2024-09-19T05:45:21.709321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"84'161","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709277+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:45:21.709277+0000","last_peered":"2024-09-19T05:45:21.709277+0000","last_clean":"2024-09-19T05:45:21.709277+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:21.709277+0000","last_undegraded":"2024-09-19T05:45:21.709277+0000","last_fullsized":"2024-09-19T05:45:21.709277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"85'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716921+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:45:21.716921+0000","last_peered":"2024-09-19T05:45:21.716921+0000","last_clean":"2024-09-19T05:45:21.716921+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:21.716921+0000","last_undegraded":"2024-09-19T05:45:21.716921+0000","last_fullsized":"2024-09-19T05:45:21.716921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"84'176","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709220+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:45:21.709220+0000","last_peered":"2024-09-19T05:45:21.709220+0000","last_clean":"2024-09-19T05:45:21.709220+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:21.709220+0000","last_undegraded":"2024-09-19T05:45:21.709220+0000","last_fullsized":"2024-09-19T05:45:21.709220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687243+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:45:21.687243+0000","last_peered":"2024-09-19T05:45:21.687243+0000","last_clean":"2024-09-19T05:45:21.687243+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:21.687243+0000","last_undegraded":"2024-09-19T05:45:21.687243+0000","last_fullsized":"2024-09-19T05:45:21.687243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"84'160","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716327+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:45:21.716327+0000","last_peered":"2024-09-19T05:45:21.716327+0000","last_clean":"2024-09-19T05:45:21.716327+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:21.716327+0000","last_undegraded":"2024-09-19T05:45:21.716327+0000","last_fullsized":"2024-09-19T05:45:21.716327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709021+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:45:21.709021+0000","last_peered":"2024-09-19T05:45:21.709021+0000","last_clean":"2024-09-19T05:45:21.709021+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:21.709021+0000","last_undegraded":"2024-09-19T05:45:21.709021+0000","last_fullsized":"2024-09-19T05:45:21.709021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708902+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:45:21.708902+0000","last_peered":"2024-09-19T05:45:21.708902+0000","last_clean":"2024-09-19T05:45:21.708902+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:21.708902+0000","last_undegraded":"2024-09-19T05:45:21.708902+0000","last_fullsized":"2024-09-19T05:45:21.708902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773238+0000","last_change":"2024-09-19T05:44:53.065594+0000","last_active":"2024-09-19T05:45:21.773238+0000","last_peered":"2024-09-19T05:45:21.773238+0000","last_clean":"2024-09-19T05:45:21.773238+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:21.773238+0000","last_undegraded":"2024-09-19T05:45:21.773238+0000","last_fullsized":"2024-09-19T05:45:21.773238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:47.254673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"84'156","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687718+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:45:21.687718+0000","last_peered":"2024-09-19T05:45:21.687718+0000","last_clean":"2024-09-19T05:45:21.687718+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:21.687718+0000","last_undegraded":"2024-09-19T05:45:21.687718+0000","last_fullsized":"2024-09-19T05:45:21.687718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"59'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716236+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:45:21.716236+0000","last_peered":"2024-09-19T05:45:21.716236+0000","last_clean":"2024-09-19T05:45:21.716236+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:21.716236+0000","last_undegraded":"2024-09-19T05:45:21.716236+0000","last_fullsized":"2024-09-19T05:45:21.716236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"84'149","reported_seq":330,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716425+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:45:21.716425+0000","last_peered":"2024-09-19T05:45:21.716425+0000","last_clean":"2024-09-19T05:45:21.716425+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:21.716425+0000","last_undegraded":"2024-09-19T05:45:21.716425+0000","last_fullsized":"2024-09-19T05:45:21.716425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"84'190","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715986+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:45:21.715986+0000","last_peered":"2024-09-19T05:45:21.715986+0000","last_clean":"2024-09-19T05:45:21.715986+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:21.715986+0000","last_undegraded":"2024-09-19T05:45:21.715986+0000","last_fullsized":"2024-09-19T05:45:21.715986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":194,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688923+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:45:21.688923+0000","last_peered":"2024-09-19T05:45:21.688923+0000","last_clean":"2024-09-19T05:45:21.688923+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:21.688923+0000","last_undegraded":"2024-09-19T05:45:21.688923+0000","last_fullsized":"2024-09-19T05:45:21.688923+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:59.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"84'153","reported_seq":266,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189956+0000","last_change":"2024-09-19T05:45:10.627058+0000","last_active":"2024-09-19T05:45:22.189956+0000","last_peered":"2024-09-19T05:45:22.189956+0000","last_clean":"2024-09-19T05:45:22.189956+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:22.189956+0000","last_undegraded":"2024-09-19T05:45:22.189956+0000","last_fullsized":"2024-09-19T05:45:22.189956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:52.373727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"70'4","reported_seq":196,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688300+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:45:21.688300+0000","last_peered":"2024-09-19T05:45:21.688300+0000","last_clean":"2024-09-19T05:45:21.688300+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:21.688300+0000","last_undegraded":"2024-09-19T05:45:21.688300+0000","last_fullsized":"2024-09-19T05:45:21.688300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:52.607339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"84'154","reported_seq":314,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688325+0000","last_change":"2024-09-19T05:44:57.957776+0000","last_active":"2024-09-19T05:45:21.688325+0000","last_peered":"2024-09-19T05:45:21.688325+0000","last_clean":"2024-09-19T05:45:21.688325+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:21.688325+0000","last_undegraded":"2024-09-19T05:45:21.688325+0000","last_fullsized":"2024-09-19T05:45:21.688325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:37.587819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":156,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688211+0000","last_change":"2024-09-19T05:44:53.061701+0000","last_active":"2024-09-19T05:45:21.688211+0000","last_peered":"2024-09-19T05:45:21.688211+0000","last_clean":"2024-09-19T05:45:21.688211+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:21.688211+0000","last_undegraded":"2024-09-19T05:45:21.688211+0000","last_fullsized":"2024-09-19T05:45:21.688211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:26.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"84'176","reported_seq":321,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688312+0000","last_change":"2024-09-19T05:44:57.503018+0000","last_active":"2024-09-19T05:45:21.688312+0000","last_peered":"2024-09-19T05:45:21.688312+0000","last_clean":"2024-09-19T05:45:21.688312+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:21.688312+0000","last_undegraded":"2024-09-19T05:45:21.688312+0000","last_fullsized":"2024-09-19T05:45:21.688312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:44.482643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709387+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:45:21.709387+0000","last_peered":"2024-09-19T05:45:21.709387+0000","last_clean":"2024-09-19T05:45:21.709387+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:21.709387+0000","last_undegraded":"2024-09-19T05:45:21.709387+0000","last_fullsized":"2024-09-19T05:45:21.709387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970373,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3906,"num_write_kb":11500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15940746,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3893,"ondisk_log_size":3893,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":5272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":5398528,"data_stored":4749255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":4984832,"data_stored":4334380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449096,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46800,"kb_used_data":1724,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325040,"statfs":{"total":96636764160,"available":96588840960,"internally_reserved":0,"allocated":1765376,"data_stored":4919338,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50364,"kb_used_data":5280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321476,"statfs":{"total":96636764160,"available":96585191424,"internally_reserved":0,"allocated":5406720,"data_stored":4749927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542183,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40680,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331160,"statfs":{"total":96636764160,"available":96595107840,"internally_reserved":0,"allocated":1200128,"data_stored":554861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40732,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331108,"statfs":{"total":96636764160,"available":96595054592,"internally_reserved":0,"allocated":1249280,"data_stored":1144717,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40700,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331140,"statfs":{"total":96636764160,"available":96595087360,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774691,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:45:25.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:45:25.489 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.487+0000 7f1d85318640 1 Processor -- start 2024-09-19T05:45:25.490 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.487+0000 7f1d85318640 1 -- start start 2024-09-19T05:45:25.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d85318640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 0x7f1d800cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.493 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d85318640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d800cb400 con 0x7f1d8015eb50 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d85318640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d800cb6e0 con 0x7f1d800d06e0 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d85318640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1d800cc970 con 0x7f1d800d0ac0 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d7f7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f1d8015eb50 0x7f1d800cd450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54146/0 (socket says 172.21.15.2:54146) 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d7f7fe640 1 -- 172.21.15.2:0/3487672331 learned_addr learned my addr 172.21.15.2:0/3487672331 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d7e7fc640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 0x7f1d800cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.488+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894135698 0 0) 0x7f1d800cb400 con 0x7f1d8015eb50 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d68003520 con 0x7f1d8015eb50 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2536786172 0 0) 0x7f1d800cb6e0 con 0x7f1d800d06e0 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d800cb400 con 0x7f1d800d06e0 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2891864447 0 0) 0x7f1d68003520 con 0x7f1d8015eb50 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 msgr2=0x7f1d800cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 0x7f1d800cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.494 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 >> v1:172.21.15.123:6789/0 conn(0x7f1d800d06e0 legacy=0x7f1d80154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d800ccc50 con 0x7f1d8015eb50 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f1d70001970 con 0x7f1d8015eb50 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 >> v1:172.21.15.2:6789/0 conn(0x7f1d8015eb50 legacy=0x7f1d800cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d68003be0 0x7f1d68003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 0x7f1d68005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d800cc970 con 0x7f1d68004800 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6800baf0 con 0x7f1d68003be0 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6800bcf0 con 0x7f1d800d0ac0 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7e7fc640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7effd640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d68003be0 0x7f1d68003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7f7fe640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 0x7f1d68005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7e7fc640 1 -- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 msgr2=0x7f1d68005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7e7fc640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d800d0ac0 0x7f1d68005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.495 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7e7fc640 1 -- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d68003be0 msgr2=0x7f1d68003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7e7fc640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d68003be0 0x7f1d68003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.489+0000 7f1d7e7fc640 1 -- 172.21.15.2:0/3487672331 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d68003520 con 0x7f1d68004800 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.490+0000 7f1d7e7fc640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d68004c80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009ac0 tx=0x7f1d68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746e8e263ab48a4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.490+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74036070 con 0x7f1d68004800 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.490+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d74031400 con 0x7f1d68004800 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.490+0000 7f1d7dffb640 1 -- 172.21.15.2:0/3487672331 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74009690 con 0x7f1d68004800 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.491+0000 7f1d85318640 1 -- 172.21.15.2:0/3487672331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 msgr2=0x7f1d68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.491+0000 7f1d85318640 1 --2- 172.21.15.2:0/3487672331 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d68004c80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009ac0 tx=0x7f1d68005830 comp rx=0 tx=0).stop 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.491+0000 7f1d7f7fe640 1 -- 172.21.15.2:0/3487672331 reap_dead start 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.491+0000 7f1d85318640 1 -- 172.21.15.2:0/3487672331 shutdown_connections 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.491+0000 7f1d85318640 1 -- 172.21.15.2:0/3487672331 >> 172.21.15.2:0/3487672331 conn(0x7f1d8006d740 msgr2=0x7f1d80073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.491+0000 7f1d85318640 1 -- 172.21.15.2:0/3487672331 shutdown_connections 2024-09-19T05:45:25.496 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.491+0000 7f1d85318640 1 -- 172.21.15.2:0/3487672331 wait complete. 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.492+0000 7f1d85318640 1 Processor -- start 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.492+0000 7f1d85318640 1 -- start start 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d85318640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d801f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d85318640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d800c90d0 0x7f1d8014b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d85318640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d8015b070 0x7f1d800b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d85318640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f1d800c5830 con 0x7f1d68004800 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d85318640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8015f000 con 0x7f1d800c90d0 2024-09-19T05:45:25.499 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d85318640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1d80155200 con 0x7f1d8015b070 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d801f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d801f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54794/0 (socket says 172.21.15.2:54794) 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 -- 172.21.15.2:0/186230476 learned_addr learned my addr 172.21.15.2:0/186230476 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 -- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d8015b070 msgr2=0x7f1d800b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d8015b070 0x7f1d800b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 -- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d800c90d0 msgr2=0x7f1d8014b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d800c90d0 0x7f1d8014b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 -- 172.21.15.2:0/186230476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8015b910 con 0x7f1d68004800 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d7effd640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d801f3d50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800f770 tx=0x7f1d7800fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.493+0000 7f1d677fe640 1 -- 172.21.15.2:0/186230476 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7801a070 con 0x7f1d68004800 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.494+0000 7f1d677fe640 1 -- 172.21.15.2:0/186230476 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d78005110 con 0x7f1d68004800 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.494+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8015a9a0 con 0x7f1d68004800 2024-09-19T05:45:25.500 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.494+0000 7f1d677fe640 1 -- 172.21.15.2:0/186230476 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78007050 con 0x7f1d68004800 2024-09-19T05:45:25.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.494+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d801f4290 con 0x7f1d68004800 2024-09-19T05:45:25.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.495+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d800cbb40 con 0x7f1d68004800 2024-09-19T05:45:25.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.495+0000 7f1d677fe640 1 -- 172.21.15.2:0/186230476 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1d7801d070 con 0x7f1d68004800 2024-09-19T05:45:25.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.496+0000 7f1d677fe640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1d6c06c2b0 0x7f1d6c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.496+0000 7f1d677fe640 1 -- 172.21.15.2:0/186230476 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f1d7808ac00 con 0x7f1d68004800 2024-09-19T05:45:25.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.496+0000 7f1d7e7fc640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1d6c06c2b0 0x7f1d6c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.501 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.496+0000 7f1d7e7fc640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1d6c06c2b0 0x7f1d6c06e770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d74033000 tx=0x7f1d7402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.500+0000 7f1d677fe640 1 -- 172.21.15.2:0/186230476 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f1d78091080 con 0x7f1d68004800 2024-09-19T05:45:25.675 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.673+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d8015b6c0 con 0x7f1d6c06c2b0 2024-09-19T05:45:25.681 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.679+0000 7f1d677fe640 1 -- 172.21.15.2:0/186230476 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253668 (secure 0 0 0) 0x7f1d8015b6c0 con 0x7f1d6c06c2b0 2024-09-19T05:45:25.684 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:25.685 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:45:25.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1d6c06c2b0 msgr2=0x7f1d6c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1d6c06c2b0 0x7f1d6c06e770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d74033000 tx=0x7f1d7402e000 comp rx=0 tx=0).stop 2024-09-19T05:45:25.686 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 msgr2=0x7f1d801f3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d801f3d50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800f770 tx=0x7f1d7800fb70 comp rx=0 tx=0).stop 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 shutdown_connections 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f1d6c06c2b0 0x7f1d6c06e770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d800c90d0 0x7f1d8014b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d8015b070 0x7f1d800b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 --2- 172.21.15.2:0/186230476 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d68004800 0x7f1d801f3d50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.682+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 >> 172.21.15.2:0/186230476 conn(0x7f1d8006d740 msgr2=0x7f1d80152580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.683+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 shutdown_connections 2024-09-19T05:45:25.687 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.683+0000 7f1d85318640 1 -- 172.21.15.2:0/186230476 wait complete. 2024-09-19T05:45:25.696 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-19T05:45:23.854635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560741,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13281,"num_write_kb":25179740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14765,"ondisk_log_size":14765,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342248,"kb_used_data":21916,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632472,"statfs":{"total":773094113280,"available":772743651328,"internally_reserved":0,"allocated":22441984,"data_stored":21573735,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.189314"},"pg_stats":[{"pgid":"3.3f","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687024+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:45:21.687024+0000","last_peered":"2024-09-19T05:45:21.687024+0000","last_clean":"2024-09-19T05:45:21.687024+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:21.687024+0000","last_undegraded":"2024-09-19T05:45:21.687024+0000","last_fullsized":"2024-09-19T05:45:21.687024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'148","reported_seq":301,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688407+0000","last_change":"2024-09-19T05:44:54.646789+0000","last_active":"2024-09-19T05:45:21.688407+0000","last_peered":"2024-09-19T05:45:21.688407+0000","last_clean":"2024-09-19T05:45:21.688407+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:21.688407+0000","last_undegraded":"2024-09-19T05:45:21.688407+0000","last_fullsized":"2024-09-19T05:45:21.688407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:11.761535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"84'171","reported_seq":325,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688172+0000","last_change":"2024-09-19T05:44:53.475804+0000","last_active":"2024-09-19T05:45:21.688172+0000","last_peered":"2024-09-19T05:45:21.688172+0000","last_clean":"2024-09-19T05:45:21.688172+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:21.688172+0000","last_undegraded":"2024-09-19T05:45:21.688172+0000","last_fullsized":"2024-09-19T05:45:21.688172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:57.009039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"84'185","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935211+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:45:21.935211+0000","last_peered":"2024-09-19T05:45:21.935211+0000","last_clean":"2024-09-19T05:45:21.935211+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:21.935211+0000","last_undegraded":"2024-09-19T05:45:21.935211+0000","last_fullsized":"2024-09-19T05:45:21.935211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"84'170","reported_seq":438,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687949+0000","last_change":"2024-09-19T05:44:53.953498+0000","last_active":"2024-09-19T05:45:21.687949+0000","last_peered":"2024-09-19T05:45:21.687949+0000","last_clean":"2024-09-19T05:45:21.687949+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:21.687949+0000","last_undegraded":"2024-09-19T05:45:21.687949+0000","last_fullsized":"2024-09-19T05:45:21.687949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:39.367204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"84'156","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935150+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:45:21.935150+0000","last_peered":"2024-09-19T05:45:21.935150+0000","last_clean":"2024-09-19T05:45:21.935150+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:21.935150+0000","last_undegraded":"2024-09-19T05:45:21.935150+0000","last_fullsized":"2024-09-19T05:45:21.935150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"84'183","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686741+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:45:21.686741+0000","last_peered":"2024-09-19T05:45:21.686741+0000","last_clean":"2024-09-19T05:45:21.686741+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:21.686741+0000","last_undegraded":"2024-09-19T05:45:21.686741+0000","last_fullsized":"2024-09-19T05:45:21.686741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"84'195","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687356+0000","last_change":"2024-09-19T05:44:56.567156+0000","last_active":"2024-09-19T05:45:21.687356+0000","last_peered":"2024-09-19T05:45:21.687356+0000","last_clean":"2024-09-19T05:45:21.687356+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:21.687356+0000","last_undegraded":"2024-09-19T05:45:21.687356+0000","last_fullsized":"2024-09-19T05:45:21.687356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:17.811359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"84'167","reported_seq":280,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190149+0000","last_change":"2024-09-19T05:45:10.627715+0000","last_active":"2024-09-19T05:45:22.190149+0000","last_peered":"2024-09-19T05:45:22.190149+0000","last_clean":"2024-09-19T05:45:22.190149+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:22.190149+0000","last_undegraded":"2024-09-19T05:45:22.190149+0000","last_fullsized":"2024-09-19T05:45:22.190149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:38.174880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686962+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:45:21.686962+0000","last_peered":"2024-09-19T05:45:21.686962+0000","last_clean":"2024-09-19T05:45:21.686962+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:21.686962+0000","last_undegraded":"2024-09-19T05:45:21.686962+0000","last_fullsized":"2024-09-19T05:45:21.686962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"84'208","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686760+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:45:21.686760+0000","last_peered":"2024-09-19T05:45:21.686760+0000","last_clean":"2024-09-19T05:45:21.686760+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:21.686760+0000","last_undegraded":"2024-09-19T05:45:21.686760+0000","last_fullsized":"2024-09-19T05:45:21.686760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"84'159","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688440+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:45:21.688440+0000","last_peered":"2024-09-19T05:45:21.688440+0000","last_clean":"2024-09-19T05:45:21.688440+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:21.688440+0000","last_undegraded":"2024-09-19T05:45:21.688440+0000","last_fullsized":"2024-09-19T05:45:21.688440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:54.111389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'167","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773293+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:45:21.773293+0000","last_peered":"2024-09-19T05:45:21.773293+0000","last_clean":"2024-09-19T05:45:21.773293+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:21.773293+0000","last_undegraded":"2024-09-19T05:45:21.773293+0000","last_fullsized":"2024-09-19T05:45:21.773293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773059+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:45:21.773059+0000","last_peered":"2024-09-19T05:45:21.773059+0000","last_clean":"2024-09-19T05:45:21.773059+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:21.773059+0000","last_undegraded":"2024-09-19T05:45:21.773059+0000","last_fullsized":"2024-09-19T05:45:21.773059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"84'170","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687615+0000","last_change":"2024-09-19T05:44:53.930699+0000","last_active":"2024-09-19T05:45:21.687615+0000","last_peered":"2024-09-19T05:45:21.687615+0000","last_clean":"2024-09-19T05:45:21.687615+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:21.687615+0000","last_undegraded":"2024-09-19T05:45:21.687615+0000","last_fullsized":"2024-09-19T05:45:21.687615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:02.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"84'153","reported_seq":324,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716095+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:45:21.716095+0000","last_peered":"2024-09-19T05:45:21.716095+0000","last_clean":"2024-09-19T05:45:21.716095+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:21.716095+0000","last_undegraded":"2024-09-19T05:45:21.716095+0000","last_fullsized":"2024-09-19T05:45:21.716095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"84'168","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715985+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:45:21.715985+0000","last_peered":"2024-09-19T05:45:21.715985+0000","last_clean":"2024-09-19T05:45:21.715985+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:21.715985+0000","last_undegraded":"2024-09-19T05:45:21.715985+0000","last_fullsized":"2024-09-19T05:45:21.715985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"84'169","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687671+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:45:21.687671+0000","last_peered":"2024-09-19T05:45:21.687671+0000","last_clean":"2024-09-19T05:45:21.687671+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:21.687671+0000","last_undegraded":"2024-09-19T05:45:21.687671+0000","last_fullsized":"2024-09-19T05:45:21.687671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"84'198","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716789+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:45:21.716789+0000","last_peered":"2024-09-19T05:45:21.716789+0000","last_clean":"2024-09-19T05:45:21.716789+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:21.716789+0000","last_undegraded":"2024-09-19T05:45:21.716789+0000","last_fullsized":"2024-09-19T05:45:21.716789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"84'171","reported_seq":353,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688653+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:45:21.688653+0000","last_peered":"2024-09-19T05:45:21.688653+0000","last_clean":"2024-09-19T05:45:21.688653+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:21.688653+0000","last_undegraded":"2024-09-19T05:45:21.688653+0000","last_fullsized":"2024-09-19T05:45:21.688653+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:17.766409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"84'143","reported_seq":315,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708984+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:45:21.708984+0000","last_peered":"2024-09-19T05:45:21.708984+0000","last_clean":"2024-09-19T05:45:21.708984+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:21.708984+0000","last_undegraded":"2024-09-19T05:45:21.708984+0000","last_fullsized":"2024-09-19T05:45:21.708984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"84'189","reported_seq":246,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687810+0000","last_change":"2024-09-19T05:44:55.559370+0000","last_active":"2024-09-19T05:45:21.687810+0000","last_peered":"2024-09-19T05:45:21.687810+0000","last_clean":"2024-09-19T05:45:21.687810+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:21.687810+0000","last_undegraded":"2024-09-19T05:45:21.687810+0000","last_fullsized":"2024-09-19T05:45:21.687810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:25.838393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'142","reported_seq":295,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687168+0000","last_change":"2024-09-19T05:44:56.134769+0000","last_active":"2024-09-19T05:45:21.687168+0000","last_peered":"2024-09-19T05:45:21.687168+0000","last_clean":"2024-09-19T05:45:21.687168+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:21.687168+0000","last_undegraded":"2024-09-19T05:45:21.687168+0000","last_fullsized":"2024-09-19T05:45:21.687168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:06.255089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"84'201","reported_seq":492,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716107+0000","last_change":"2024-09-19T05:44:56.565770+0000","last_active":"2024-09-19T05:45:21.716107+0000","last_peered":"2024-09-19T05:45:21.716107+0000","last_clean":"2024-09-19T05:45:21.716107+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:21.716107+0000","last_undegraded":"2024-09-19T05:45:21.716107+0000","last_fullsized":"2024-09-19T05:45:21.716107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:03.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"84'158","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688447+0000","last_change":"2024-09-19T05:44:56.582314+0000","last_active":"2024-09-19T05:45:21.688447+0000","last_peered":"2024-09-19T05:45:21.688447+0000","last_clean":"2024-09-19T05:45:21.688447+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:21.688447+0000","last_undegraded":"2024-09-19T05:45:21.688447+0000","last_fullsized":"2024-09-19T05:45:21.688447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:41.223871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"84'202","reported_seq":374,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935387+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:45:21.935387+0000","last_peered":"2024-09-19T05:45:21.935387+0000","last_clean":"2024-09-19T05:45:21.935387+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:21.935387+0000","last_undegraded":"2024-09-19T05:45:21.935387+0000","last_fullsized":"2024-09-19T05:45:21.935387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773382+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:45:21.773382+0000","last_peered":"2024-09-19T05:45:21.773382+0000","last_clean":"2024-09-19T05:45:21.773382+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:21.773382+0000","last_undegraded":"2024-09-19T05:45:21.773382+0000","last_fullsized":"2024-09-19T05:45:21.773382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"84'148","reported_seq":320,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773260+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:45:21.773260+0000","last_peered":"2024-09-19T05:45:21.773260+0000","last_clean":"2024-09-19T05:45:21.773260+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:21.773260+0000","last_undegraded":"2024-09-19T05:45:21.773260+0000","last_fullsized":"2024-09-19T05:45:21.773260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'127","reported_seq":281,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688046+0000","last_change":"2024-09-19T05:44:54.977580+0000","last_active":"2024-09-19T05:45:21.688046+0000","last_peered":"2024-09-19T05:45:21.688046+0000","last_clean":"2024-09-19T05:45:21.688046+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:21.688046+0000","last_undegraded":"2024-09-19T05:45:21.688046+0000","last_fullsized":"2024-09-19T05:45:21.688046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:12.165530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"84'145","reported_seq":408,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716323+0000","last_change":"2024-09-19T05:44:53.510921+0000","last_active":"2024-09-19T05:45:21.716323+0000","last_peered":"2024-09-19T05:45:21.716323+0000","last_clean":"2024-09-19T05:45:21.716323+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:21.716323+0000","last_undegraded":"2024-09-19T05:45:21.716323+0000","last_fullsized":"2024-09-19T05:45:21.716323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:39.530204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"84'146","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687436+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:45:21.687436+0000","last_peered":"2024-09-19T05:45:21.687436+0000","last_clean":"2024-09-19T05:45:21.687436+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:21.687436+0000","last_undegraded":"2024-09-19T05:45:21.687436+0000","last_fullsized":"2024-09-19T05:45:21.687436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'171","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935092+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:45:21.935092+0000","last_peered":"2024-09-19T05:45:21.935092+0000","last_clean":"2024-09-19T05:45:21.935092+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:21.935092+0000","last_undegraded":"2024-09-19T05:45:21.935092+0000","last_fullsized":"2024-09-19T05:45:21.935092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"84'142","reported_seq":240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189849+0000","last_change":"2024-09-19T05:45:10.624968+0000","last_active":"2024-09-19T05:45:22.189849+0000","last_peered":"2024-09-19T05:45:22.189849+0000","last_clean":"2024-09-19T05:45:22.189849+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:22.189849+0000","last_undegraded":"2024-09-19T05:45:22.189849+0000","last_fullsized":"2024-09-19T05:45:22.189849+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:59.539763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"94'1809","reported_seq":1885,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687685+0000","last_change":"2024-09-19T05:44:55.560257+0000","last_active":"2024-09-19T05:45:21.687685+0000","last_peered":"2024-09-19T05:45:21.687685+0000","last_clean":"2024-09-19T05:45:21.687685+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:21.687685+0000","last_undegraded":"2024-09-19T05:45:21.687685+0000","last_fullsized":"2024-09-19T05:45:21.687685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1809,"log_dups_size":0,"ondisk_log_size":1809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:25.647547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'154","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708831+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:45:21.708831+0000","last_peered":"2024-09-19T05:45:21.708831+0000","last_clean":"2024-09-19T05:45:21.708831+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:21.708831+0000","last_undegraded":"2024-09-19T05:45:21.708831+0000","last_fullsized":"2024-09-19T05:45:21.708831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716460+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:45:21.716460+0000","last_peered":"2024-09-19T05:45:21.716460+0000","last_clean":"2024-09-19T05:45:21.716460+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:21.716460+0000","last_undegraded":"2024-09-19T05:45:21.716460+0000","last_fullsized":"2024-09-19T05:45:21.716460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773054+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:45:21.773054+0000","last_peered":"2024-09-19T05:45:21.773054+0000","last_clean":"2024-09-19T05:45:21.773054+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:21.773054+0000","last_undegraded":"2024-09-19T05:45:21.773054+0000","last_fullsized":"2024-09-19T05:45:21.773054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'182","reported_seq":337,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687351+0000","last_change":"2024-09-19T05:44:59.989289+0000","last_active":"2024-09-19T05:45:21.687351+0000","last_peered":"2024-09-19T05:45:21.687351+0000","last_clean":"2024-09-19T05:45:21.687351+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:21.687351+0000","last_undegraded":"2024-09-19T05:45:21.687351+0000","last_fullsized":"2024-09-19T05:45:21.687351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:52.052457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716243+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:45:21.716243+0000","last_peered":"2024-09-19T05:45:21.716243+0000","last_clean":"2024-09-19T05:45:21.716243+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:21.716243+0000","last_undegraded":"2024-09-19T05:45:21.716243+0000","last_fullsized":"2024-09-19T05:45:21.716243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"84'177","reported_seq":359,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708983+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:45:21.708983+0000","last_peered":"2024-09-19T05:45:21.708983+0000","last_clean":"2024-09-19T05:45:21.708983+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:21.708983+0000","last_undegraded":"2024-09-19T05:45:21.708983+0000","last_fullsized":"2024-09-19T05:45:21.708983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190091+0000","last_change":"2024-09-19T05:45:10.622931+0000","last_active":"2024-09-19T05:45:22.190091+0000","last_peered":"2024-09-19T05:45:22.190091+0000","last_clean":"2024-09-19T05:45:22.190091+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:22.190091+0000","last_undegraded":"2024-09-19T05:45:22.190091+0000","last_fullsized":"2024-09-19T05:45:22.190091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:34:06.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"84'241","reported_seq":413,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935402+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:45:21.935402+0000","last_peered":"2024-09-19T05:45:21.935402+0000","last_clean":"2024-09-19T05:45:21.935402+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:21.935402+0000","last_undegraded":"2024-09-19T05:45:21.935402+0000","last_fullsized":"2024-09-19T05:45:21.935402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688495+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:45:21.688495+0000","last_peered":"2024-09-19T05:45:21.688495+0000","last_clean":"2024-09-19T05:45:21.688495+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:21.688495+0000","last_undegraded":"2024-09-19T05:45:21.688495+0000","last_fullsized":"2024-09-19T05:45:21.688495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:39.063921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"84'151","reported_seq":300,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688324+0000","last_change":"2024-09-19T05:44:59.517543+0000","last_active":"2024-09-19T05:45:21.688324+0000","last_peered":"2024-09-19T05:45:21.688324+0000","last_clean":"2024-09-19T05:45:21.688324+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:21.688324+0000","last_undegraded":"2024-09-19T05:45:21.688324+0000","last_fullsized":"2024-09-19T05:45:21.688324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.453343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"89'10","reported_seq":185,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687787+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:45:21.687787+0000","last_peered":"2024-09-19T05:45:21.687787+0000","last_clean":"2024-09-19T05:45:21.687787+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:21.687787+0000","last_undegraded":"2024-09-19T05:45:21.687787+0000","last_fullsized":"2024-09-19T05:45:21.687787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"84'157","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716300+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:45:21.716300+0000","last_peered":"2024-09-19T05:45:21.716300+0000","last_clean":"2024-09-19T05:45:21.716300+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:21.716300+0000","last_undegraded":"2024-09-19T05:45:21.716300+0000","last_fullsized":"2024-09-19T05:45:21.716300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"94'2017","reported_seq":2191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935286+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:45:21.935286+0000","last_peered":"2024-09-19T05:45:21.935286+0000","last_clean":"2024-09-19T05:45:21.935286+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:21.935286+0000","last_undegraded":"2024-09-19T05:45:21.935286+0000","last_fullsized":"2024-09-19T05:45:21.935286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688257+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:45:21.688257+0000","last_peered":"2024-09-19T05:45:21.688257+0000","last_clean":"2024-09-19T05:45:21.688257+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:21.688257+0000","last_undegraded":"2024-09-19T05:45:21.688257+0000","last_fullsized":"2024-09-19T05:45:21.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687838+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:45:21.687838+0000","last_peered":"2024-09-19T05:45:21.687838+0000","last_clean":"2024-09-19T05:45:21.687838+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:21.687838+0000","last_undegraded":"2024-09-19T05:45:21.687838+0000","last_fullsized":"2024-09-19T05:45:21.687838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"84'168","reported_seq":340,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773221+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:45:21.773221+0000","last_peered":"2024-09-19T05:45:21.773221+0000","last_clean":"2024-09-19T05:45:21.773221+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:21.773221+0000","last_undegraded":"2024-09-19T05:45:21.773221+0000","last_fullsized":"2024-09-19T05:45:21.773221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716531+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:45:21.716531+0000","last_peered":"2024-09-19T05:45:21.716531+0000","last_clean":"2024-09-19T05:45:21.716531+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:21.716531+0000","last_undegraded":"2024-09-19T05:45:21.716531+0000","last_fullsized":"2024-09-19T05:45:21.716531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"84'186","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715983+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:45:21.715983+0000","last_peered":"2024-09-19T05:45:21.715983+0000","last_clean":"2024-09-19T05:45:21.715983+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:21.715983+0000","last_undegraded":"2024-09-19T05:45:21.715983+0000","last_fullsized":"2024-09-19T05:45:21.715983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687854+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:45:21.687854+0000","last_peered":"2024-09-19T05:45:21.687854+0000","last_clean":"2024-09-19T05:45:21.687854+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:21.687854+0000","last_undegraded":"2024-09-19T05:45:21.687854+0000","last_fullsized":"2024-09-19T05:45:21.687854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"84'191","reported_seq":283,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189767+0000","last_change":"2024-09-19T05:45:10.627674+0000","last_active":"2024-09-19T05:45:22.189767+0000","last_peered":"2024-09-19T05:45:22.189767+0000","last_clean":"2024-09-19T05:45:22.189767+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:22.189767+0000","last_undegraded":"2024-09-19T05:45:22.189767+0000","last_fullsized":"2024-09-19T05:45:22.189767+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:21.685895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"89'9","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708963+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:45:21.708963+0000","last_peered":"2024-09-19T05:45:21.708963+0000","last_clean":"2024-09-19T05:45:21.708963+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:21.708963+0000","last_undegraded":"2024-09-19T05:45:21.708963+0000","last_fullsized":"2024-09-19T05:45:21.708963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"84'209","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716252+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:45:21.716252+0000","last_peered":"2024-09-19T05:45:21.716252+0000","last_clean":"2024-09-19T05:45:21.716252+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:21.716252+0000","last_undegraded":"2024-09-19T05:45:21.716252+0000","last_fullsized":"2024-09-19T05:45:21.716252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688782+0000","last_change":"2024-09-19T05:44:53.065559+0000","last_active":"2024-09-19T05:45:21.688782+0000","last_peered":"2024-09-19T05:45:21.688782+0000","last_clean":"2024-09-19T05:45:21.688782+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:21.688782+0000","last_undegraded":"2024-09-19T05:45:21.688782+0000","last_fullsized":"2024-09-19T05:45:21.688782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:13.889455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935000+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:45:21.935000+0000","last_peered":"2024-09-19T05:45:21.935000+0000","last_clean":"2024-09-19T05:45:21.935000+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:21.935000+0000","last_undegraded":"2024-09-19T05:45:21.935000+0000","last_fullsized":"2024-09-19T05:45:21.935000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709231+0000","last_change":"2024-09-19T05:44:53.059769+0000","last_active":"2024-09-19T05:45:21.709231+0000","last_peered":"2024-09-19T05:45:21.709231+0000","last_clean":"2024-09-19T05:45:21.709231+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:21.709231+0000","last_undegraded":"2024-09-19T05:45:21.709231+0000","last_fullsized":"2024-09-19T05:45:21.709231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:03.909805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"84'153","reported_seq":231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687781+0000","last_change":"2024-09-19T05:44:15.787415+0000","last_active":"2024-09-19T05:45:21.687781+0000","last_peered":"2024-09-19T05:45:21.687781+0000","last_clean":"2024-09-19T05:45:21.687781+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:21.687781+0000","last_undegraded":"2024-09-19T05:45:21.687781+0000","last_fullsized":"2024-09-19T05:45:21.687781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:10.095369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":125,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688186+0000","last_change":"2024-09-19T05:44:53.058979+0000","last_active":"2024-09-19T05:45:21.688186+0000","last_peered":"2024-09-19T05:45:21.688186+0000","last_clean":"2024-09-19T05:45:21.688186+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:21.688186+0000","last_undegraded":"2024-09-19T05:45:21.688186+0000","last_fullsized":"2024-09-19T05:45:21.688186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:04.530725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686740+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:45:21.686740+0000","last_peered":"2024-09-19T05:45:21.686740+0000","last_clean":"2024-09-19T05:45:21.686740+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:21.686740+0000","last_undegraded":"2024-09-19T05:45:21.686740+0000","last_fullsized":"2024-09-19T05:45:21.686740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773397+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:45:21.773397+0000","last_peered":"2024-09-19T05:45:21.773397+0000","last_clean":"2024-09-19T05:45:21.773397+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:21.773397+0000","last_undegraded":"2024-09-19T05:45:21.773397+0000","last_fullsized":"2024-09-19T05:45:21.773397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"84'168","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716718+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:45:21.716718+0000","last_peered":"2024-09-19T05:45:21.716718+0000","last_clean":"2024-09-19T05:45:21.716718+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:21.716718+0000","last_undegraded":"2024-09-19T05:45:21.716718+0000","last_fullsized":"2024-09-19T05:45:21.716718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":260,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189834+0000","last_change":"2024-09-19T05:45:10.627237+0000","last_active":"2024-09-19T05:45:22.189834+0000","last_peered":"2024-09-19T05:45:22.189834+0000","last_clean":"2024-09-19T05:45:22.189834+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:22.189834+0000","last_undegraded":"2024-09-19T05:45:22.189834+0000","last_fullsized":"2024-09-19T05:45:22.189834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:04.886546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716440+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:45:21.716440+0000","last_peered":"2024-09-19T05:45:21.716440+0000","last_clean":"2024-09-19T05:45:21.716440+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:21.716440+0000","last_undegraded":"2024-09-19T05:45:21.716440+0000","last_fullsized":"2024-09-19T05:45:21.716440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708974+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:45:21.708974+0000","last_peered":"2024-09-19T05:45:21.708974+0000","last_clean":"2024-09-19T05:45:21.708974+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:21.708974+0000","last_undegraded":"2024-09-19T05:45:21.708974+0000","last_fullsized":"2024-09-19T05:45:21.708974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716388+0000","last_change":"2024-09-19T05:44:53.069283+0000","last_active":"2024-09-19T05:45:21.716388+0000","last_peered":"2024-09-19T05:45:21.716388+0000","last_clean":"2024-09-19T05:45:21.716388+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:21.716388+0000","last_undegraded":"2024-09-19T05:45:21.716388+0000","last_fullsized":"2024-09-19T05:45:21.716388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:51.209720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773046+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:45:21.773046+0000","last_peered":"2024-09-19T05:45:21.773046+0000","last_clean":"2024-09-19T05:45:21.773046+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:21.773046+0000","last_undegraded":"2024-09-19T05:45:21.773046+0000","last_fullsized":"2024-09-19T05:45:21.773046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687539+0000","last_change":"2024-09-19T05:44:53.068415+0000","last_active":"2024-09-19T05:45:21.687539+0000","last_peered":"2024-09-19T05:45:21.687539+0000","last_clean":"2024-09-19T05:45:21.687539+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:21.687539+0000","last_undegraded":"2024-09-19T05:45:21.687539+0000","last_fullsized":"2024-09-19T05:45:21.687539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:16.609871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"84'157","reported_seq":310,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687083+0000","last_change":"2024-09-19T05:44:58.820818+0000","last_active":"2024-09-19T05:45:21.687083+0000","last_peered":"2024-09-19T05:45:21.687083+0000","last_clean":"2024-09-19T05:45:21.687083+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:21.687083+0000","last_undegraded":"2024-09-19T05:45:21.687083+0000","last_fullsized":"2024-09-19T05:45:21.687083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:32.427214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716567+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:45:21.716567+0000","last_peered":"2024-09-19T05:45:21.716567+0000","last_clean":"2024-09-19T05:45:21.716567+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:21.716567+0000","last_undegraded":"2024-09-19T05:45:21.716567+0000","last_fullsized":"2024-09-19T05:45:21.716567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686759+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:45:21.686759+0000","last_peered":"2024-09-19T05:45:21.686759+0000","last_clean":"2024-09-19T05:45:21.686759+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:21.686759+0000","last_undegraded":"2024-09-19T05:45:21.686759+0000","last_fullsized":"2024-09-19T05:45:21.686759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687970+0000","last_change":"2024-09-19T05:44:53.063483+0000","last_active":"2024-09-19T05:45:21.687970+0000","last_peered":"2024-09-19T05:45:21.687970+0000","last_clean":"2024-09-19T05:45:21.687970+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:21.687970+0000","last_undegraded":"2024-09-19T05:45:21.687970+0000","last_fullsized":"2024-09-19T05:45:21.687970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:40.306784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"84'155","reported_seq":327,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935176+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:45:21.935176+0000","last_peered":"2024-09-19T05:45:21.935176+0000","last_clean":"2024-09-19T05:45:21.935176+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:21.935176+0000","last_undegraded":"2024-09-19T05:45:21.935176+0000","last_fullsized":"2024-09-19T05:45:21.935176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716905+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:45:21.716905+0000","last_peered":"2024-09-19T05:45:21.716905+0000","last_clean":"2024-09-19T05:45:21.716905+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:21.716905+0000","last_undegraded":"2024-09-19T05:45:21.716905+0000","last_fullsized":"2024-09-19T05:45:21.716905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'153","reported_seq":424,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773336+0000","last_change":"2024-09-19T05:44:55.675882+0000","last_active":"2024-09-19T05:45:21.773336+0000","last_peered":"2024-09-19T05:45:21.773336+0000","last_clean":"2024-09-19T05:45:21.773336+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:21.773336+0000","last_undegraded":"2024-09-19T05:45:21.773336+0000","last_fullsized":"2024-09-19T05:45:21.773336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:04.593886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709321+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:45:21.709321+0000","last_peered":"2024-09-19T05:45:21.709321+0000","last_clean":"2024-09-19T05:45:21.709321+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:21.709321+0000","last_undegraded":"2024-09-19T05:45:21.709321+0000","last_fullsized":"2024-09-19T05:45:21.709321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"84'161","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709277+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:45:21.709277+0000","last_peered":"2024-09-19T05:45:21.709277+0000","last_clean":"2024-09-19T05:45:21.709277+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:21.709277+0000","last_undegraded":"2024-09-19T05:45:21.709277+0000","last_fullsized":"2024-09-19T05:45:21.709277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"85'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716921+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:45:21.716921+0000","last_peered":"2024-09-19T05:45:21.716921+0000","last_clean":"2024-09-19T05:45:21.716921+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:21.716921+0000","last_undegraded":"2024-09-19T05:45:21.716921+0000","last_fullsized":"2024-09-19T05:45:21.716921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"84'176","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709220+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:45:21.709220+0000","last_peered":"2024-09-19T05:45:21.709220+0000","last_clean":"2024-09-19T05:45:21.709220+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:21.709220+0000","last_undegraded":"2024-09-19T05:45:21.709220+0000","last_fullsized":"2024-09-19T05:45:21.709220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687243+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:45:21.687243+0000","last_peered":"2024-09-19T05:45:21.687243+0000","last_clean":"2024-09-19T05:45:21.687243+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:21.687243+0000","last_undegraded":"2024-09-19T05:45:21.687243+0000","last_fullsized":"2024-09-19T05:45:21.687243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"84'160","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716327+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:45:21.716327+0000","last_peered":"2024-09-19T05:45:21.716327+0000","last_clean":"2024-09-19T05:45:21.716327+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:21.716327+0000","last_undegraded":"2024-09-19T05:45:21.716327+0000","last_fullsized":"2024-09-19T05:45:21.716327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709021+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:45:21.709021+0000","last_peered":"2024-09-19T05:45:21.709021+0000","last_clean":"2024-09-19T05:45:21.709021+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:21.709021+0000","last_undegraded":"2024-09-19T05:45:21.709021+0000","last_fullsized":"2024-09-19T05:45:21.709021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708902+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:45:21.708902+0000","last_peered":"2024-09-19T05:45:21.708902+0000","last_clean":"2024-09-19T05:45:21.708902+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:21.708902+0000","last_undegraded":"2024-09-19T05:45:21.708902+0000","last_fullsized":"2024-09-19T05:45:21.708902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773238+0000","last_change":"2024-09-19T05:44:53.065594+0000","last_active":"2024-09-19T05:45:21.773238+0000","last_peered":"2024-09-19T05:45:21.773238+0000","last_clean":"2024-09-19T05:45:21.773238+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:21.773238+0000","last_undegraded":"2024-09-19T05:45:21.773238+0000","last_fullsized":"2024-09-19T05:45:21.773238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:47.254673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"84'156","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687718+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:45:21.687718+0000","last_peered":"2024-09-19T05:45:21.687718+0000","last_clean":"2024-09-19T05:45:21.687718+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:21.687718+0000","last_undegraded":"2024-09-19T05:45:21.687718+0000","last_fullsized":"2024-09-19T05:45:21.687718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"59'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716236+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:45:21.716236+0000","last_peered":"2024-09-19T05:45:21.716236+0000","last_clean":"2024-09-19T05:45:21.716236+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:21.716236+0000","last_undegraded":"2024-09-19T05:45:21.716236+0000","last_fullsized":"2024-09-19T05:45:21.716236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"84'149","reported_seq":330,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716425+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:45:21.716425+0000","last_peered":"2024-09-19T05:45:21.716425+0000","last_clean":"2024-09-19T05:45:21.716425+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:21.716425+0000","last_undegraded":"2024-09-19T05:45:21.716425+0000","last_fullsized":"2024-09-19T05:45:21.716425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"84'190","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715986+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:45:21.715986+0000","last_peered":"2024-09-19T05:45:21.715986+0000","last_clean":"2024-09-19T05:45:21.715986+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:21.715986+0000","last_undegraded":"2024-09-19T05:45:21.715986+0000","last_fullsized":"2024-09-19T05:45:21.715986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":194,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688923+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:45:21.688923+0000","last_peered":"2024-09-19T05:45:21.688923+0000","last_clean":"2024-09-19T05:45:21.688923+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:21.688923+0000","last_undegraded":"2024-09-19T05:45:21.688923+0000","last_fullsized":"2024-09-19T05:45:21.688923+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:59.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"84'153","reported_seq":266,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189956+0000","last_change":"2024-09-19T05:45:10.627058+0000","last_active":"2024-09-19T05:45:22.189956+0000","last_peered":"2024-09-19T05:45:22.189956+0000","last_clean":"2024-09-19T05:45:22.189956+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:22.189956+0000","last_undegraded":"2024-09-19T05:45:22.189956+0000","last_fullsized":"2024-09-19T05:45:22.189956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:52.373727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"70'4","reported_seq":196,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688300+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:45:21.688300+0000","last_peered":"2024-09-19T05:45:21.688300+0000","last_clean":"2024-09-19T05:45:21.688300+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:21.688300+0000","last_undegraded":"2024-09-19T05:45:21.688300+0000","last_fullsized":"2024-09-19T05:45:21.688300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:52.607339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"84'154","reported_seq":314,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688325+0000","last_change":"2024-09-19T05:44:57.957776+0000","last_active":"2024-09-19T05:45:21.688325+0000","last_peered":"2024-09-19T05:45:21.688325+0000","last_clean":"2024-09-19T05:45:21.688325+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:21.688325+0000","last_undegraded":"2024-09-19T05:45:21.688325+0000","last_fullsized":"2024-09-19T05:45:21.688325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:37.587819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":156,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688211+0000","last_change":"2024-09-19T05:44:53.061701+0000","last_active":"2024-09-19T05:45:21.688211+0000","last_peered":"2024-09-19T05:45:21.688211+0000","last_clean":"2024-09-19T05:45:21.688211+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:21.688211+0000","last_undegraded":"2024-09-19T05:45:21.688211+0000","last_fullsized":"2024-09-19T05:45:21.688211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:26.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"84'176","reported_seq":321,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688312+0000","last_change":"2024-09-19T05:44:57.503018+0000","last_active":"2024-09-19T05:45:21.688312+0000","last_peered":"2024-09-19T05:45:21.688312+0000","last_clean":"2024-09-19T05:45:21.688312+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:21.688312+0000","last_undegraded":"2024-09-19T05:45:21.688312+0000","last_fullsized":"2024-09-19T05:45:21.688312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:44.482643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709387+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:45:21.709387+0000","last_peered":"2024-09-19T05:45:21.709387+0000","last_clean":"2024-09-19T05:45:21.709387+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:21.709387+0000","last_undegraded":"2024-09-19T05:45:21.709387+0000","last_fullsized":"2024-09-19T05:45:21.709387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970373,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3906,"num_write_kb":11500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15940746,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3893,"ondisk_log_size":3893,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":5272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":5398528,"data_stored":4749255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":4984832,"data_stored":4334380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449096,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46800,"kb_used_data":1724,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325040,"statfs":{"total":96636764160,"available":96588840960,"internally_reserved":0,"allocated":1765376,"data_stored":4919338,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542183,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50364,"kb_used_data":5280,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321476,"statfs":{"total":96636764160,"available":96585191424,"internally_reserved":0,"allocated":5406720,"data_stored":4749927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542183,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40680,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331160,"statfs":{"total":96636764160,"available":96595107840,"internally_reserved":0,"allocated":1200128,"data_stored":554861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40732,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331108,"statfs":{"total":96636764160,"available":96595054592,"internally_reserved":0,"allocated":1249280,"data_stored":1144717,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40700,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331140,"statfs":{"total":96636764160,"available":96595087360,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774691,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:45:25.700 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-19T05:45:25.700 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-19T05:45:25.840 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.838+0000 7f919e0eb640 1 Processor -- start 2024-09-19T05:45:25.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.838+0000 7f919e0eb640 1 -- start start 2024-09-19T05:45:25.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f919e0eb640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9198154d50 0x7f9198073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f919e0eb640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f91980738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.843 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f919e0eb640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f91980cbec0 0x7f919815af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f919e0eb640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f9198155130 con 0x7f91980cbec0 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f919e0eb640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f91980cbc60 con 0x7f9198154d50 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f919e0eb640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f91980cb050 con 0x7f91980cc3f0 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f9197fff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f91980cbec0 0x7f919815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f9197fff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f91980cbec0 0x7f919815af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54804/0 (socket says 172.21.15.2:54804) 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f9197fff640 1 -- 172.21.15.2:0/3547563027 learned_addr learned my addr 172.21.15.2:0/3547563027 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f9196ffd640 1 --2- 172.21.15.2:0/3547563027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f91980738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.839+0000 7f91977fe640 1 --2- 172.21.15.2:0/3547563027 >> v2:172.21.15.123:3300/0 conn(0x7f9198154d50 0x7f9198073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f9197fff640 1 -- 172.21.15.2:0/3547563027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 msgr2=0x7f91980738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f9197fff640 1 --2- 172.21.15.2:0/3547563027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f91980738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f9197fff640 1 -- 172.21.15.2:0/3547563027 >> v2:172.21.15.123:3300/0 conn(0x7f9198154d50 msgr2=0x7f9198073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f9197fff640 1 --2- 172.21.15.2:0/3547563027 >> v2:172.21.15.123:3300/0 conn(0x7f9198154d50 0x7f9198073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f9197fff640 1 -- 172.21.15.2:0/3547563027 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91980cb330 con 0x7f91980cbec0 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f91977fe640 1 --2- 172.21.15.2:0/3547563027 >> v2:172.21.15.123:3300/0 conn(0x7f9198154d50 0x7f9198073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f9196ffd640 1 --2- 172.21.15.2:0/3547563027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f91980738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.840+0000 7f9197fff640 1 --2- 172.21.15.2:0/3547563027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f919815af50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f91980ce150 tx=0x7f919002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaed3cccceb2f521 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f91967fc640 1 -- 172.21.15.2:0/3547563027 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190036070 con 0x7f91980cbec0 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f91967fc640 1 -- 172.21.15.2:0/3547563027 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9190031600 con 0x7f91980cbec0 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f91967fc640 1 -- 172.21.15.2:0/3547563027 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91900319d0 con 0x7f91980cbec0 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 -- 172.21.15.2:0/3547563027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 msgr2=0x7f919815af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 --2- 172.21.15.2:0/3547563027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f919815af50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f91980ce150 tx=0x7f919002a730 comp rx=0 tx=0).stop 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 -- 172.21.15.2:0/3547563027 shutdown_connections 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 --2- 172.21.15.2:0/3547563027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f919815af50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 --2- 172.21.15.2:0/3547563027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f91980738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 --2- 172.21.15.2:0/3547563027 >> v2:172.21.15.123:3300/0 conn(0x7f9198154d50 0x7f9198073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 -- 172.21.15.2:0/3547563027 >> 172.21.15.2:0/3547563027 conn(0x7f919806d250 msgr2=0x7f919806d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 -- 172.21.15.2:0/3547563027 shutdown_connections 2024-09-19T05:45:25.846 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.841+0000 7f919e0eb640 1 -- 172.21.15.2:0/3547563027 wait complete. 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.842+0000 7f919e0eb640 1 Processor -- start 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f919e0eb640 1 -- start start 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f919e0eb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f91980cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f919e0eb640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f9198152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f919e0eb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9198154d50 0x7f9198152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f919e0eb640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f919815aa00 con 0x7f91980cbec0 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f919e0eb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91980c5c40 con 0x7f9198154d50 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f919e0eb640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f91980c69f0 con 0x7f91980cc3f0 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f91977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f91980cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f91977fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f91980cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54816/0 (socket says 172.21.15.2:54816) 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.843+0000 7f91977fe640 1 -- 172.21.15.2:0/2005400137 learned_addr learned my addr 172.21.15.2:0/2005400137 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f9196ffd640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f9198152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f9197fff640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9198154d50 0x7f9198152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f91977fe640 1 -- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 msgr2=0x7f9198152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f91977fe640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f9198152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f91977fe640 1 -- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9198154d50 msgr2=0x7f9198152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f91977fe640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9198154d50 0x7f9198152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f91977fe640 1 -- 172.21.15.2:0/2005400137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91980c7f40 con 0x7f91980cbec0 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f9197fff640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9198154d50 0x7f9198152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f91977fe640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f91980cf750 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f918800d800 tx=0x7f918800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f917ffff640 1 -- 172.21.15.2:0/2005400137 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188004030 con 0x7f91980cbec0 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f917ffff640 1 -- 172.21.15.2:0/2005400137 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91880041d0 con 0x7f91980cbec0 2024-09-19T05:45:25.851 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91980c8220 con 0x7f91980cbec0 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f917ffff640 1 -- 172.21.15.2:0/2005400137 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91880096a0 con 0x7f91980cbec0 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.844+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91980c9850 con 0x7f91980cbec0 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.845+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9198073390 con 0x7f91980cbec0 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.846+0000 7f917ffff640 1 -- 172.21.15.2:0/2005400137 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9188009840 con 0x7f91980cbec0 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.846+0000 7f917ffff640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f918406c2e0 0x7f918406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.846+0000 7f917ffff640 1 -- 172.21.15.2:0/2005400137 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f918808b780 con 0x7f91980cbec0 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.846+0000 7f9196ffd640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f918406c2e0 0x7f918406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:25.852 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.847+0000 7f9196ffd640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f918406c2e0 0x7f918406e7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f918c006fd0 tx=0x7f918c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:25.853 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:25.850+0000 7f917ffff640 1 -- 172.21.15.2:0/2005400137 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9188059e90 con 0x7f91980cbec0 2024-09-19T05:45:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.025+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) -- 0x7f919814d920 con 0x7f91980cbec0 2024-09-19T05:45:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.028+0000 7f917ffff640 1 -- 172.21.15.2:0/2005400137 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v11) ==== 118+0+0 (secure 0 0 0) 0x7f9188059790 con 0x7f91980cbec0 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f918406c2e0 msgr2=0x7f918406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f918406c2e0 0x7f918406e7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f918c006fd0 tx=0x7f918c008040 comp rx=0 tx=0).stop 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 msgr2=0x7f91980cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f91980cf750 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f918800d800 tx=0x7f918800dcc0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 shutdown_connections 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f918406c2e0 0x7f918406e7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9198154d50 0x7f9198152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91980cc3f0 0x7f9198152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.033 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 --2- 172.21.15.2:0/2005400137 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91980cbec0 0x7f91980cf750 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 >> 172.21.15.2:0/2005400137 conn(0x7f919806d250 msgr2=0x7f9198159300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:26.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 shutdown_connections 2024-09-19T05:45:26.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.030+0000 7f919e0eb640 1 -- 172.21.15.2:0/2005400137 wait complete. 2024-09-19T05:45:26.043 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T05:45:26.054 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T05:45:26.055 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:45:26.055 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:45:26.071 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-19T05:45:26.071 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T05:45:26.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T05:45:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.269+0000 7f4257896640 1 Processor -- start 2024-09-19T05:45:26.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.269+0000 7f4257896640 1 -- start start 2024-09-19T05:45:26.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4257896640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f42500cb640 0x7f42500c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4257896640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f425015ede0 0x7f42500c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4257896640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f425015ab40 con 0x7f42500cba20 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4257896640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f42500c5ef0 con 0x7f425015ede0 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4257896640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f425015ade0 con 0x7f42500cb640 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255e33640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f425015ede0 0x7f42500c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255e33640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f425015ede0 0x7f42500c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54824/0 (socket says 172.21.15.2:54824) 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255e33640 1 -- 172.21.15.2:0/1680973129 learned_addr learned my addr 172.21.15.2:0/1680973129 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255632640 1 --2- 172.21.15.2:0/1680973129 >> v2:172.21.15.123:3300/0 conn(0x7f42500cb640 0x7f42500c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255e33640 1 -- 172.21.15.2:0/1680973129 >> v2:172.21.15.123:3300/0 conn(0x7f42500cb640 msgr2=0x7f42500c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255e33640 1 --2- 172.21.15.2:0/1680973129 >> v2:172.21.15.123:3300/0 conn(0x7f42500cb640 0x7f42500c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255e33640 1 -- 172.21.15.2:0/1680973129 >> v1:172.21.15.123:6789/0 conn(0x7f42500cba20 legacy=0x7f42501549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:26.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255e33640 1 -- 172.21.15.2:0/1680973129 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425015b0c0 con 0x7f425015ede0 2024-09-19T05:45:26.274 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.270+0000 7f4255632640 1 --2- 172.21.15.2:0/1680973129 >> v2:172.21.15.123:3300/0 conn(0x7f42500cb640 0x7f42500c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.271+0000 7f4255e33640 1 --2- 172.21.15.2:0/1680973129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f42500c6380 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f42500ce0a0 tx=0x7f424802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d80acf07fc5c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.271+0000 7f4247fff640 1 -- 172.21.15.2:0/1680973129 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248036070 con 0x7f425015ede0 2024-09-19T05:45:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.271+0000 7f4247fff640 1 -- 172.21.15.2:0/1680973129 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4248031600 con 0x7f425015ede0 2024-09-19T05:45:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.271+0000 7f4247fff640 1 -- 172.21.15.2:0/1680973129 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42480319d0 con 0x7f425015ede0 2024-09-19T05:45:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 -- 172.21.15.2:0/1680973129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 msgr2=0x7f42500c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 --2- 172.21.15.2:0/1680973129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f42500c6380 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f42500ce0a0 tx=0x7f424802a760 comp rx=0 tx=0).stop 2024-09-19T05:45:26.275 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 -- 172.21.15.2:0/1680973129 shutdown_connections 2024-09-19T05:45:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 --2- 172.21.15.2:0/1680973129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f42500c6380 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 --2- 172.21.15.2:0/1680973129 >> v2:172.21.15.123:3300/0 conn(0x7f42500cb640 0x7f42500c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 -- 172.21.15.2:0/1680973129 >> 172.21.15.2:0/1680973129 conn(0x7f4250067950 msgr2=0x7f4250067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 -- 172.21.15.2:0/1680973129 shutdown_connections 2024-09-19T05:45:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.272+0000 7f4257896640 1 -- 172.21.15.2:0/1680973129 wait complete. 2024-09-19T05:45:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.273+0000 7f4257896640 1 Processor -- start 2024-09-19T05:45:26.276 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.273+0000 7f4257896640 1 -- start start 2024-09-19T05:45:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4257896640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42500cb640 0x7f42501f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4257896640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42500cba20 0x7f425015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4257896640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f425014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4257896640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f425015b960 con 0x7f425015ede0 2024-09-19T05:45:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4257896640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f425015b680 con 0x7f42500cb640 2024-09-19T05:45:26.278 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4257896640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f42501550f0 con 0x7f42500cba20 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f425014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f425014e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54830/0 (socket says 172.21.15.2:54830) 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 -- 172.21.15.2:0/3017554515 learned_addr learned my addr 172.21.15.2:0/3017554515 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255632640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42500cb640 0x7f42501f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4254e31640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42500cba20 0x7f425015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 -- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42500cba20 msgr2=0x7f425015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42500cba20 0x7f425015cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 -- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42500cb640 msgr2=0x7f42501f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42500cb640 0x7f42501f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 -- 172.21.15.2:0/3017554515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425015bce0 con 0x7f425015ede0 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4254e31640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42500cba20 0x7f425015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4255e33640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f425014e510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4248009f10 tx=0x7f4248004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:26.279 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4245ffb640 1 -- 172.21.15.2:0/3017554515 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248036050 con 0x7f425015ede0 2024-09-19T05:45:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.274+0000 7f4245ffb640 1 -- 172.21.15.2:0/3017554515 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4248042070 con 0x7f425015ede0 2024-09-19T05:45:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.275+0000 7f4245ffb640 1 -- 172.21.15.2:0/3017554515 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248007b90 con 0x7f425015ede0 2024-09-19T05:45:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.275+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42500b5e40 con 0x7f425015ede0 2024-09-19T05:45:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.275+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42500b6400 con 0x7f425015ede0 2024-09-19T05:45:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.276+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42501549e0 con 0x7f425015ede0 2024-09-19T05:45:26.280 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.276+0000 7f4245ffb640 1 -- 172.21.15.2:0/3017554515 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4248047460 con 0x7f425015ede0 2024-09-19T05:45:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.277+0000 7f4245ffb640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f423406c2b0 0x7f423406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.277+0000 7f4245ffb640 1 -- 172.21.15.2:0/3017554515 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f42480ac430 con 0x7f425015ede0 2024-09-19T05:45:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.277+0000 7f4255632640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f423406c2b0 0x7f423406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.281 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.278+0000 7f4255632640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f423406c2b0 0x7f423406e770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f424c00ab60 tx=0x7f424c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:26.283 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.281+0000 7f4245ffb640 1 -- 172.21.15.2:0/3017554515 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f424807aa10 con 0x7f425015ede0 2024-09-19T05:45:26.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.469+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4250158e50 con 0x7f425015ede0 2024-09-19T05:45:26.472 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.470+0000 7f4245ffb640 1 -- 172.21.15.2:0/3017554515 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) ==== 80+0+0 (secure 0 0 0) 0x7f424807a310 con 0x7f425015ede0 2024-09-19T05:45:26.472 INFO:teuthology.orchestra.run.smithi002.stderr:listed 0 entries 2024-09-19T05:45:26.474 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f423406c2b0 msgr2=0x7f423406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f423406c2b0 0x7f423406e770 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f424c00ab60 tx=0x7f424c008040 comp rx=0 tx=0).stop 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 msgr2=0x7f425014e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f425014e510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4248009f10 tx=0x7f4248004a40 comp rx=0 tx=0).stop 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 shutdown_connections 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f423406c2b0 0x7f423406e770 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f425015ede0 0x7f425014e510 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f42500cba20 0x7f425015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 --2- 172.21.15.2:0/3017554515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42500cb640 0x7f42501f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 >> 172.21.15.2:0/3017554515 conn(0x7f4250067950 msgr2=0x7f4250067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 shutdown_connections 2024-09-19T05:45:26.475 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.472+0000 7f4257896640 1 -- 172.21.15.2:0/3017554515 wait complete. 2024-09-19T05:45:26.486 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T05:45:26.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:45:26.486 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T05:45:26.532 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:45:26.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:45:26.532 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T05:45:26.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:45:26.574 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2024-09-19T05:45:26.631 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T05:45:26.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:45:26.632 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T05:45:26.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T05:45:26.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2024-09-19T05:45:26.797 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-19T05:45:26.807 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T05:45:26.807 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:45:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.909+0000 7fcfc4ee6640 1 Processor -- start 2024-09-19T05:45:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.909+0000 7fcfc4ee6640 1 -- start start 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfc4ee6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc015b480 0x7fcfc0070b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfc4ee6640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcfc006e9e0 0x7fcfc00ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfc4ee6640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc0070720 con 0x7fcfc00c9660 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfc4ee6640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc006cd20 con 0x7fcfc006e9e0 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfc4ee6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc00b13a0 con 0x7fcfc015b480 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbed9d640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fcfc00c9660 0x7fcfc015dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54150/0 (socket says 172.21.15.2:54150) 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbed9d640 1 -- 172.21.15.2:0/3013304927 learned_addr learned my addr 172.21.15.2:0/3013304927 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbdd9b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcfc006e9e0 0x7fcfc00ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbd59a640 1 -- 172.21.15.2:0/3013304927 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248239583 0 0) 0x7fcfc0070720 con 0x7fcfc00c9660 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbe59c640 1 --2- 172.21.15.2:0/3013304927 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc015b480 0x7fcfc0070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbd59a640 1 -- 172.21.15.2:0/3013304927 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcfa8003520 con 0x7fcfc00c9660 2024-09-19T05:45:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbdd9b640 1 -- 172.21.15.2:0/3013304927 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc015b480 msgr2=0x7fcfc0070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbdd9b640 1 --2- 172.21.15.2:0/3013304927 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc015b480 0x7fcfc0070b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbdd9b640 1 -- 172.21.15.2:0/3013304927 >> v1:172.21.15.2:6789/0 conn(0x7fcfc00c9660 legacy=0x7fcfc015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.910+0000 7fcfbdd9b640 1 -- 172.21.15.2:0/3013304927 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc006cf70 con 0x7fcfc006e9e0 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.911+0000 7fcfbdd9b640 1 --2- 172.21.15.2:0/3013304927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc006e9e0 0x7fcfc00ce550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40097a0 tx=0x7fcfb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf859202b6568f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.911+0000 7fcfbd59a640 1 -- 172.21.15.2:0/3013304927 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4037070 con 0x7fcfc006e9e0 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.911+0000 7fcfbd59a640 1 -- 172.21.15.2:0/3013304927 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb4032440 con 0x7fcfc006e9e0 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.911+0000 7fcfbd59a640 1 -- 172.21.15.2:0/3013304927 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4032810 con 0x7fcfc006e9e0 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/3013304927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc006e9e0 msgr2=0x7fcfc00ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/3013304927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc006e9e0 0x7fcfc00ce550 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40097a0 tx=0x7fcfb402a180 comp rx=0 tx=0).stop 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/3013304927 shutdown_connections 2024-09-19T05:45:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/3013304927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc006e9e0 0x7fcfc00ce550 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/3013304927 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc015b480 0x7fcfc0070b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/3013304927 >> 172.21.15.2:0/3013304927 conn(0x7fcfc006c240 msgr2=0x7fcfc0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/3013304927 shutdown_connections 2024-09-19T05:45:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.912+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/3013304927 wait complete. 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.913+0000 7fcfc4ee6640 1 Processor -- start 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.913+0000 7fcfc4ee6640 1 -- start start 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfc4ee6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc006e9e0 0x7fcfc00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfc4ee6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfc00c9660 0x7fcfc01f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfc4ee6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc015b480 0x7fcfc00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfc4ee6640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00c9000 con 0x7fcfc015b480 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfc4ee6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00b0e80 con 0x7fcfc00c9660 2024-09-19T05:45:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfc4ee6640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc015b860 con 0x7fcfc006e9e0 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfbed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc015b480 0x7fcfc00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfbed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc015b480 0x7fcfc00cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54842/0 (socket says 172.21.15.2:54842) 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfbed9d640 1 -- 172.21.15.2:0/4184326417 learned_addr learned my addr 172.21.15.2:0/4184326417 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfbe59c640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc006e9e0 0x7fcfc00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.914+0000 7fcfbdd9b640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfc00c9660 0x7fcfc01f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfbed9d640 1 -- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc006e9e0 msgr2=0x7fcfc00ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfbed9d640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc006e9e0 0x7fcfc00ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfbed9d640 1 -- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfc00c9660 msgr2=0x7fcfc01f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfbed9d640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfc00c9660 0x7fcfc01f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfbed9d640 1 -- 172.21.15.2:0/4184326417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc00c3da0 con 0x7fcfc015b480 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfbed9d640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc015b480 0x7fcfc00cc500 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800cb70 tx=0x7fcfb800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfaeffd640 1 -- 172.21.15.2:0/4184326417 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8017070 con 0x7fcfc015b480 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfaeffd640 1 -- 172.21.15.2:0/4184326417 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb800ad40 con 0x7fcfc015b480 2024-09-19T05:45:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc00c37e0 con 0x7fcfc015b480 2024-09-19T05:45:26.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfaeffd640 1 -- 172.21.15.2:0/4184326417 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8012620 con 0x7fcfc015b480 2024-09-19T05:45:26.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.915+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc01f3520 con 0x7fcfc015b480 2024-09-19T05:45:26.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.916+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc006c8a0 con 0x7fcfc015b480 2024-09-19T05:45:26.920 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.917+0000 7fcfaeffd640 1 -- 172.21.15.2:0/4184326417 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcfb801a070 con 0x7fcfc015b480 2024-09-19T05:45:26.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.917+0000 7fcfaeffd640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcfa006c2b0 0x7fcfa006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:26.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.917+0000 7fcfaeffd640 1 -- 172.21.15.2:0/4184326417 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fcfb808b1d0 con 0x7fcfc015b480 2024-09-19T05:45:26.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.917+0000 7fcfbe59c640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcfa006c2b0 0x7fcfa006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:26.921 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.918+0000 7fcfbe59c640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcfa006c2b0 0x7fcfa006e770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006fd0 tx=0x7fcfb0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:26.923 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:26.921+0000 7fcfaeffd640 1 -- 172.21.15.2:0/4184326417 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fcfb8059320 con 0x7fcfc015b480 2024-09-19T05:45:27.097 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.094+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcfc0154350 con 0x7fcfa006c2b0 2024-09-19T05:45:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.101+0000 7fcfaeffd640 1 -- 172.21.15.2:0/4184326417 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253668 (secure 0 0 0) 0x7fcfb0010990 con 0x7fcfa006c2b0 2024-09-19T05:45:27.106 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcfa006c2b0 msgr2=0x7fcfa006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcfa006c2b0 0x7fcfa006e770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006fd0 tx=0x7fcfb0008040 comp rx=0 tx=0).stop 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc015b480 msgr2=0x7fcfc00cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc015b480 0x7fcfc00cc500 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800cb70 tx=0x7fcfb800a430 comp rx=0 tx=0).stop 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 shutdown_connections 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fcfa006c2b0 0x7fcfa006e770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfc00c9660 0x7fcfc01f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc015b480 0x7fcfc00cc500 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 --2- 172.21.15.2:0/4184326417 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcfc006e9e0 0x7fcfc00ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 >> 172.21.15.2:0/4184326417 conn(0x7fcfc006c240 msgr2=0x7fcfc0155a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 shutdown_connections 2024-09-19T05:45:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.104+0000 7fcfc4ee6640 1 -- 172.21.15.2:0/4184326417 wait complete. 2024-09-19T05:45:27.117 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-19T05:45:25.855290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560741,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13281,"num_write_kb":25179740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14765,"ondisk_log_size":14765,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342284,"kb_used_data":21952,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632436,"statfs":{"total":773094113280,"available":772743614464,"internally_reserved":0,"allocated":22478848,"data_stored":21591756,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.183313"},"pg_stats":[{"pgid":"3.3f","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687024+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:45:21.687024+0000","last_peered":"2024-09-19T05:45:21.687024+0000","last_clean":"2024-09-19T05:45:21.687024+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:21.687024+0000","last_undegraded":"2024-09-19T05:45:21.687024+0000","last_fullsized":"2024-09-19T05:45:21.687024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'148","reported_seq":301,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688407+0000","last_change":"2024-09-19T05:44:54.646789+0000","last_active":"2024-09-19T05:45:21.688407+0000","last_peered":"2024-09-19T05:45:21.688407+0000","last_clean":"2024-09-19T05:45:21.688407+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:21.688407+0000","last_undegraded":"2024-09-19T05:45:21.688407+0000","last_fullsized":"2024-09-19T05:45:21.688407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:11.761535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"84'171","reported_seq":325,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688172+0000","last_change":"2024-09-19T05:44:53.475804+0000","last_active":"2024-09-19T05:45:21.688172+0000","last_peered":"2024-09-19T05:45:21.688172+0000","last_clean":"2024-09-19T05:45:21.688172+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:21.688172+0000","last_undegraded":"2024-09-19T05:45:21.688172+0000","last_fullsized":"2024-09-19T05:45:21.688172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:57.009039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"84'185","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935211+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:45:21.935211+0000","last_peered":"2024-09-19T05:45:21.935211+0000","last_clean":"2024-09-19T05:45:21.935211+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:21.935211+0000","last_undegraded":"2024-09-19T05:45:21.935211+0000","last_fullsized":"2024-09-19T05:45:21.935211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"84'170","reported_seq":438,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687949+0000","last_change":"2024-09-19T05:44:53.953498+0000","last_active":"2024-09-19T05:45:21.687949+0000","last_peered":"2024-09-19T05:45:21.687949+0000","last_clean":"2024-09-19T05:45:21.687949+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:21.687949+0000","last_undegraded":"2024-09-19T05:45:21.687949+0000","last_fullsized":"2024-09-19T05:45:21.687949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:39.367204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"84'156","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935150+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:45:21.935150+0000","last_peered":"2024-09-19T05:45:21.935150+0000","last_clean":"2024-09-19T05:45:21.935150+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:21.935150+0000","last_undegraded":"2024-09-19T05:45:21.935150+0000","last_fullsized":"2024-09-19T05:45:21.935150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"84'183","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686741+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:45:21.686741+0000","last_peered":"2024-09-19T05:45:21.686741+0000","last_clean":"2024-09-19T05:45:21.686741+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:21.686741+0000","last_undegraded":"2024-09-19T05:45:21.686741+0000","last_fullsized":"2024-09-19T05:45:21.686741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"84'195","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687356+0000","last_change":"2024-09-19T05:44:56.567156+0000","last_active":"2024-09-19T05:45:21.687356+0000","last_peered":"2024-09-19T05:45:21.687356+0000","last_clean":"2024-09-19T05:45:21.687356+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:21.687356+0000","last_undegraded":"2024-09-19T05:45:21.687356+0000","last_fullsized":"2024-09-19T05:45:21.687356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:17.811359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"84'167","reported_seq":280,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190149+0000","last_change":"2024-09-19T05:45:10.627715+0000","last_active":"2024-09-19T05:45:22.190149+0000","last_peered":"2024-09-19T05:45:22.190149+0000","last_clean":"2024-09-19T05:45:22.190149+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:22.190149+0000","last_undegraded":"2024-09-19T05:45:22.190149+0000","last_fullsized":"2024-09-19T05:45:22.190149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:38.174880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686962+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:45:21.686962+0000","last_peered":"2024-09-19T05:45:21.686962+0000","last_clean":"2024-09-19T05:45:21.686962+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:21.686962+0000","last_undegraded":"2024-09-19T05:45:21.686962+0000","last_fullsized":"2024-09-19T05:45:21.686962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"84'208","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686760+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:45:21.686760+0000","last_peered":"2024-09-19T05:45:21.686760+0000","last_clean":"2024-09-19T05:45:21.686760+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:21.686760+0000","last_undegraded":"2024-09-19T05:45:21.686760+0000","last_fullsized":"2024-09-19T05:45:21.686760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"84'159","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688440+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:45:21.688440+0000","last_peered":"2024-09-19T05:45:21.688440+0000","last_clean":"2024-09-19T05:45:21.688440+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:21.688440+0000","last_undegraded":"2024-09-19T05:45:21.688440+0000","last_fullsized":"2024-09-19T05:45:21.688440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:54.111389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'167","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773293+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:45:21.773293+0000","last_peered":"2024-09-19T05:45:21.773293+0000","last_clean":"2024-09-19T05:45:21.773293+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:21.773293+0000","last_undegraded":"2024-09-19T05:45:21.773293+0000","last_fullsized":"2024-09-19T05:45:21.773293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773059+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:45:21.773059+0000","last_peered":"2024-09-19T05:45:21.773059+0000","last_clean":"2024-09-19T05:45:21.773059+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:21.773059+0000","last_undegraded":"2024-09-19T05:45:21.773059+0000","last_fullsized":"2024-09-19T05:45:21.773059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"84'170","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687615+0000","last_change":"2024-09-19T05:44:53.930699+0000","last_active":"2024-09-19T05:45:21.687615+0000","last_peered":"2024-09-19T05:45:21.687615+0000","last_clean":"2024-09-19T05:45:21.687615+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:21.687615+0000","last_undegraded":"2024-09-19T05:45:21.687615+0000","last_fullsized":"2024-09-19T05:45:21.687615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:02.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"84'153","reported_seq":324,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716095+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:45:21.716095+0000","last_peered":"2024-09-19T05:45:21.716095+0000","last_clean":"2024-09-19T05:45:21.716095+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:21.716095+0000","last_undegraded":"2024-09-19T05:45:21.716095+0000","last_fullsized":"2024-09-19T05:45:21.716095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"84'168","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715985+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:45:21.715985+0000","last_peered":"2024-09-19T05:45:21.715985+0000","last_clean":"2024-09-19T05:45:21.715985+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:21.715985+0000","last_undegraded":"2024-09-19T05:45:21.715985+0000","last_fullsized":"2024-09-19T05:45:21.715985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"84'169","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687671+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:45:21.687671+0000","last_peered":"2024-09-19T05:45:21.687671+0000","last_clean":"2024-09-19T05:45:21.687671+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:21.687671+0000","last_undegraded":"2024-09-19T05:45:21.687671+0000","last_fullsized":"2024-09-19T05:45:21.687671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"84'198","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716789+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:45:21.716789+0000","last_peered":"2024-09-19T05:45:21.716789+0000","last_clean":"2024-09-19T05:45:21.716789+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:21.716789+0000","last_undegraded":"2024-09-19T05:45:21.716789+0000","last_fullsized":"2024-09-19T05:45:21.716789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"84'171","reported_seq":353,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688653+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:45:21.688653+0000","last_peered":"2024-09-19T05:45:21.688653+0000","last_clean":"2024-09-19T05:45:21.688653+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:21.688653+0000","last_undegraded":"2024-09-19T05:45:21.688653+0000","last_fullsized":"2024-09-19T05:45:21.688653+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:17.766409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"84'143","reported_seq":315,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708984+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:45:21.708984+0000","last_peered":"2024-09-19T05:45:21.708984+0000","last_clean":"2024-09-19T05:45:21.708984+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:21.708984+0000","last_undegraded":"2024-09-19T05:45:21.708984+0000","last_fullsized":"2024-09-19T05:45:21.708984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"84'189","reported_seq":246,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687810+0000","last_change":"2024-09-19T05:44:55.559370+0000","last_active":"2024-09-19T05:45:21.687810+0000","last_peered":"2024-09-19T05:45:21.687810+0000","last_clean":"2024-09-19T05:45:21.687810+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:21.687810+0000","last_undegraded":"2024-09-19T05:45:21.687810+0000","last_fullsized":"2024-09-19T05:45:21.687810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:25.838393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'142","reported_seq":295,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687168+0000","last_change":"2024-09-19T05:44:56.134769+0000","last_active":"2024-09-19T05:45:21.687168+0000","last_peered":"2024-09-19T05:45:21.687168+0000","last_clean":"2024-09-19T05:45:21.687168+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:21.687168+0000","last_undegraded":"2024-09-19T05:45:21.687168+0000","last_fullsized":"2024-09-19T05:45:21.687168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:06.255089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"84'201","reported_seq":492,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716107+0000","last_change":"2024-09-19T05:44:56.565770+0000","last_active":"2024-09-19T05:45:21.716107+0000","last_peered":"2024-09-19T05:45:21.716107+0000","last_clean":"2024-09-19T05:45:21.716107+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:21.716107+0000","last_undegraded":"2024-09-19T05:45:21.716107+0000","last_fullsized":"2024-09-19T05:45:21.716107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:03.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"84'158","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688447+0000","last_change":"2024-09-19T05:44:56.582314+0000","last_active":"2024-09-19T05:45:21.688447+0000","last_peered":"2024-09-19T05:45:21.688447+0000","last_clean":"2024-09-19T05:45:21.688447+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:21.688447+0000","last_undegraded":"2024-09-19T05:45:21.688447+0000","last_fullsized":"2024-09-19T05:45:21.688447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:41.223871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"84'202","reported_seq":374,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935387+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:45:21.935387+0000","last_peered":"2024-09-19T05:45:21.935387+0000","last_clean":"2024-09-19T05:45:21.935387+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:21.935387+0000","last_undegraded":"2024-09-19T05:45:21.935387+0000","last_fullsized":"2024-09-19T05:45:21.935387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773382+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:45:21.773382+0000","last_peered":"2024-09-19T05:45:21.773382+0000","last_clean":"2024-09-19T05:45:21.773382+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:21.773382+0000","last_undegraded":"2024-09-19T05:45:21.773382+0000","last_fullsized":"2024-09-19T05:45:21.773382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"84'148","reported_seq":320,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773260+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:45:21.773260+0000","last_peered":"2024-09-19T05:45:21.773260+0000","last_clean":"2024-09-19T05:45:21.773260+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:21.773260+0000","last_undegraded":"2024-09-19T05:45:21.773260+0000","last_fullsized":"2024-09-19T05:45:21.773260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'127","reported_seq":281,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688046+0000","last_change":"2024-09-19T05:44:54.977580+0000","last_active":"2024-09-19T05:45:21.688046+0000","last_peered":"2024-09-19T05:45:21.688046+0000","last_clean":"2024-09-19T05:45:21.688046+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:21.688046+0000","last_undegraded":"2024-09-19T05:45:21.688046+0000","last_fullsized":"2024-09-19T05:45:21.688046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:12.165530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"84'145","reported_seq":408,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716323+0000","last_change":"2024-09-19T05:44:53.510921+0000","last_active":"2024-09-19T05:45:21.716323+0000","last_peered":"2024-09-19T05:45:21.716323+0000","last_clean":"2024-09-19T05:45:21.716323+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:21.716323+0000","last_undegraded":"2024-09-19T05:45:21.716323+0000","last_fullsized":"2024-09-19T05:45:21.716323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:39.530204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"84'146","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687436+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:45:21.687436+0000","last_peered":"2024-09-19T05:45:21.687436+0000","last_clean":"2024-09-19T05:45:21.687436+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:21.687436+0000","last_undegraded":"2024-09-19T05:45:21.687436+0000","last_fullsized":"2024-09-19T05:45:21.687436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'171","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935092+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:45:21.935092+0000","last_peered":"2024-09-19T05:45:21.935092+0000","last_clean":"2024-09-19T05:45:21.935092+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:21.935092+0000","last_undegraded":"2024-09-19T05:45:21.935092+0000","last_fullsized":"2024-09-19T05:45:21.935092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"84'142","reported_seq":240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189849+0000","last_change":"2024-09-19T05:45:10.624968+0000","last_active":"2024-09-19T05:45:22.189849+0000","last_peered":"2024-09-19T05:45:22.189849+0000","last_clean":"2024-09-19T05:45:22.189849+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:22.189849+0000","last_undegraded":"2024-09-19T05:45:22.189849+0000","last_fullsized":"2024-09-19T05:45:22.189849+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:59.539763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"94'1809","reported_seq":1885,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687685+0000","last_change":"2024-09-19T05:44:55.560257+0000","last_active":"2024-09-19T05:45:21.687685+0000","last_peered":"2024-09-19T05:45:21.687685+0000","last_clean":"2024-09-19T05:45:21.687685+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:21.687685+0000","last_undegraded":"2024-09-19T05:45:21.687685+0000","last_fullsized":"2024-09-19T05:45:21.687685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1809,"log_dups_size":0,"ondisk_log_size":1809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:25.647547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'154","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708831+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:45:21.708831+0000","last_peered":"2024-09-19T05:45:21.708831+0000","last_clean":"2024-09-19T05:45:21.708831+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:21.708831+0000","last_undegraded":"2024-09-19T05:45:21.708831+0000","last_fullsized":"2024-09-19T05:45:21.708831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716460+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:45:21.716460+0000","last_peered":"2024-09-19T05:45:21.716460+0000","last_clean":"2024-09-19T05:45:21.716460+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:21.716460+0000","last_undegraded":"2024-09-19T05:45:21.716460+0000","last_fullsized":"2024-09-19T05:45:21.716460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773054+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:45:21.773054+0000","last_peered":"2024-09-19T05:45:21.773054+0000","last_clean":"2024-09-19T05:45:21.773054+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:21.773054+0000","last_undegraded":"2024-09-19T05:45:21.773054+0000","last_fullsized":"2024-09-19T05:45:21.773054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'182","reported_seq":337,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687351+0000","last_change":"2024-09-19T05:44:59.989289+0000","last_active":"2024-09-19T05:45:21.687351+0000","last_peered":"2024-09-19T05:45:21.687351+0000","last_clean":"2024-09-19T05:45:21.687351+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:21.687351+0000","last_undegraded":"2024-09-19T05:45:21.687351+0000","last_fullsized":"2024-09-19T05:45:21.687351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:52.052457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716243+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:45:21.716243+0000","last_peered":"2024-09-19T05:45:21.716243+0000","last_clean":"2024-09-19T05:45:21.716243+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:21.716243+0000","last_undegraded":"2024-09-19T05:45:21.716243+0000","last_fullsized":"2024-09-19T05:45:21.716243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"84'177","reported_seq":359,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708983+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:45:21.708983+0000","last_peered":"2024-09-19T05:45:21.708983+0000","last_clean":"2024-09-19T05:45:21.708983+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:21.708983+0000","last_undegraded":"2024-09-19T05:45:21.708983+0000","last_fullsized":"2024-09-19T05:45:21.708983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190091+0000","last_change":"2024-09-19T05:45:10.622931+0000","last_active":"2024-09-19T05:45:22.190091+0000","last_peered":"2024-09-19T05:45:22.190091+0000","last_clean":"2024-09-19T05:45:22.190091+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:22.190091+0000","last_undegraded":"2024-09-19T05:45:22.190091+0000","last_fullsized":"2024-09-19T05:45:22.190091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:34:06.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"84'241","reported_seq":413,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935402+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:45:21.935402+0000","last_peered":"2024-09-19T05:45:21.935402+0000","last_clean":"2024-09-19T05:45:21.935402+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:21.935402+0000","last_undegraded":"2024-09-19T05:45:21.935402+0000","last_fullsized":"2024-09-19T05:45:21.935402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688495+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:45:21.688495+0000","last_peered":"2024-09-19T05:45:21.688495+0000","last_clean":"2024-09-19T05:45:21.688495+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:21.688495+0000","last_undegraded":"2024-09-19T05:45:21.688495+0000","last_fullsized":"2024-09-19T05:45:21.688495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:39.063921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"84'151","reported_seq":300,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688324+0000","last_change":"2024-09-19T05:44:59.517543+0000","last_active":"2024-09-19T05:45:21.688324+0000","last_peered":"2024-09-19T05:45:21.688324+0000","last_clean":"2024-09-19T05:45:21.688324+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:21.688324+0000","last_undegraded":"2024-09-19T05:45:21.688324+0000","last_fullsized":"2024-09-19T05:45:21.688324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.453343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"89'10","reported_seq":185,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687787+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:45:21.687787+0000","last_peered":"2024-09-19T05:45:21.687787+0000","last_clean":"2024-09-19T05:45:21.687787+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:21.687787+0000","last_undegraded":"2024-09-19T05:45:21.687787+0000","last_fullsized":"2024-09-19T05:45:21.687787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"84'157","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716300+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:45:21.716300+0000","last_peered":"2024-09-19T05:45:21.716300+0000","last_clean":"2024-09-19T05:45:21.716300+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:21.716300+0000","last_undegraded":"2024-09-19T05:45:21.716300+0000","last_fullsized":"2024-09-19T05:45:21.716300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"94'2017","reported_seq":2191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935286+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:45:21.935286+0000","last_peered":"2024-09-19T05:45:21.935286+0000","last_clean":"2024-09-19T05:45:21.935286+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:21.935286+0000","last_undegraded":"2024-09-19T05:45:21.935286+0000","last_fullsized":"2024-09-19T05:45:21.935286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688257+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:45:21.688257+0000","last_peered":"2024-09-19T05:45:21.688257+0000","last_clean":"2024-09-19T05:45:21.688257+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:21.688257+0000","last_undegraded":"2024-09-19T05:45:21.688257+0000","last_fullsized":"2024-09-19T05:45:21.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687838+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:45:21.687838+0000","last_peered":"2024-09-19T05:45:21.687838+0000","last_clean":"2024-09-19T05:45:21.687838+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:21.687838+0000","last_undegraded":"2024-09-19T05:45:21.687838+0000","last_fullsized":"2024-09-19T05:45:21.687838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"84'168","reported_seq":340,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773221+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:45:21.773221+0000","last_peered":"2024-09-19T05:45:21.773221+0000","last_clean":"2024-09-19T05:45:21.773221+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:21.773221+0000","last_undegraded":"2024-09-19T05:45:21.773221+0000","last_fullsized":"2024-09-19T05:45:21.773221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716531+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:45:21.716531+0000","last_peered":"2024-09-19T05:45:21.716531+0000","last_clean":"2024-09-19T05:45:21.716531+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:21.716531+0000","last_undegraded":"2024-09-19T05:45:21.716531+0000","last_fullsized":"2024-09-19T05:45:21.716531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"84'186","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715983+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:45:21.715983+0000","last_peered":"2024-09-19T05:45:21.715983+0000","last_clean":"2024-09-19T05:45:21.715983+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:21.715983+0000","last_undegraded":"2024-09-19T05:45:21.715983+0000","last_fullsized":"2024-09-19T05:45:21.715983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687854+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:45:21.687854+0000","last_peered":"2024-09-19T05:45:21.687854+0000","last_clean":"2024-09-19T05:45:21.687854+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:21.687854+0000","last_undegraded":"2024-09-19T05:45:21.687854+0000","last_fullsized":"2024-09-19T05:45:21.687854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"84'191","reported_seq":283,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189767+0000","last_change":"2024-09-19T05:45:10.627674+0000","last_active":"2024-09-19T05:45:22.189767+0000","last_peered":"2024-09-19T05:45:22.189767+0000","last_clean":"2024-09-19T05:45:22.189767+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:22.189767+0000","last_undegraded":"2024-09-19T05:45:22.189767+0000","last_fullsized":"2024-09-19T05:45:22.189767+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:21.685895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"89'9","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708963+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:45:21.708963+0000","last_peered":"2024-09-19T05:45:21.708963+0000","last_clean":"2024-09-19T05:45:21.708963+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:21.708963+0000","last_undegraded":"2024-09-19T05:45:21.708963+0000","last_fullsized":"2024-09-19T05:45:21.708963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"84'209","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716252+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:45:21.716252+0000","last_peered":"2024-09-19T05:45:21.716252+0000","last_clean":"2024-09-19T05:45:21.716252+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:21.716252+0000","last_undegraded":"2024-09-19T05:45:21.716252+0000","last_fullsized":"2024-09-19T05:45:21.716252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688782+0000","last_change":"2024-09-19T05:44:53.065559+0000","last_active":"2024-09-19T05:45:21.688782+0000","last_peered":"2024-09-19T05:45:21.688782+0000","last_clean":"2024-09-19T05:45:21.688782+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:21.688782+0000","last_undegraded":"2024-09-19T05:45:21.688782+0000","last_fullsized":"2024-09-19T05:45:21.688782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:13.889455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935000+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:45:21.935000+0000","last_peered":"2024-09-19T05:45:21.935000+0000","last_clean":"2024-09-19T05:45:21.935000+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:21.935000+0000","last_undegraded":"2024-09-19T05:45:21.935000+0000","last_fullsized":"2024-09-19T05:45:21.935000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709231+0000","last_change":"2024-09-19T05:44:53.059769+0000","last_active":"2024-09-19T05:45:21.709231+0000","last_peered":"2024-09-19T05:45:21.709231+0000","last_clean":"2024-09-19T05:45:21.709231+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:21.709231+0000","last_undegraded":"2024-09-19T05:45:21.709231+0000","last_fullsized":"2024-09-19T05:45:21.709231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:03.909805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"84'153","reported_seq":231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687781+0000","last_change":"2024-09-19T05:44:15.787415+0000","last_active":"2024-09-19T05:45:21.687781+0000","last_peered":"2024-09-19T05:45:21.687781+0000","last_clean":"2024-09-19T05:45:21.687781+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:21.687781+0000","last_undegraded":"2024-09-19T05:45:21.687781+0000","last_fullsized":"2024-09-19T05:45:21.687781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:10.095369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":125,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688186+0000","last_change":"2024-09-19T05:44:53.058979+0000","last_active":"2024-09-19T05:45:21.688186+0000","last_peered":"2024-09-19T05:45:21.688186+0000","last_clean":"2024-09-19T05:45:21.688186+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:21.688186+0000","last_undegraded":"2024-09-19T05:45:21.688186+0000","last_fullsized":"2024-09-19T05:45:21.688186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:04.530725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686740+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:45:21.686740+0000","last_peered":"2024-09-19T05:45:21.686740+0000","last_clean":"2024-09-19T05:45:21.686740+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:21.686740+0000","last_undegraded":"2024-09-19T05:45:21.686740+0000","last_fullsized":"2024-09-19T05:45:21.686740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773397+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:45:21.773397+0000","last_peered":"2024-09-19T05:45:21.773397+0000","last_clean":"2024-09-19T05:45:21.773397+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:21.773397+0000","last_undegraded":"2024-09-19T05:45:21.773397+0000","last_fullsized":"2024-09-19T05:45:21.773397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"84'168","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716718+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:45:21.716718+0000","last_peered":"2024-09-19T05:45:21.716718+0000","last_clean":"2024-09-19T05:45:21.716718+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:21.716718+0000","last_undegraded":"2024-09-19T05:45:21.716718+0000","last_fullsized":"2024-09-19T05:45:21.716718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189880+0000","last_change":"2024-09-19T05:45:10.627237+0000","last_active":"2024-09-19T05:45:22.189880+0000","last_peered":"2024-09-19T05:45:22.189880+0000","last_clean":"2024-09-19T05:45:22.189880+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:22.189880+0000","last_undegraded":"2024-09-19T05:45:22.189880+0000","last_fullsized":"2024-09-19T05:45:22.189880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:04.886546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716440+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:45:21.716440+0000","last_peered":"2024-09-19T05:45:21.716440+0000","last_clean":"2024-09-19T05:45:21.716440+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:21.716440+0000","last_undegraded":"2024-09-19T05:45:21.716440+0000","last_fullsized":"2024-09-19T05:45:21.716440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708974+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:45:21.708974+0000","last_peered":"2024-09-19T05:45:21.708974+0000","last_clean":"2024-09-19T05:45:21.708974+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:21.708974+0000","last_undegraded":"2024-09-19T05:45:21.708974+0000","last_fullsized":"2024-09-19T05:45:21.708974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716388+0000","last_change":"2024-09-19T05:44:53.069283+0000","last_active":"2024-09-19T05:45:21.716388+0000","last_peered":"2024-09-19T05:45:21.716388+0000","last_clean":"2024-09-19T05:45:21.716388+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:21.716388+0000","last_undegraded":"2024-09-19T05:45:21.716388+0000","last_fullsized":"2024-09-19T05:45:21.716388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:51.209720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773046+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:45:21.773046+0000","last_peered":"2024-09-19T05:45:21.773046+0000","last_clean":"2024-09-19T05:45:21.773046+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:21.773046+0000","last_undegraded":"2024-09-19T05:45:21.773046+0000","last_fullsized":"2024-09-19T05:45:21.773046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687539+0000","last_change":"2024-09-19T05:44:53.068415+0000","last_active":"2024-09-19T05:45:21.687539+0000","last_peered":"2024-09-19T05:45:21.687539+0000","last_clean":"2024-09-19T05:45:21.687539+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:21.687539+0000","last_undegraded":"2024-09-19T05:45:21.687539+0000","last_fullsized":"2024-09-19T05:45:21.687539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:16.609871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"84'157","reported_seq":310,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687083+0000","last_change":"2024-09-19T05:44:58.820818+0000","last_active":"2024-09-19T05:45:21.687083+0000","last_peered":"2024-09-19T05:45:21.687083+0000","last_clean":"2024-09-19T05:45:21.687083+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:21.687083+0000","last_undegraded":"2024-09-19T05:45:21.687083+0000","last_fullsized":"2024-09-19T05:45:21.687083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:32.427214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716567+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:45:21.716567+0000","last_peered":"2024-09-19T05:45:21.716567+0000","last_clean":"2024-09-19T05:45:21.716567+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:21.716567+0000","last_undegraded":"2024-09-19T05:45:21.716567+0000","last_fullsized":"2024-09-19T05:45:21.716567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686759+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:45:21.686759+0000","last_peered":"2024-09-19T05:45:21.686759+0000","last_clean":"2024-09-19T05:45:21.686759+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:21.686759+0000","last_undegraded":"2024-09-19T05:45:21.686759+0000","last_fullsized":"2024-09-19T05:45:21.686759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687970+0000","last_change":"2024-09-19T05:44:53.063483+0000","last_active":"2024-09-19T05:45:21.687970+0000","last_peered":"2024-09-19T05:45:21.687970+0000","last_clean":"2024-09-19T05:45:21.687970+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:21.687970+0000","last_undegraded":"2024-09-19T05:45:21.687970+0000","last_fullsized":"2024-09-19T05:45:21.687970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:40.306784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"84'155","reported_seq":327,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935176+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:45:21.935176+0000","last_peered":"2024-09-19T05:45:21.935176+0000","last_clean":"2024-09-19T05:45:21.935176+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:21.935176+0000","last_undegraded":"2024-09-19T05:45:21.935176+0000","last_fullsized":"2024-09-19T05:45:21.935176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716905+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:45:21.716905+0000","last_peered":"2024-09-19T05:45:21.716905+0000","last_clean":"2024-09-19T05:45:21.716905+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:21.716905+0000","last_undegraded":"2024-09-19T05:45:21.716905+0000","last_fullsized":"2024-09-19T05:45:21.716905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'153","reported_seq":424,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773336+0000","last_change":"2024-09-19T05:44:55.675882+0000","last_active":"2024-09-19T05:45:21.773336+0000","last_peered":"2024-09-19T05:45:21.773336+0000","last_clean":"2024-09-19T05:45:21.773336+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:21.773336+0000","last_undegraded":"2024-09-19T05:45:21.773336+0000","last_fullsized":"2024-09-19T05:45:21.773336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:04.593886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709321+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:45:21.709321+0000","last_peered":"2024-09-19T05:45:21.709321+0000","last_clean":"2024-09-19T05:45:21.709321+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:21.709321+0000","last_undegraded":"2024-09-19T05:45:21.709321+0000","last_fullsized":"2024-09-19T05:45:21.709321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"84'161","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709277+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:45:21.709277+0000","last_peered":"2024-09-19T05:45:21.709277+0000","last_clean":"2024-09-19T05:45:21.709277+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:21.709277+0000","last_undegraded":"2024-09-19T05:45:21.709277+0000","last_fullsized":"2024-09-19T05:45:21.709277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"85'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716921+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:45:21.716921+0000","last_peered":"2024-09-19T05:45:21.716921+0000","last_clean":"2024-09-19T05:45:21.716921+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:21.716921+0000","last_undegraded":"2024-09-19T05:45:21.716921+0000","last_fullsized":"2024-09-19T05:45:21.716921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"84'176","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709220+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:45:21.709220+0000","last_peered":"2024-09-19T05:45:21.709220+0000","last_clean":"2024-09-19T05:45:21.709220+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:21.709220+0000","last_undegraded":"2024-09-19T05:45:21.709220+0000","last_fullsized":"2024-09-19T05:45:21.709220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687243+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:45:21.687243+0000","last_peered":"2024-09-19T05:45:21.687243+0000","last_clean":"2024-09-19T05:45:21.687243+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:21.687243+0000","last_undegraded":"2024-09-19T05:45:21.687243+0000","last_fullsized":"2024-09-19T05:45:21.687243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"84'160","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716327+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:45:21.716327+0000","last_peered":"2024-09-19T05:45:21.716327+0000","last_clean":"2024-09-19T05:45:21.716327+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:21.716327+0000","last_undegraded":"2024-09-19T05:45:21.716327+0000","last_fullsized":"2024-09-19T05:45:21.716327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709021+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:45:21.709021+0000","last_peered":"2024-09-19T05:45:21.709021+0000","last_clean":"2024-09-19T05:45:21.709021+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:21.709021+0000","last_undegraded":"2024-09-19T05:45:21.709021+0000","last_fullsized":"2024-09-19T05:45:21.709021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708902+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:45:21.708902+0000","last_peered":"2024-09-19T05:45:21.708902+0000","last_clean":"2024-09-19T05:45:21.708902+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:21.708902+0000","last_undegraded":"2024-09-19T05:45:21.708902+0000","last_fullsized":"2024-09-19T05:45:21.708902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773238+0000","last_change":"2024-09-19T05:44:53.065594+0000","last_active":"2024-09-19T05:45:21.773238+0000","last_peered":"2024-09-19T05:45:21.773238+0000","last_clean":"2024-09-19T05:45:21.773238+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:21.773238+0000","last_undegraded":"2024-09-19T05:45:21.773238+0000","last_fullsized":"2024-09-19T05:45:21.773238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:47.254673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"84'156","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687718+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:45:21.687718+0000","last_peered":"2024-09-19T05:45:21.687718+0000","last_clean":"2024-09-19T05:45:21.687718+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:21.687718+0000","last_undegraded":"2024-09-19T05:45:21.687718+0000","last_fullsized":"2024-09-19T05:45:21.687718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"59'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716236+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:45:21.716236+0000","last_peered":"2024-09-19T05:45:21.716236+0000","last_clean":"2024-09-19T05:45:21.716236+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:21.716236+0000","last_undegraded":"2024-09-19T05:45:21.716236+0000","last_fullsized":"2024-09-19T05:45:21.716236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"84'149","reported_seq":330,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716425+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:45:21.716425+0000","last_peered":"2024-09-19T05:45:21.716425+0000","last_clean":"2024-09-19T05:45:21.716425+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:21.716425+0000","last_undegraded":"2024-09-19T05:45:21.716425+0000","last_fullsized":"2024-09-19T05:45:21.716425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"84'190","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715986+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:45:21.715986+0000","last_peered":"2024-09-19T05:45:21.715986+0000","last_clean":"2024-09-19T05:45:21.715986+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:21.715986+0000","last_undegraded":"2024-09-19T05:45:21.715986+0000","last_fullsized":"2024-09-19T05:45:21.715986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":194,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688923+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:45:21.688923+0000","last_peered":"2024-09-19T05:45:21.688923+0000","last_clean":"2024-09-19T05:45:21.688923+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:21.688923+0000","last_undegraded":"2024-09-19T05:45:21.688923+0000","last_fullsized":"2024-09-19T05:45:21.688923+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:59.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"84'153","reported_seq":266,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189956+0000","last_change":"2024-09-19T05:45:10.627058+0000","last_active":"2024-09-19T05:45:22.189956+0000","last_peered":"2024-09-19T05:45:22.189956+0000","last_clean":"2024-09-19T05:45:22.189956+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:22.189956+0000","last_undegraded":"2024-09-19T05:45:22.189956+0000","last_fullsized":"2024-09-19T05:45:22.189956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:52.373727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"70'4","reported_seq":196,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688300+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:45:21.688300+0000","last_peered":"2024-09-19T05:45:21.688300+0000","last_clean":"2024-09-19T05:45:21.688300+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:21.688300+0000","last_undegraded":"2024-09-19T05:45:21.688300+0000","last_fullsized":"2024-09-19T05:45:21.688300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:52.607339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"84'154","reported_seq":314,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688325+0000","last_change":"2024-09-19T05:44:57.957776+0000","last_active":"2024-09-19T05:45:21.688325+0000","last_peered":"2024-09-19T05:45:21.688325+0000","last_clean":"2024-09-19T05:45:21.688325+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:21.688325+0000","last_undegraded":"2024-09-19T05:45:21.688325+0000","last_fullsized":"2024-09-19T05:45:21.688325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:37.587819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":156,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688211+0000","last_change":"2024-09-19T05:44:53.061701+0000","last_active":"2024-09-19T05:45:21.688211+0000","last_peered":"2024-09-19T05:45:21.688211+0000","last_clean":"2024-09-19T05:45:21.688211+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:21.688211+0000","last_undegraded":"2024-09-19T05:45:21.688211+0000","last_fullsized":"2024-09-19T05:45:21.688211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:26.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"84'176","reported_seq":321,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688312+0000","last_change":"2024-09-19T05:44:57.503018+0000","last_active":"2024-09-19T05:45:21.688312+0000","last_peered":"2024-09-19T05:45:21.688312+0000","last_clean":"2024-09-19T05:45:21.688312+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:21.688312+0000","last_undegraded":"2024-09-19T05:45:21.688312+0000","last_fullsized":"2024-09-19T05:45:21.688312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:44.482643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709387+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:45:21.709387+0000","last_peered":"2024-09-19T05:45:21.709387+0000","last_clean":"2024-09-19T05:45:21.709387+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:21.709387+0000","last_undegraded":"2024-09-19T05:45:21.709387+0000","last_fullsized":"2024-09-19T05:45:21.709387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970373,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3906,"num_write_kb":11500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15940746,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3893,"ondisk_log_size":3893,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":5272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":5398528,"data_stored":4749255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":4984832,"data_stored":4334380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449096,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46800,"kb_used_data":1724,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325040,"statfs":{"total":96636764160,"available":96588840960,"internally_reserved":0,"allocated":1765376,"data_stored":4919338,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50368,"kb_used_data":5292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321472,"statfs":{"total":96636764160,"available":96585187328,"internally_reserved":0,"allocated":5419008,"data_stored":4755934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40696,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331144,"statfs":{"total":96636764160,"available":96595091456,"internally_reserved":0,"allocated":1212416,"data_stored":560868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40748,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331092,"statfs":{"total":96636764160,"available":96595038208,"internally_reserved":0,"allocated":1261568,"data_stored":1150724,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40700,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331140,"statfs":{"total":96636764160,"available":96595087360,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774691,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:45:27.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:45:27.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.259+0000 7fadc36f1640 1 Processor -- start 2024-09-19T05:45:27.261 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.259+0000 7fadc36f1640 1 -- start start 2024-09-19T05:45:27.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc36f1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc0c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc36f1640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fadbc0c6c70 0x7fadbc06f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc36f1640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadbc0c7390 con 0x7fadbc06fa00 2024-09-19T05:45:27.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc36f1640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fadbc0c3770 con 0x7fadbc0c6c70 2024-09-19T05:45:27.263 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc36f1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc0c3a30 con 0x7fadbc14e640 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc0c8c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fadbc0c6c70 0x7fadbc06f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc1c8e640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fadbc06fa00 0x7fadbc0c8160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54164/0 (socket says 172.21.15.2:54164) 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc1c8e640 1 -- 172.21.15.2:0/1661162239 learned_addr learned my addr 172.21.15.2:0/1661162239 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc148d640 1 --2- 172.21.15.2:0/1661162239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc0c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc0c8c640 1 -- 172.21.15.2:0/1661162239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 msgr2=0x7fadbc0c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc0c8c640 1 --2- 172.21.15.2:0/1661162239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc0c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc0c8c640 1 -- 172.21.15.2:0/1661162239 >> v1:172.21.15.2:6789/0 conn(0x7fadbc06fa00 legacy=0x7fadbc0c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc0c8c640 1 -- 172.21.15.2:0/1661162239 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc0c7630 con 0x7fadbc0c6c70 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadb3fff640 1 -- 172.21.15.2:0/1661162239 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 859358033 0 0) 0x7fadbc0c7390 con 0x7fadbc06fa00 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.260+0000 7fadc148d640 1 --2- 172.21.15.2:0/1661162239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc0c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:27.264 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.261+0000 7fadc0c8c640 1 --2- 172.21.15.2:0/1661162239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc06f4c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ee57ad7bc38415 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.261+0000 7fadb3fff640 1 -- 172.21.15.2:0/1661162239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac037070 con 0x7fadbc0c6c70 2024-09-19T05:45:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.261+0000 7fadb3fff640 1 -- 172.21.15.2:0/1661162239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac032440 con 0x7fadbc0c6c70 2024-09-19T05:45:27.265 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.261+0000 7fadb3fff640 1 -- 172.21.15.2:0/1661162239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac032810 con 0x7fadbc0c6c70 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 -- 172.21.15.2:0/1661162239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 msgr2=0x7fadbc06f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 --2- 172.21.15.2:0/1661162239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc06f4c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fadac0097a0 tx=0x7fadac02a210 comp rx=0 tx=0).stop 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 -- 172.21.15.2:0/1661162239 shutdown_connections 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 --2- 172.21.15.2:0/1661162239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc06f4c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 --2- 172.21.15.2:0/1661162239 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc0c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 -- 172.21.15.2:0/1661162239 >> 172.21.15.2:0/1661162239 conn(0x7fadbc0cc270 msgr2=0x7fadbc0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 -- 172.21.15.2:0/1661162239 shutdown_connections 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.262+0000 7fadc36f1640 1 -- 172.21.15.2:0/1661162239 wait complete. 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.263+0000 7fadc36f1640 1 Processor -- start 2024-09-19T05:45:27.268 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.263+0000 7fadc36f1640 1 -- start start 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc36f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc06fa00 0x7fadbc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc36f1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc36f1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc1e26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc36f1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0c3d10 con 0x7fadbc0c6c70 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc36f1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0c7bf0 con 0x7fadbc06fa00 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc36f1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc0c3e90 con 0x7fadbc14e640 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc147240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54864/0 (socket says 172.21.15.2:54864) 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 -- 172.21.15.2:0/4238993194 learned_addr learned my addr 172.21.15.2:0/4238993194 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc1c8e640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc1e26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.269 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 -- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 msgr2=0x7fadbc1e26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc148d640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc06fa00 0x7fadbc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc1e26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 -- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc06fa00 msgr2=0x7fadbc071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc06fa00 0x7fadbc071e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 -- 172.21.15.2:0/4238993194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc1e33f0 con 0x7fadbc0c6c70 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc1c8e640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc1e26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc148d640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc06fa00 0x7fadbc071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc0c8c640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc147240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fadac035040 tx=0x7fadac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadb1ffb640 1 -- 172.21.15.2:0/4238993194 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac037040 con 0x7fadbc0c6c70 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadb1ffb640 1 -- 172.21.15.2:0/4238993194 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac030770 con 0x7fadbc0c6c70 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc149990 con 0x7fadbc0c6c70 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.264+0000 7fadb1ffb640 1 -- 172.21.15.2:0/4238993194 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac030b80 con 0x7fadbc0c6c70 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.265+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc0c91a0 con 0x7fadbc0c6c70 2024-09-19T05:45:27.270 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.266+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadbc0cbe90 con 0x7fadbc0c6c70 2024-09-19T05:45:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.266+0000 7fadb1ffb640 1 -- 172.21.15.2:0/4238993194 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fadac044020 con 0x7fadbc0c6c70 2024-09-19T05:45:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.267+0000 7fadb1ffb640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fada006c2b0 0x7fada006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.271 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.267+0000 7fadb1ffb640 1 -- 172.21.15.2:0/4238993194 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fadac0ab9e0 con 0x7fadbc0c6c70 2024-09-19T05:45:27.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.267+0000 7fadc148d640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fada006c2b0 0x7fada006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.272 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.268+0000 7fadc148d640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fada006c2b0 0x7fada006e770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fadb8006fd0 tx=0x7fadb8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.273 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.271+0000 7fadb1ffb640 1 -- 172.21.15.2:0/4238993194 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fadac032e20 con 0x7fadbc0c6c70 2024-09-19T05:45:27.445 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.442+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadbc0cbb20 con 0x7fada006c2b0 2024-09-19T05:45:27.451 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.449+0000 7fadb1ffb640 1 -- 172.21.15.2:0/4238993194 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253668 (secure 0 0 0) 0x7fadbc0cbb20 con 0x7fada006c2b0 2024-09-19T05:45:27.454 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:27.455 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:45:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fada006c2b0 msgr2=0x7fada006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fada006c2b0 0x7fada006e770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fadb8006fd0 tx=0x7fadb8008040 comp rx=0 tx=0).stop 2024-09-19T05:45:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 msgr2=0x7fadbc147240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc147240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fadac035040 tx=0x7fadac0046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 shutdown_connections 2024-09-19T05:45:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fada006c2b0 0x7fada006e770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fadbc14e640 0x7fadbc1e26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadbc0c6c70 0x7fadbc147240 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 --2- 172.21.15.2:0/4238993194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadbc06fa00 0x7fadbc071e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 >> 172.21.15.2:0/4238993194 conn(0x7fadbc0cc270 msgr2=0x7fadbc0cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 shutdown_connections 2024-09-19T05:45:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.452+0000 7fadc36f1640 1 -- 172.21.15.2:0/4238993194 wait complete. 2024-09-19T05:45:27.465 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-19T05:45:25.855290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560741,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13281,"num_write_kb":25179740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14765,"ondisk_log_size":14765,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342284,"kb_used_data":21952,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632436,"statfs":{"total":773094113280,"available":772743614464,"internally_reserved":0,"allocated":22478848,"data_stored":21591756,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.183313"},"pg_stats":[{"pgid":"3.3f","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687024+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:45:21.687024+0000","last_peered":"2024-09-19T05:45:21.687024+0000","last_clean":"2024-09-19T05:45:21.687024+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:21.687024+0000","last_undegraded":"2024-09-19T05:45:21.687024+0000","last_fullsized":"2024-09-19T05:45:21.687024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'148","reported_seq":301,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688407+0000","last_change":"2024-09-19T05:44:54.646789+0000","last_active":"2024-09-19T05:45:21.688407+0000","last_peered":"2024-09-19T05:45:21.688407+0000","last_clean":"2024-09-19T05:45:21.688407+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:21.688407+0000","last_undegraded":"2024-09-19T05:45:21.688407+0000","last_fullsized":"2024-09-19T05:45:21.688407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:11.761535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"84'171","reported_seq":325,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688172+0000","last_change":"2024-09-19T05:44:53.475804+0000","last_active":"2024-09-19T05:45:21.688172+0000","last_peered":"2024-09-19T05:45:21.688172+0000","last_clean":"2024-09-19T05:45:21.688172+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:21.688172+0000","last_undegraded":"2024-09-19T05:45:21.688172+0000","last_fullsized":"2024-09-19T05:45:21.688172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:57.009039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"84'185","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935211+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:45:21.935211+0000","last_peered":"2024-09-19T05:45:21.935211+0000","last_clean":"2024-09-19T05:45:21.935211+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:21.935211+0000","last_undegraded":"2024-09-19T05:45:21.935211+0000","last_fullsized":"2024-09-19T05:45:21.935211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"84'170","reported_seq":438,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687949+0000","last_change":"2024-09-19T05:44:53.953498+0000","last_active":"2024-09-19T05:45:21.687949+0000","last_peered":"2024-09-19T05:45:21.687949+0000","last_clean":"2024-09-19T05:45:21.687949+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:21.687949+0000","last_undegraded":"2024-09-19T05:45:21.687949+0000","last_fullsized":"2024-09-19T05:45:21.687949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:39.367204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"84'156","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935150+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:45:21.935150+0000","last_peered":"2024-09-19T05:45:21.935150+0000","last_clean":"2024-09-19T05:45:21.935150+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:21.935150+0000","last_undegraded":"2024-09-19T05:45:21.935150+0000","last_fullsized":"2024-09-19T05:45:21.935150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"84'183","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686741+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:45:21.686741+0000","last_peered":"2024-09-19T05:45:21.686741+0000","last_clean":"2024-09-19T05:45:21.686741+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:21.686741+0000","last_undegraded":"2024-09-19T05:45:21.686741+0000","last_fullsized":"2024-09-19T05:45:21.686741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"84'195","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687356+0000","last_change":"2024-09-19T05:44:56.567156+0000","last_active":"2024-09-19T05:45:21.687356+0000","last_peered":"2024-09-19T05:45:21.687356+0000","last_clean":"2024-09-19T05:45:21.687356+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:21.687356+0000","last_undegraded":"2024-09-19T05:45:21.687356+0000","last_fullsized":"2024-09-19T05:45:21.687356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:17.811359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"84'167","reported_seq":280,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190149+0000","last_change":"2024-09-19T05:45:10.627715+0000","last_active":"2024-09-19T05:45:22.190149+0000","last_peered":"2024-09-19T05:45:22.190149+0000","last_clean":"2024-09-19T05:45:22.190149+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:22.190149+0000","last_undegraded":"2024-09-19T05:45:22.190149+0000","last_fullsized":"2024-09-19T05:45:22.190149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:38.174880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686962+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:45:21.686962+0000","last_peered":"2024-09-19T05:45:21.686962+0000","last_clean":"2024-09-19T05:45:21.686962+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:21.686962+0000","last_undegraded":"2024-09-19T05:45:21.686962+0000","last_fullsized":"2024-09-19T05:45:21.686962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"84'208","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686760+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:45:21.686760+0000","last_peered":"2024-09-19T05:45:21.686760+0000","last_clean":"2024-09-19T05:45:21.686760+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:21.686760+0000","last_undegraded":"2024-09-19T05:45:21.686760+0000","last_fullsized":"2024-09-19T05:45:21.686760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"84'159","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688440+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:45:21.688440+0000","last_peered":"2024-09-19T05:45:21.688440+0000","last_clean":"2024-09-19T05:45:21.688440+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:21.688440+0000","last_undegraded":"2024-09-19T05:45:21.688440+0000","last_fullsized":"2024-09-19T05:45:21.688440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:54.111389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'167","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773293+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:45:21.773293+0000","last_peered":"2024-09-19T05:45:21.773293+0000","last_clean":"2024-09-19T05:45:21.773293+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:21.773293+0000","last_undegraded":"2024-09-19T05:45:21.773293+0000","last_fullsized":"2024-09-19T05:45:21.773293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773059+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:45:21.773059+0000","last_peered":"2024-09-19T05:45:21.773059+0000","last_clean":"2024-09-19T05:45:21.773059+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:21.773059+0000","last_undegraded":"2024-09-19T05:45:21.773059+0000","last_fullsized":"2024-09-19T05:45:21.773059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"84'170","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687615+0000","last_change":"2024-09-19T05:44:53.930699+0000","last_active":"2024-09-19T05:45:21.687615+0000","last_peered":"2024-09-19T05:45:21.687615+0000","last_clean":"2024-09-19T05:45:21.687615+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:21.687615+0000","last_undegraded":"2024-09-19T05:45:21.687615+0000","last_fullsized":"2024-09-19T05:45:21.687615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:02.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"84'153","reported_seq":324,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716095+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:45:21.716095+0000","last_peered":"2024-09-19T05:45:21.716095+0000","last_clean":"2024-09-19T05:45:21.716095+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:21.716095+0000","last_undegraded":"2024-09-19T05:45:21.716095+0000","last_fullsized":"2024-09-19T05:45:21.716095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"84'168","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715985+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:45:21.715985+0000","last_peered":"2024-09-19T05:45:21.715985+0000","last_clean":"2024-09-19T05:45:21.715985+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:21.715985+0000","last_undegraded":"2024-09-19T05:45:21.715985+0000","last_fullsized":"2024-09-19T05:45:21.715985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"84'169","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687671+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:45:21.687671+0000","last_peered":"2024-09-19T05:45:21.687671+0000","last_clean":"2024-09-19T05:45:21.687671+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:21.687671+0000","last_undegraded":"2024-09-19T05:45:21.687671+0000","last_fullsized":"2024-09-19T05:45:21.687671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"84'198","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716789+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:45:21.716789+0000","last_peered":"2024-09-19T05:45:21.716789+0000","last_clean":"2024-09-19T05:45:21.716789+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:21.716789+0000","last_undegraded":"2024-09-19T05:45:21.716789+0000","last_fullsized":"2024-09-19T05:45:21.716789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"84'171","reported_seq":353,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688653+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:45:21.688653+0000","last_peered":"2024-09-19T05:45:21.688653+0000","last_clean":"2024-09-19T05:45:21.688653+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:21.688653+0000","last_undegraded":"2024-09-19T05:45:21.688653+0000","last_fullsized":"2024-09-19T05:45:21.688653+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:17.766409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"84'143","reported_seq":315,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708984+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:45:21.708984+0000","last_peered":"2024-09-19T05:45:21.708984+0000","last_clean":"2024-09-19T05:45:21.708984+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:21.708984+0000","last_undegraded":"2024-09-19T05:45:21.708984+0000","last_fullsized":"2024-09-19T05:45:21.708984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"84'189","reported_seq":246,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687810+0000","last_change":"2024-09-19T05:44:55.559370+0000","last_active":"2024-09-19T05:45:21.687810+0000","last_peered":"2024-09-19T05:45:21.687810+0000","last_clean":"2024-09-19T05:45:21.687810+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:21.687810+0000","last_undegraded":"2024-09-19T05:45:21.687810+0000","last_fullsized":"2024-09-19T05:45:21.687810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:25.838393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'142","reported_seq":295,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687168+0000","last_change":"2024-09-19T05:44:56.134769+0000","last_active":"2024-09-19T05:45:21.687168+0000","last_peered":"2024-09-19T05:45:21.687168+0000","last_clean":"2024-09-19T05:45:21.687168+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:21.687168+0000","last_undegraded":"2024-09-19T05:45:21.687168+0000","last_fullsized":"2024-09-19T05:45:21.687168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:06.255089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"84'201","reported_seq":492,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716107+0000","last_change":"2024-09-19T05:44:56.565770+0000","last_active":"2024-09-19T05:45:21.716107+0000","last_peered":"2024-09-19T05:45:21.716107+0000","last_clean":"2024-09-19T05:45:21.716107+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:21.716107+0000","last_undegraded":"2024-09-19T05:45:21.716107+0000","last_fullsized":"2024-09-19T05:45:21.716107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:03.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"84'158","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688447+0000","last_change":"2024-09-19T05:44:56.582314+0000","last_active":"2024-09-19T05:45:21.688447+0000","last_peered":"2024-09-19T05:45:21.688447+0000","last_clean":"2024-09-19T05:45:21.688447+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:21.688447+0000","last_undegraded":"2024-09-19T05:45:21.688447+0000","last_fullsized":"2024-09-19T05:45:21.688447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:41.223871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"84'202","reported_seq":374,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935387+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:45:21.935387+0000","last_peered":"2024-09-19T05:45:21.935387+0000","last_clean":"2024-09-19T05:45:21.935387+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:21.935387+0000","last_undegraded":"2024-09-19T05:45:21.935387+0000","last_fullsized":"2024-09-19T05:45:21.935387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773382+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:45:21.773382+0000","last_peered":"2024-09-19T05:45:21.773382+0000","last_clean":"2024-09-19T05:45:21.773382+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:21.773382+0000","last_undegraded":"2024-09-19T05:45:21.773382+0000","last_fullsized":"2024-09-19T05:45:21.773382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"84'148","reported_seq":320,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773260+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:45:21.773260+0000","last_peered":"2024-09-19T05:45:21.773260+0000","last_clean":"2024-09-19T05:45:21.773260+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:21.773260+0000","last_undegraded":"2024-09-19T05:45:21.773260+0000","last_fullsized":"2024-09-19T05:45:21.773260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'127","reported_seq":281,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688046+0000","last_change":"2024-09-19T05:44:54.977580+0000","last_active":"2024-09-19T05:45:21.688046+0000","last_peered":"2024-09-19T05:45:21.688046+0000","last_clean":"2024-09-19T05:45:21.688046+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:21.688046+0000","last_undegraded":"2024-09-19T05:45:21.688046+0000","last_fullsized":"2024-09-19T05:45:21.688046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:12.165530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"84'145","reported_seq":408,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716323+0000","last_change":"2024-09-19T05:44:53.510921+0000","last_active":"2024-09-19T05:45:21.716323+0000","last_peered":"2024-09-19T05:45:21.716323+0000","last_clean":"2024-09-19T05:45:21.716323+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:21.716323+0000","last_undegraded":"2024-09-19T05:45:21.716323+0000","last_fullsized":"2024-09-19T05:45:21.716323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:39.530204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"84'146","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687436+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:45:21.687436+0000","last_peered":"2024-09-19T05:45:21.687436+0000","last_clean":"2024-09-19T05:45:21.687436+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:21.687436+0000","last_undegraded":"2024-09-19T05:45:21.687436+0000","last_fullsized":"2024-09-19T05:45:21.687436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'171","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935092+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:45:21.935092+0000","last_peered":"2024-09-19T05:45:21.935092+0000","last_clean":"2024-09-19T05:45:21.935092+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:21.935092+0000","last_undegraded":"2024-09-19T05:45:21.935092+0000","last_fullsized":"2024-09-19T05:45:21.935092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"84'142","reported_seq":240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189849+0000","last_change":"2024-09-19T05:45:10.624968+0000","last_active":"2024-09-19T05:45:22.189849+0000","last_peered":"2024-09-19T05:45:22.189849+0000","last_clean":"2024-09-19T05:45:22.189849+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:22.189849+0000","last_undegraded":"2024-09-19T05:45:22.189849+0000","last_fullsized":"2024-09-19T05:45:22.189849+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:59.539763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"94'1809","reported_seq":1885,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687685+0000","last_change":"2024-09-19T05:44:55.560257+0000","last_active":"2024-09-19T05:45:21.687685+0000","last_peered":"2024-09-19T05:45:21.687685+0000","last_clean":"2024-09-19T05:45:21.687685+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:21.687685+0000","last_undegraded":"2024-09-19T05:45:21.687685+0000","last_fullsized":"2024-09-19T05:45:21.687685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1809,"log_dups_size":0,"ondisk_log_size":1809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:25.647547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'154","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708831+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:45:21.708831+0000","last_peered":"2024-09-19T05:45:21.708831+0000","last_clean":"2024-09-19T05:45:21.708831+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:21.708831+0000","last_undegraded":"2024-09-19T05:45:21.708831+0000","last_fullsized":"2024-09-19T05:45:21.708831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716460+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:45:21.716460+0000","last_peered":"2024-09-19T05:45:21.716460+0000","last_clean":"2024-09-19T05:45:21.716460+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:21.716460+0000","last_undegraded":"2024-09-19T05:45:21.716460+0000","last_fullsized":"2024-09-19T05:45:21.716460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773054+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:45:21.773054+0000","last_peered":"2024-09-19T05:45:21.773054+0000","last_clean":"2024-09-19T05:45:21.773054+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:21.773054+0000","last_undegraded":"2024-09-19T05:45:21.773054+0000","last_fullsized":"2024-09-19T05:45:21.773054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'182","reported_seq":337,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687351+0000","last_change":"2024-09-19T05:44:59.989289+0000","last_active":"2024-09-19T05:45:21.687351+0000","last_peered":"2024-09-19T05:45:21.687351+0000","last_clean":"2024-09-19T05:45:21.687351+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:21.687351+0000","last_undegraded":"2024-09-19T05:45:21.687351+0000","last_fullsized":"2024-09-19T05:45:21.687351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:52.052457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716243+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:45:21.716243+0000","last_peered":"2024-09-19T05:45:21.716243+0000","last_clean":"2024-09-19T05:45:21.716243+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:21.716243+0000","last_undegraded":"2024-09-19T05:45:21.716243+0000","last_fullsized":"2024-09-19T05:45:21.716243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"84'177","reported_seq":359,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708983+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:45:21.708983+0000","last_peered":"2024-09-19T05:45:21.708983+0000","last_clean":"2024-09-19T05:45:21.708983+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:21.708983+0000","last_undegraded":"2024-09-19T05:45:21.708983+0000","last_fullsized":"2024-09-19T05:45:21.708983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190091+0000","last_change":"2024-09-19T05:45:10.622931+0000","last_active":"2024-09-19T05:45:22.190091+0000","last_peered":"2024-09-19T05:45:22.190091+0000","last_clean":"2024-09-19T05:45:22.190091+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:22.190091+0000","last_undegraded":"2024-09-19T05:45:22.190091+0000","last_fullsized":"2024-09-19T05:45:22.190091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:34:06.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"84'241","reported_seq":413,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935402+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:45:21.935402+0000","last_peered":"2024-09-19T05:45:21.935402+0000","last_clean":"2024-09-19T05:45:21.935402+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:21.935402+0000","last_undegraded":"2024-09-19T05:45:21.935402+0000","last_fullsized":"2024-09-19T05:45:21.935402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688495+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:45:21.688495+0000","last_peered":"2024-09-19T05:45:21.688495+0000","last_clean":"2024-09-19T05:45:21.688495+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:21.688495+0000","last_undegraded":"2024-09-19T05:45:21.688495+0000","last_fullsized":"2024-09-19T05:45:21.688495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:39.063921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"84'151","reported_seq":300,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688324+0000","last_change":"2024-09-19T05:44:59.517543+0000","last_active":"2024-09-19T05:45:21.688324+0000","last_peered":"2024-09-19T05:45:21.688324+0000","last_clean":"2024-09-19T05:45:21.688324+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:21.688324+0000","last_undegraded":"2024-09-19T05:45:21.688324+0000","last_fullsized":"2024-09-19T05:45:21.688324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.453343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"89'10","reported_seq":185,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687787+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:45:21.687787+0000","last_peered":"2024-09-19T05:45:21.687787+0000","last_clean":"2024-09-19T05:45:21.687787+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:21.687787+0000","last_undegraded":"2024-09-19T05:45:21.687787+0000","last_fullsized":"2024-09-19T05:45:21.687787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"84'157","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716300+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:45:21.716300+0000","last_peered":"2024-09-19T05:45:21.716300+0000","last_clean":"2024-09-19T05:45:21.716300+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:21.716300+0000","last_undegraded":"2024-09-19T05:45:21.716300+0000","last_fullsized":"2024-09-19T05:45:21.716300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"94'2017","reported_seq":2191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935286+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:45:21.935286+0000","last_peered":"2024-09-19T05:45:21.935286+0000","last_clean":"2024-09-19T05:45:21.935286+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:21.935286+0000","last_undegraded":"2024-09-19T05:45:21.935286+0000","last_fullsized":"2024-09-19T05:45:21.935286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688257+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:45:21.688257+0000","last_peered":"2024-09-19T05:45:21.688257+0000","last_clean":"2024-09-19T05:45:21.688257+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:21.688257+0000","last_undegraded":"2024-09-19T05:45:21.688257+0000","last_fullsized":"2024-09-19T05:45:21.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687838+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:45:21.687838+0000","last_peered":"2024-09-19T05:45:21.687838+0000","last_clean":"2024-09-19T05:45:21.687838+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:21.687838+0000","last_undegraded":"2024-09-19T05:45:21.687838+0000","last_fullsized":"2024-09-19T05:45:21.687838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"84'168","reported_seq":340,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773221+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:45:21.773221+0000","last_peered":"2024-09-19T05:45:21.773221+0000","last_clean":"2024-09-19T05:45:21.773221+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:21.773221+0000","last_undegraded":"2024-09-19T05:45:21.773221+0000","last_fullsized":"2024-09-19T05:45:21.773221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716531+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:45:21.716531+0000","last_peered":"2024-09-19T05:45:21.716531+0000","last_clean":"2024-09-19T05:45:21.716531+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:21.716531+0000","last_undegraded":"2024-09-19T05:45:21.716531+0000","last_fullsized":"2024-09-19T05:45:21.716531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"84'186","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715983+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:45:21.715983+0000","last_peered":"2024-09-19T05:45:21.715983+0000","last_clean":"2024-09-19T05:45:21.715983+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:21.715983+0000","last_undegraded":"2024-09-19T05:45:21.715983+0000","last_fullsized":"2024-09-19T05:45:21.715983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687854+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:45:21.687854+0000","last_peered":"2024-09-19T05:45:21.687854+0000","last_clean":"2024-09-19T05:45:21.687854+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:21.687854+0000","last_undegraded":"2024-09-19T05:45:21.687854+0000","last_fullsized":"2024-09-19T05:45:21.687854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"84'191","reported_seq":283,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189767+0000","last_change":"2024-09-19T05:45:10.627674+0000","last_active":"2024-09-19T05:45:22.189767+0000","last_peered":"2024-09-19T05:45:22.189767+0000","last_clean":"2024-09-19T05:45:22.189767+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:22.189767+0000","last_undegraded":"2024-09-19T05:45:22.189767+0000","last_fullsized":"2024-09-19T05:45:22.189767+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:21.685895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"89'9","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708963+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:45:21.708963+0000","last_peered":"2024-09-19T05:45:21.708963+0000","last_clean":"2024-09-19T05:45:21.708963+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:21.708963+0000","last_undegraded":"2024-09-19T05:45:21.708963+0000","last_fullsized":"2024-09-19T05:45:21.708963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"84'209","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716252+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:45:21.716252+0000","last_peered":"2024-09-19T05:45:21.716252+0000","last_clean":"2024-09-19T05:45:21.716252+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:21.716252+0000","last_undegraded":"2024-09-19T05:45:21.716252+0000","last_fullsized":"2024-09-19T05:45:21.716252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688782+0000","last_change":"2024-09-19T05:44:53.065559+0000","last_active":"2024-09-19T05:45:21.688782+0000","last_peered":"2024-09-19T05:45:21.688782+0000","last_clean":"2024-09-19T05:45:21.688782+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:21.688782+0000","last_undegraded":"2024-09-19T05:45:21.688782+0000","last_fullsized":"2024-09-19T05:45:21.688782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:13.889455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935000+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:45:21.935000+0000","last_peered":"2024-09-19T05:45:21.935000+0000","last_clean":"2024-09-19T05:45:21.935000+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:21.935000+0000","last_undegraded":"2024-09-19T05:45:21.935000+0000","last_fullsized":"2024-09-19T05:45:21.935000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709231+0000","last_change":"2024-09-19T05:44:53.059769+0000","last_active":"2024-09-19T05:45:21.709231+0000","last_peered":"2024-09-19T05:45:21.709231+0000","last_clean":"2024-09-19T05:45:21.709231+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:21.709231+0000","last_undegraded":"2024-09-19T05:45:21.709231+0000","last_fullsized":"2024-09-19T05:45:21.709231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:03.909805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"84'153","reported_seq":231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687781+0000","last_change":"2024-09-19T05:44:15.787415+0000","last_active":"2024-09-19T05:45:21.687781+0000","last_peered":"2024-09-19T05:45:21.687781+0000","last_clean":"2024-09-19T05:45:21.687781+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:21.687781+0000","last_undegraded":"2024-09-19T05:45:21.687781+0000","last_fullsized":"2024-09-19T05:45:21.687781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:10.095369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":125,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688186+0000","last_change":"2024-09-19T05:44:53.058979+0000","last_active":"2024-09-19T05:45:21.688186+0000","last_peered":"2024-09-19T05:45:21.688186+0000","last_clean":"2024-09-19T05:45:21.688186+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:21.688186+0000","last_undegraded":"2024-09-19T05:45:21.688186+0000","last_fullsized":"2024-09-19T05:45:21.688186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:04.530725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686740+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:45:21.686740+0000","last_peered":"2024-09-19T05:45:21.686740+0000","last_clean":"2024-09-19T05:45:21.686740+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:21.686740+0000","last_undegraded":"2024-09-19T05:45:21.686740+0000","last_fullsized":"2024-09-19T05:45:21.686740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773397+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:45:21.773397+0000","last_peered":"2024-09-19T05:45:21.773397+0000","last_clean":"2024-09-19T05:45:21.773397+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:21.773397+0000","last_undegraded":"2024-09-19T05:45:21.773397+0000","last_fullsized":"2024-09-19T05:45:21.773397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"84'168","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716718+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:45:21.716718+0000","last_peered":"2024-09-19T05:45:21.716718+0000","last_clean":"2024-09-19T05:45:21.716718+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:21.716718+0000","last_undegraded":"2024-09-19T05:45:21.716718+0000","last_fullsized":"2024-09-19T05:45:21.716718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189880+0000","last_change":"2024-09-19T05:45:10.627237+0000","last_active":"2024-09-19T05:45:22.189880+0000","last_peered":"2024-09-19T05:45:22.189880+0000","last_clean":"2024-09-19T05:45:22.189880+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:22.189880+0000","last_undegraded":"2024-09-19T05:45:22.189880+0000","last_fullsized":"2024-09-19T05:45:22.189880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:04.886546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716440+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:45:21.716440+0000","last_peered":"2024-09-19T05:45:21.716440+0000","last_clean":"2024-09-19T05:45:21.716440+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:21.716440+0000","last_undegraded":"2024-09-19T05:45:21.716440+0000","last_fullsized":"2024-09-19T05:45:21.716440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708974+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:45:21.708974+0000","last_peered":"2024-09-19T05:45:21.708974+0000","last_clean":"2024-09-19T05:45:21.708974+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:21.708974+0000","last_undegraded":"2024-09-19T05:45:21.708974+0000","last_fullsized":"2024-09-19T05:45:21.708974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716388+0000","last_change":"2024-09-19T05:44:53.069283+0000","last_active":"2024-09-19T05:45:21.716388+0000","last_peered":"2024-09-19T05:45:21.716388+0000","last_clean":"2024-09-19T05:45:21.716388+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:21.716388+0000","last_undegraded":"2024-09-19T05:45:21.716388+0000","last_fullsized":"2024-09-19T05:45:21.716388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:51.209720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773046+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:45:21.773046+0000","last_peered":"2024-09-19T05:45:21.773046+0000","last_clean":"2024-09-19T05:45:21.773046+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:21.773046+0000","last_undegraded":"2024-09-19T05:45:21.773046+0000","last_fullsized":"2024-09-19T05:45:21.773046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687539+0000","last_change":"2024-09-19T05:44:53.068415+0000","last_active":"2024-09-19T05:45:21.687539+0000","last_peered":"2024-09-19T05:45:21.687539+0000","last_clean":"2024-09-19T05:45:21.687539+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:21.687539+0000","last_undegraded":"2024-09-19T05:45:21.687539+0000","last_fullsized":"2024-09-19T05:45:21.687539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:16.609871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"84'157","reported_seq":310,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687083+0000","last_change":"2024-09-19T05:44:58.820818+0000","last_active":"2024-09-19T05:45:21.687083+0000","last_peered":"2024-09-19T05:45:21.687083+0000","last_clean":"2024-09-19T05:45:21.687083+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:21.687083+0000","last_undegraded":"2024-09-19T05:45:21.687083+0000","last_fullsized":"2024-09-19T05:45:21.687083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:32.427214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716567+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:45:21.716567+0000","last_peered":"2024-09-19T05:45:21.716567+0000","last_clean":"2024-09-19T05:45:21.716567+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:21.716567+0000","last_undegraded":"2024-09-19T05:45:21.716567+0000","last_fullsized":"2024-09-19T05:45:21.716567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686759+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:45:21.686759+0000","last_peered":"2024-09-19T05:45:21.686759+0000","last_clean":"2024-09-19T05:45:21.686759+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:21.686759+0000","last_undegraded":"2024-09-19T05:45:21.686759+0000","last_fullsized":"2024-09-19T05:45:21.686759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687970+0000","last_change":"2024-09-19T05:44:53.063483+0000","last_active":"2024-09-19T05:45:21.687970+0000","last_peered":"2024-09-19T05:45:21.687970+0000","last_clean":"2024-09-19T05:45:21.687970+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:21.687970+0000","last_undegraded":"2024-09-19T05:45:21.687970+0000","last_fullsized":"2024-09-19T05:45:21.687970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:40.306784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"84'155","reported_seq":327,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935176+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:45:21.935176+0000","last_peered":"2024-09-19T05:45:21.935176+0000","last_clean":"2024-09-19T05:45:21.935176+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:21.935176+0000","last_undegraded":"2024-09-19T05:45:21.935176+0000","last_fullsized":"2024-09-19T05:45:21.935176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716905+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:45:21.716905+0000","last_peered":"2024-09-19T05:45:21.716905+0000","last_clean":"2024-09-19T05:45:21.716905+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:21.716905+0000","last_undegraded":"2024-09-19T05:45:21.716905+0000","last_fullsized":"2024-09-19T05:45:21.716905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'153","reported_seq":424,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773336+0000","last_change":"2024-09-19T05:44:55.675882+0000","last_active":"2024-09-19T05:45:21.773336+0000","last_peered":"2024-09-19T05:45:21.773336+0000","last_clean":"2024-09-19T05:45:21.773336+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:21.773336+0000","last_undegraded":"2024-09-19T05:45:21.773336+0000","last_fullsized":"2024-09-19T05:45:21.773336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:04.593886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709321+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:45:21.709321+0000","last_peered":"2024-09-19T05:45:21.709321+0000","last_clean":"2024-09-19T05:45:21.709321+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:21.709321+0000","last_undegraded":"2024-09-19T05:45:21.709321+0000","last_fullsized":"2024-09-19T05:45:21.709321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"84'161","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709277+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:45:21.709277+0000","last_peered":"2024-09-19T05:45:21.709277+0000","last_clean":"2024-09-19T05:45:21.709277+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:21.709277+0000","last_undegraded":"2024-09-19T05:45:21.709277+0000","last_fullsized":"2024-09-19T05:45:21.709277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"85'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716921+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:45:21.716921+0000","last_peered":"2024-09-19T05:45:21.716921+0000","last_clean":"2024-09-19T05:45:21.716921+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:21.716921+0000","last_undegraded":"2024-09-19T05:45:21.716921+0000","last_fullsized":"2024-09-19T05:45:21.716921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"84'176","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709220+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:45:21.709220+0000","last_peered":"2024-09-19T05:45:21.709220+0000","last_clean":"2024-09-19T05:45:21.709220+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:21.709220+0000","last_undegraded":"2024-09-19T05:45:21.709220+0000","last_fullsized":"2024-09-19T05:45:21.709220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687243+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:45:21.687243+0000","last_peered":"2024-09-19T05:45:21.687243+0000","last_clean":"2024-09-19T05:45:21.687243+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:21.687243+0000","last_undegraded":"2024-09-19T05:45:21.687243+0000","last_fullsized":"2024-09-19T05:45:21.687243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"84'160","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716327+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:45:21.716327+0000","last_peered":"2024-09-19T05:45:21.716327+0000","last_clean":"2024-09-19T05:45:21.716327+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:21.716327+0000","last_undegraded":"2024-09-19T05:45:21.716327+0000","last_fullsized":"2024-09-19T05:45:21.716327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709021+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:45:21.709021+0000","last_peered":"2024-09-19T05:45:21.709021+0000","last_clean":"2024-09-19T05:45:21.709021+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:21.709021+0000","last_undegraded":"2024-09-19T05:45:21.709021+0000","last_fullsized":"2024-09-19T05:45:21.709021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708902+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:45:21.708902+0000","last_peered":"2024-09-19T05:45:21.708902+0000","last_clean":"2024-09-19T05:45:21.708902+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:21.708902+0000","last_undegraded":"2024-09-19T05:45:21.708902+0000","last_fullsized":"2024-09-19T05:45:21.708902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773238+0000","last_change":"2024-09-19T05:44:53.065594+0000","last_active":"2024-09-19T05:45:21.773238+0000","last_peered":"2024-09-19T05:45:21.773238+0000","last_clean":"2024-09-19T05:45:21.773238+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:21.773238+0000","last_undegraded":"2024-09-19T05:45:21.773238+0000","last_fullsized":"2024-09-19T05:45:21.773238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:47.254673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"84'156","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687718+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:45:21.687718+0000","last_peered":"2024-09-19T05:45:21.687718+0000","last_clean":"2024-09-19T05:45:21.687718+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:21.687718+0000","last_undegraded":"2024-09-19T05:45:21.687718+0000","last_fullsized":"2024-09-19T05:45:21.687718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"59'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716236+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:45:21.716236+0000","last_peered":"2024-09-19T05:45:21.716236+0000","last_clean":"2024-09-19T05:45:21.716236+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:21.716236+0000","last_undegraded":"2024-09-19T05:45:21.716236+0000","last_fullsized":"2024-09-19T05:45:21.716236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"84'149","reported_seq":330,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716425+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:45:21.716425+0000","last_peered":"2024-09-19T05:45:21.716425+0000","last_clean":"2024-09-19T05:45:21.716425+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:21.716425+0000","last_undegraded":"2024-09-19T05:45:21.716425+0000","last_fullsized":"2024-09-19T05:45:21.716425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"84'190","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715986+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:45:21.715986+0000","last_peered":"2024-09-19T05:45:21.715986+0000","last_clean":"2024-09-19T05:45:21.715986+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:21.715986+0000","last_undegraded":"2024-09-19T05:45:21.715986+0000","last_fullsized":"2024-09-19T05:45:21.715986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":194,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688923+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:45:21.688923+0000","last_peered":"2024-09-19T05:45:21.688923+0000","last_clean":"2024-09-19T05:45:21.688923+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:21.688923+0000","last_undegraded":"2024-09-19T05:45:21.688923+0000","last_fullsized":"2024-09-19T05:45:21.688923+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:59.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"84'153","reported_seq":266,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189956+0000","last_change":"2024-09-19T05:45:10.627058+0000","last_active":"2024-09-19T05:45:22.189956+0000","last_peered":"2024-09-19T05:45:22.189956+0000","last_clean":"2024-09-19T05:45:22.189956+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:22.189956+0000","last_undegraded":"2024-09-19T05:45:22.189956+0000","last_fullsized":"2024-09-19T05:45:22.189956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:52.373727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"70'4","reported_seq":196,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688300+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:45:21.688300+0000","last_peered":"2024-09-19T05:45:21.688300+0000","last_clean":"2024-09-19T05:45:21.688300+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:21.688300+0000","last_undegraded":"2024-09-19T05:45:21.688300+0000","last_fullsized":"2024-09-19T05:45:21.688300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:52.607339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"84'154","reported_seq":314,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688325+0000","last_change":"2024-09-19T05:44:57.957776+0000","last_active":"2024-09-19T05:45:21.688325+0000","last_peered":"2024-09-19T05:45:21.688325+0000","last_clean":"2024-09-19T05:45:21.688325+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:21.688325+0000","last_undegraded":"2024-09-19T05:45:21.688325+0000","last_fullsized":"2024-09-19T05:45:21.688325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:37.587819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":156,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688211+0000","last_change":"2024-09-19T05:44:53.061701+0000","last_active":"2024-09-19T05:45:21.688211+0000","last_peered":"2024-09-19T05:45:21.688211+0000","last_clean":"2024-09-19T05:45:21.688211+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:21.688211+0000","last_undegraded":"2024-09-19T05:45:21.688211+0000","last_fullsized":"2024-09-19T05:45:21.688211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:26.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"84'176","reported_seq":321,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688312+0000","last_change":"2024-09-19T05:44:57.503018+0000","last_active":"2024-09-19T05:45:21.688312+0000","last_peered":"2024-09-19T05:45:21.688312+0000","last_clean":"2024-09-19T05:45:21.688312+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:21.688312+0000","last_undegraded":"2024-09-19T05:45:21.688312+0000","last_fullsized":"2024-09-19T05:45:21.688312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:44.482643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709387+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:45:21.709387+0000","last_peered":"2024-09-19T05:45:21.709387+0000","last_clean":"2024-09-19T05:45:21.709387+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:21.709387+0000","last_undegraded":"2024-09-19T05:45:21.709387+0000","last_fullsized":"2024-09-19T05:45:21.709387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970373,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3906,"num_write_kb":11500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15940746,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3893,"ondisk_log_size":3893,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":5272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":5398528,"data_stored":4749255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":4984832,"data_stored":4334380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449096,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46800,"kb_used_data":1724,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325040,"statfs":{"total":96636764160,"available":96588840960,"internally_reserved":0,"allocated":1765376,"data_stored":4919338,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50368,"kb_used_data":5292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321472,"statfs":{"total":96636764160,"available":96585187328,"internally_reserved":0,"allocated":5419008,"data_stored":4755934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40696,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331144,"statfs":{"total":96636764160,"available":96595091456,"internally_reserved":0,"allocated":1212416,"data_stored":560868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40748,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331092,"statfs":{"total":96636764160,"available":96595038208,"internally_reserved":0,"allocated":1261568,"data_stored":1150724,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40700,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331140,"statfs":{"total":96636764160,"available":96595087360,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774691,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:45:27.469 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T05:45:27.469 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:45:27.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.605+0000 7fef2405c640 1 Processor -- start 2024-09-19T05:45:27.607 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.605+0000 7fef2405c640 1 -- start start 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef2405c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 0x7fef1c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef2405c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fef1c0cc5f0 0x7fef1c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef2405c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fef1c0c55f0 0x7fef1c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef2405c640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fef1c0cdd00 con 0x7fef1c0c55f0 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef2405c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fef1c0cd330 con 0x7fef1c0cc5f0 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef2405c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fef1c0734c0 con 0x7fef1c154cb0 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef225f9640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fef1c0c55f0 0x7fef1c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.609 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef225f9640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fef1c0c55f0 0x7fef1c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54872/0 (socket says 172.21.15.2:54872) 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef225f9640 1 -- 172.21.15.2:0/2760591265 learned_addr learned my addr 172.21.15.2:0/2760591265 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.606+0000 7fef21df8640 1 --2- 172.21.15.2:0/2760591265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 0x7fef1c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef215f7640 1 --2- 172.21.15.2:0/2760591265 >> v2:172.21.15.123:3300/0 conn(0x7fef1c0cc5f0 0x7fef1c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef225f9640 1 -- 172.21.15.2:0/2760591265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 msgr2=0x7fef1c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef225f9640 1 --2- 172.21.15.2:0/2760591265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 0x7fef1c15a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef225f9640 1 -- 172.21.15.2:0/2760591265 >> v2:172.21.15.123:3300/0 conn(0x7fef1c0cc5f0 msgr2=0x7fef1c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef225f9640 1 --2- 172.21.15.2:0/2760591265 >> v2:172.21.15.123:3300/0 conn(0x7fef1c0cc5f0 0x7fef1c0cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.610 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef225f9640 1 -- 172.21.15.2:0/2760591265 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c073780 con 0x7fef1c0c55f0 2024-09-19T05:45:27.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef225f9640 1 --2- 172.21.15.2:0/2760591265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c0cadc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cdf80 tx=0x7fef1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68242bf51cf4c6e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.607+0000 7fef20df6640 1 -- 172.21.15.2:0/2760591265 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14037070 con 0x7fef1c0c55f0 2024-09-19T05:45:27.611 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef20df6640 1 -- 172.21.15.2:0/2760591265 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef14032440 con 0x7fef1c0c55f0 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef20df6640 1 -- 172.21.15.2:0/2760591265 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14032810 con 0x7fef1c0c55f0 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 -- 172.21.15.2:0/2760591265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 msgr2=0x7fef1c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 --2- 172.21.15.2:0/2760591265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c0cadc0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cdf80 tx=0x7fef1402a270 comp rx=0 tx=0).stop 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 -- 172.21.15.2:0/2760591265 shutdown_connections 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 --2- 172.21.15.2:0/2760591265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c0cadc0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 --2- 172.21.15.2:0/2760591265 >> v2:172.21.15.123:3300/0 conn(0x7fef1c0cc5f0 0x7fef1c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 --2- 172.21.15.2:0/2760591265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 0x7fef1c15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 -- 172.21.15.2:0/2760591265 >> 172.21.15.2:0/2760591265 conn(0x7fef1c06d0c0 msgr2=0x7fef1c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 -- 172.21.15.2:0/2760591265 shutdown_connections 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.608+0000 7fef2405c640 1 -- 172.21.15.2:0/2760591265 wait complete. 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.609+0000 7fef2405c640 1 Processor -- start 2024-09-19T05:45:27.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.609+0000 7fef2405c640 1 -- start start 2024-09-19T05:45:27.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef2405c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c1f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef2405c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef1c0cc5f0 0x7fef1c0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef2405c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 0x7fef1c1e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef2405c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0b30d0 con 0x7fef1c0c55f0 2024-09-19T05:45:27.614 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef2405c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0ccfc0 con 0x7fef1c0cc5f0 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef2405c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fef1c0c5a80 con 0x7fef1c154cb0 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c1f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c1f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54888/0 (socket says 172.21.15.2:54888) 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 -- 172.21.15.2:0/2088005638 learned_addr learned my addr 172.21.15.2:0/2088005638 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 -- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 msgr2=0x7fef1c1e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 0x7fef1c1e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 -- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef1c0cc5f0 msgr2=0x7fef1c0b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef215f7640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef1c0cc5f0 0x7fef1c0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef1c0cc5f0 0x7fef1c0b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef21df8640 1 -- 172.21.15.2:0/2088005638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c1e7f50 con 0x7fef1c0c55f0 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.610+0000 7fef215f7640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef1c0cc5f0 0x7fef1c0b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.611+0000 7fef21df8640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c1f3550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00cd00 tx=0x7fef0c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.611+0000 7fef1a7fc640 1 -- 172.21.15.2:0/2088005638 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c003990 con 0x7fef1c0c55f0 2024-09-19T05:45:27.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.611+0000 7fef1a7fc640 1 -- 172.21.15.2:0/2088005638 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c003b30 con 0x7fef1c0c55f0 2024-09-19T05:45:27.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.611+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c147930 con 0x7fef1c0c55f0 2024-09-19T05:45:27.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.611+0000 7fef1a7fc640 1 -- 172.21.15.2:0/2088005638 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c004020 con 0x7fef1c0c55f0 2024-09-19T05:45:27.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.611+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c0ce640 con 0x7fef1c0c55f0 2024-09-19T05:45:27.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.612+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1c0cd7c0 con 0x7fef1c0c55f0 2024-09-19T05:45:27.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.612+0000 7fef1a7fc640 1 -- 172.21.15.2:0/2088005638 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fef0c0041c0 con 0x7fef1c0c55f0 2024-09-19T05:45:27.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.613+0000 7fef1a7fc640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef0806c2b0 0x7fef0806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.613+0000 7fef1a7fc640 1 -- 172.21.15.2:0/2088005638 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fef0c08b8c0 con 0x7fef1c0c55f0 2024-09-19T05:45:27.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.613+0000 7fef215f7640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef0806c2b0 0x7fef0806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.617 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.614+0000 7fef215f7640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef0806c2b0 0x7fef0806e770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fef10006fd0 tx=0x7fef10008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.619 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.617+0000 7fef1a7fc640 1 -- 172.21.15.2:0/2088005638 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fef0c014030 con 0x7fef1c0c55f0 2024-09-19T05:45:27.795 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.793+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef1c1470c0 con 0x7fef0806c2b0 2024-09-19T05:45:27.801 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.800+0000 7fef1a7fc640 1 -- 172.21.15.2:0/2088005638 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253668 (secure 0 0 0) 0x7fef1c1470c0 con 0x7fef0806c2b0 2024-09-19T05:45:27.804 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:27.805 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef0806c2b0 msgr2=0x7fef0806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef0806c2b0 0x7fef0806e770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fef10006fd0 tx=0x7fef10008040 comp rx=0 tx=0).stop 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 msgr2=0x7fef1c1f3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c1f3550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00cd00 tx=0x7fef0c00a5c0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 shutdown_connections 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fef0806c2b0 0x7fef0806e770 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef1c0cc5f0 0x7fef1c0b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fef1c154cb0 0x7fef1c1e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 --2- 172.21.15.2:0/2088005638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fef1c0c55f0 0x7fef1c1f3550 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.807 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 >> 172.21.15.2:0/2088005638 conn(0x7fef1c06d0c0 msgr2=0x7fef1c0cbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:27.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 shutdown_connections 2024-09-19T05:45:27.808 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.802+0000 7fef2405c640 1 -- 172.21.15.2:0/2088005638 wait complete. 2024-09-19T05:45:27.815 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-19T05:45:25.855290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560741,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13281,"num_write_kb":25179740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14765,"ondisk_log_size":14765,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342284,"kb_used_data":21952,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632436,"statfs":{"total":773094113280,"available":772743614464,"internally_reserved":0,"allocated":22478848,"data_stored":21591756,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.183313"},"pg_stats":[{"pgid":"3.3f","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687024+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:45:21.687024+0000","last_peered":"2024-09-19T05:45:21.687024+0000","last_clean":"2024-09-19T05:45:21.687024+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:21.687024+0000","last_undegraded":"2024-09-19T05:45:21.687024+0000","last_fullsized":"2024-09-19T05:45:21.687024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:27.701236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"84'148","reported_seq":301,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688407+0000","last_change":"2024-09-19T05:44:54.646789+0000","last_active":"2024-09-19T05:45:21.688407+0000","last_peered":"2024-09-19T05:45:21.688407+0000","last_clean":"2024-09-19T05:45:21.688407+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:21.688407+0000","last_undegraded":"2024-09-19T05:45:21.688407+0000","last_fullsized":"2024-09-19T05:45:21.688407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:11.761535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"84'171","reported_seq":325,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688172+0000","last_change":"2024-09-19T05:44:53.475804+0000","last_active":"2024-09-19T05:45:21.688172+0000","last_peered":"2024-09-19T05:45:21.688172+0000","last_clean":"2024-09-19T05:45:21.688172+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:21.688172+0000","last_undegraded":"2024-09-19T05:45:21.688172+0000","last_fullsized":"2024-09-19T05:45:21.688172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:57.009039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"84'185","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935211+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:45:21.935211+0000","last_peered":"2024-09-19T05:45:21.935211+0000","last_clean":"2024-09-19T05:45:21.935211+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:21.935211+0000","last_undegraded":"2024-09-19T05:45:21.935211+0000","last_fullsized":"2024-09-19T05:45:21.935211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"84'170","reported_seq":438,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687949+0000","last_change":"2024-09-19T05:44:53.953498+0000","last_active":"2024-09-19T05:45:21.687949+0000","last_peered":"2024-09-19T05:45:21.687949+0000","last_clean":"2024-09-19T05:45:21.687949+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:21.687949+0000","last_undegraded":"2024-09-19T05:45:21.687949+0000","last_fullsized":"2024-09-19T05:45:21.687949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:39.367204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"84'156","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935150+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:45:21.935150+0000","last_peered":"2024-09-19T05:45:21.935150+0000","last_clean":"2024-09-19T05:45:21.935150+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:21.935150+0000","last_undegraded":"2024-09-19T05:45:21.935150+0000","last_fullsized":"2024-09-19T05:45:21.935150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"84'183","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686741+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:45:21.686741+0000","last_peered":"2024-09-19T05:45:21.686741+0000","last_clean":"2024-09-19T05:45:21.686741+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:21.686741+0000","last_undegraded":"2024-09-19T05:45:21.686741+0000","last_fullsized":"2024-09-19T05:45:21.686741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"84'195","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687356+0000","last_change":"2024-09-19T05:44:56.567156+0000","last_active":"2024-09-19T05:45:21.687356+0000","last_peered":"2024-09-19T05:45:21.687356+0000","last_clean":"2024-09-19T05:45:21.687356+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:21.687356+0000","last_undegraded":"2024-09-19T05:45:21.687356+0000","last_fullsized":"2024-09-19T05:45:21.687356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:17.811359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"84'167","reported_seq":280,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190149+0000","last_change":"2024-09-19T05:45:10.627715+0000","last_active":"2024-09-19T05:45:22.190149+0000","last_peered":"2024-09-19T05:45:22.190149+0000","last_clean":"2024-09-19T05:45:22.190149+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:22.190149+0000","last_undegraded":"2024-09-19T05:45:22.190149+0000","last_fullsized":"2024-09-19T05:45:22.190149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:38.174880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686962+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:45:21.686962+0000","last_peered":"2024-09-19T05:45:21.686962+0000","last_clean":"2024-09-19T05:45:21.686962+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:21.686962+0000","last_undegraded":"2024-09-19T05:45:21.686962+0000","last_fullsized":"2024-09-19T05:45:21.686962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:12.275813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"84'208","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686760+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:45:21.686760+0000","last_peered":"2024-09-19T05:45:21.686760+0000","last_clean":"2024-09-19T05:45:21.686760+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:21.686760+0000","last_undegraded":"2024-09-19T05:45:21.686760+0000","last_fullsized":"2024-09-19T05:45:21.686760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:37.494781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"84'159","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688440+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:45:21.688440+0000","last_peered":"2024-09-19T05:45:21.688440+0000","last_clean":"2024-09-19T05:45:21.688440+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:21.688440+0000","last_undegraded":"2024-09-19T05:45:21.688440+0000","last_fullsized":"2024-09-19T05:45:21.688440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:54.111389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"84'167","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773293+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:45:21.773293+0000","last_peered":"2024-09-19T05:45:21.773293+0000","last_clean":"2024-09-19T05:45:21.773293+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:21.773293+0000","last_undegraded":"2024-09-19T05:45:21.773293+0000","last_fullsized":"2024-09-19T05:45:21.773293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773059+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:45:21.773059+0000","last_peered":"2024-09-19T05:45:21.773059+0000","last_clean":"2024-09-19T05:45:21.773059+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:21.773059+0000","last_undegraded":"2024-09-19T05:45:21.773059+0000","last_fullsized":"2024-09-19T05:45:21.773059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"84'170","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687615+0000","last_change":"2024-09-19T05:44:53.930699+0000","last_active":"2024-09-19T05:45:21.687615+0000","last_peered":"2024-09-19T05:45:21.687615+0000","last_clean":"2024-09-19T05:45:21.687615+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:21.687615+0000","last_undegraded":"2024-09-19T05:45:21.687615+0000","last_fullsized":"2024-09-19T05:45:21.687615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:02.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"84'153","reported_seq":324,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716095+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:45:21.716095+0000","last_peered":"2024-09-19T05:45:21.716095+0000","last_clean":"2024-09-19T05:45:21.716095+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:21.716095+0000","last_undegraded":"2024-09-19T05:45:21.716095+0000","last_fullsized":"2024-09-19T05:45:21.716095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"84'168","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715985+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:45:21.715985+0000","last_peered":"2024-09-19T05:45:21.715985+0000","last_clean":"2024-09-19T05:45:21.715985+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:21.715985+0000","last_undegraded":"2024-09-19T05:45:21.715985+0000","last_fullsized":"2024-09-19T05:45:21.715985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"84'169","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687671+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:45:21.687671+0000","last_peered":"2024-09-19T05:45:21.687671+0000","last_clean":"2024-09-19T05:45:21.687671+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:21.687671+0000","last_undegraded":"2024-09-19T05:45:21.687671+0000","last_fullsized":"2024-09-19T05:45:21.687671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"84'198","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716789+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:45:21.716789+0000","last_peered":"2024-09-19T05:45:21.716789+0000","last_clean":"2024-09-19T05:45:21.716789+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:21.716789+0000","last_undegraded":"2024-09-19T05:45:21.716789+0000","last_fullsized":"2024-09-19T05:45:21.716789+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"84'171","reported_seq":353,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688653+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:45:21.688653+0000","last_peered":"2024-09-19T05:45:21.688653+0000","last_clean":"2024-09-19T05:45:21.688653+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:21.688653+0000","last_undegraded":"2024-09-19T05:45:21.688653+0000","last_fullsized":"2024-09-19T05:45:21.688653+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:17.766409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"84'143","reported_seq":315,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708984+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:45:21.708984+0000","last_peered":"2024-09-19T05:45:21.708984+0000","last_clean":"2024-09-19T05:45:21.708984+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:21.708984+0000","last_undegraded":"2024-09-19T05:45:21.708984+0000","last_fullsized":"2024-09-19T05:45:21.708984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"84'189","reported_seq":246,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687810+0000","last_change":"2024-09-19T05:44:55.559370+0000","last_active":"2024-09-19T05:45:21.687810+0000","last_peered":"2024-09-19T05:45:21.687810+0000","last_clean":"2024-09-19T05:45:21.687810+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:21.687810+0000","last_undegraded":"2024-09-19T05:45:21.687810+0000","last_fullsized":"2024-09-19T05:45:21.687810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:25.838393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'142","reported_seq":295,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687168+0000","last_change":"2024-09-19T05:44:56.134769+0000","last_active":"2024-09-19T05:45:21.687168+0000","last_peered":"2024-09-19T05:45:21.687168+0000","last_clean":"2024-09-19T05:45:21.687168+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:21.687168+0000","last_undegraded":"2024-09-19T05:45:21.687168+0000","last_fullsized":"2024-09-19T05:45:21.687168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:06.255089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"84'201","reported_seq":492,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716107+0000","last_change":"2024-09-19T05:44:56.565770+0000","last_active":"2024-09-19T05:45:21.716107+0000","last_peered":"2024-09-19T05:45:21.716107+0000","last_clean":"2024-09-19T05:45:21.716107+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:21.716107+0000","last_undegraded":"2024-09-19T05:45:21.716107+0000","last_fullsized":"2024-09-19T05:45:21.716107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:03.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"84'158","reported_seq":319,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688447+0000","last_change":"2024-09-19T05:44:56.582314+0000","last_active":"2024-09-19T05:45:21.688447+0000","last_peered":"2024-09-19T05:45:21.688447+0000","last_clean":"2024-09-19T05:45:21.688447+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:21.688447+0000","last_undegraded":"2024-09-19T05:45:21.688447+0000","last_fullsized":"2024-09-19T05:45:21.688447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:41.223871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"84'202","reported_seq":374,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935387+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:45:21.935387+0000","last_peered":"2024-09-19T05:45:21.935387+0000","last_clean":"2024-09-19T05:45:21.935387+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:21.935387+0000","last_undegraded":"2024-09-19T05:45:21.935387+0000","last_fullsized":"2024-09-19T05:45:21.935387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773382+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:45:21.773382+0000","last_peered":"2024-09-19T05:45:21.773382+0000","last_clean":"2024-09-19T05:45:21.773382+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:21.773382+0000","last_undegraded":"2024-09-19T05:45:21.773382+0000","last_fullsized":"2024-09-19T05:45:21.773382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"84'148","reported_seq":320,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773260+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:45:21.773260+0000","last_peered":"2024-09-19T05:45:21.773260+0000","last_clean":"2024-09-19T05:45:21.773260+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:21.773260+0000","last_undegraded":"2024-09-19T05:45:21.773260+0000","last_fullsized":"2024-09-19T05:45:21.773260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'127","reported_seq":281,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688046+0000","last_change":"2024-09-19T05:44:54.977580+0000","last_active":"2024-09-19T05:45:21.688046+0000","last_peered":"2024-09-19T05:45:21.688046+0000","last_clean":"2024-09-19T05:45:21.688046+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:21.688046+0000","last_undegraded":"2024-09-19T05:45:21.688046+0000","last_fullsized":"2024-09-19T05:45:21.688046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:12.165530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"84'145","reported_seq":408,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716323+0000","last_change":"2024-09-19T05:44:53.510921+0000","last_active":"2024-09-19T05:45:21.716323+0000","last_peered":"2024-09-19T05:45:21.716323+0000","last_clean":"2024-09-19T05:45:21.716323+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:21.716323+0000","last_undegraded":"2024-09-19T05:45:21.716323+0000","last_fullsized":"2024-09-19T05:45:21.716323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:39.530204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"84'146","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687436+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:45:21.687436+0000","last_peered":"2024-09-19T05:45:21.687436+0000","last_clean":"2024-09-19T05:45:21.687436+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:21.687436+0000","last_undegraded":"2024-09-19T05:45:21.687436+0000","last_fullsized":"2024-09-19T05:45:21.687436+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'171","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935092+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:45:21.935092+0000","last_peered":"2024-09-19T05:45:21.935092+0000","last_clean":"2024-09-19T05:45:21.935092+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:21.935092+0000","last_undegraded":"2024-09-19T05:45:21.935092+0000","last_fullsized":"2024-09-19T05:45:21.935092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"84'142","reported_seq":240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189849+0000","last_change":"2024-09-19T05:45:10.624968+0000","last_active":"2024-09-19T05:45:22.189849+0000","last_peered":"2024-09-19T05:45:22.189849+0000","last_clean":"2024-09-19T05:45:22.189849+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:22.189849+0000","last_undegraded":"2024-09-19T05:45:22.189849+0000","last_fullsized":"2024-09-19T05:45:22.189849+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:59.539763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"94'1809","reported_seq":1885,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687685+0000","last_change":"2024-09-19T05:44:55.560257+0000","last_active":"2024-09-19T05:45:21.687685+0000","last_peered":"2024-09-19T05:45:21.687685+0000","last_clean":"2024-09-19T05:45:21.687685+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:21.687685+0000","last_undegraded":"2024-09-19T05:45:21.687685+0000","last_fullsized":"2024-09-19T05:45:21.687685+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1809,"log_dups_size":0,"ondisk_log_size":1809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:25.647547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774094,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1808,"num_write_kb":5430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'154","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708831+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:45:21.708831+0000","last_peered":"2024-09-19T05:45:21.708831+0000","last_clean":"2024-09-19T05:45:21.708831+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:21.708831+0000","last_undegraded":"2024-09-19T05:45:21.708831+0000","last_fullsized":"2024-09-19T05:45:21.708831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716460+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:45:21.716460+0000","last_peered":"2024-09-19T05:45:21.716460+0000","last_clean":"2024-09-19T05:45:21.716460+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:21.716460+0000","last_undegraded":"2024-09-19T05:45:21.716460+0000","last_fullsized":"2024-09-19T05:45:21.716460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773054+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:45:21.773054+0000","last_peered":"2024-09-19T05:45:21.773054+0000","last_clean":"2024-09-19T05:45:21.773054+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:21.773054+0000","last_undegraded":"2024-09-19T05:45:21.773054+0000","last_fullsized":"2024-09-19T05:45:21.773054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'182","reported_seq":337,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687351+0000","last_change":"2024-09-19T05:44:59.989289+0000","last_active":"2024-09-19T05:45:21.687351+0000","last_peered":"2024-09-19T05:45:21.687351+0000","last_clean":"2024-09-19T05:45:21.687351+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:21.687351+0000","last_undegraded":"2024-09-19T05:45:21.687351+0000","last_fullsized":"2024-09-19T05:45:21.687351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:52.052457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716243+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:45:21.716243+0000","last_peered":"2024-09-19T05:45:21.716243+0000","last_clean":"2024-09-19T05:45:21.716243+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:21.716243+0000","last_undegraded":"2024-09-19T05:45:21.716243+0000","last_fullsized":"2024-09-19T05:45:21.716243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"84'177","reported_seq":359,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708983+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:45:21.708983+0000","last_peered":"2024-09-19T05:45:21.708983+0000","last_clean":"2024-09-19T05:45:21.708983+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:21.708983+0000","last_undegraded":"2024-09-19T05:45:21.708983+0000","last_fullsized":"2024-09-19T05:45:21.708983+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190091+0000","last_change":"2024-09-19T05:45:10.622931+0000","last_active":"2024-09-19T05:45:22.190091+0000","last_peered":"2024-09-19T05:45:22.190091+0000","last_clean":"2024-09-19T05:45:22.190091+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:22.190091+0000","last_undegraded":"2024-09-19T05:45:22.190091+0000","last_fullsized":"2024-09-19T05:45:22.190091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:34:06.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"84'241","reported_seq":413,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935402+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:45:21.935402+0000","last_peered":"2024-09-19T05:45:21.935402+0000","last_clean":"2024-09-19T05:45:21.935402+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:21.935402+0000","last_undegraded":"2024-09-19T05:45:21.935402+0000","last_fullsized":"2024-09-19T05:45:21.935402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688495+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:45:21.688495+0000","last_peered":"2024-09-19T05:45:21.688495+0000","last_clean":"2024-09-19T05:45:21.688495+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:21.688495+0000","last_undegraded":"2024-09-19T05:45:21.688495+0000","last_fullsized":"2024-09-19T05:45:21.688495+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:39.063921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"84'151","reported_seq":300,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688324+0000","last_change":"2024-09-19T05:44:59.517543+0000","last_active":"2024-09-19T05:45:21.688324+0000","last_peered":"2024-09-19T05:45:21.688324+0000","last_clean":"2024-09-19T05:45:21.688324+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:21.688324+0000","last_undegraded":"2024-09-19T05:45:21.688324+0000","last_fullsized":"2024-09-19T05:45:21.688324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.453343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"89'10","reported_seq":185,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687787+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:45:21.687787+0000","last_peered":"2024-09-19T05:45:21.687787+0000","last_clean":"2024-09-19T05:45:21.687787+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:21.687787+0000","last_undegraded":"2024-09-19T05:45:21.687787+0000","last_fullsized":"2024-09-19T05:45:21.687787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"84'157","reported_seq":328,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716300+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:45:21.716300+0000","last_peered":"2024-09-19T05:45:21.716300+0000","last_clean":"2024-09-19T05:45:21.716300+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:21.716300+0000","last_undegraded":"2024-09-19T05:45:21.716300+0000","last_fullsized":"2024-09-19T05:45:21.716300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"94'2017","reported_seq":2191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935286+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:45:21.935286+0000","last_peered":"2024-09-19T05:45:21.935286+0000","last_clean":"2024-09-19T05:45:21.935286+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:21.935286+0000","last_undegraded":"2024-09-19T05:45:21.935286+0000","last_fullsized":"2024-09-19T05:45:21.935286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688257+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:45:21.688257+0000","last_peered":"2024-09-19T05:45:21.688257+0000","last_clean":"2024-09-19T05:45:21.688257+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:21.688257+0000","last_undegraded":"2024-09-19T05:45:21.688257+0000","last_fullsized":"2024-09-19T05:45:21.688257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687838+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:45:21.687838+0000","last_peered":"2024-09-19T05:45:21.687838+0000","last_clean":"2024-09-19T05:45:21.687838+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:21.687838+0000","last_undegraded":"2024-09-19T05:45:21.687838+0000","last_fullsized":"2024-09-19T05:45:21.687838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"84'168","reported_seq":340,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773221+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:45:21.773221+0000","last_peered":"2024-09-19T05:45:21.773221+0000","last_clean":"2024-09-19T05:45:21.773221+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:21.773221+0000","last_undegraded":"2024-09-19T05:45:21.773221+0000","last_fullsized":"2024-09-19T05:45:21.773221+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716531+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:45:21.716531+0000","last_peered":"2024-09-19T05:45:21.716531+0000","last_clean":"2024-09-19T05:45:21.716531+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:21.716531+0000","last_undegraded":"2024-09-19T05:45:21.716531+0000","last_fullsized":"2024-09-19T05:45:21.716531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"84'186","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715983+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:45:21.715983+0000","last_peered":"2024-09-19T05:45:21.715983+0000","last_clean":"2024-09-19T05:45:21.715983+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:21.715983+0000","last_undegraded":"2024-09-19T05:45:21.715983+0000","last_fullsized":"2024-09-19T05:45:21.715983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687854+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:45:21.687854+0000","last_peered":"2024-09-19T05:45:21.687854+0000","last_clean":"2024-09-19T05:45:21.687854+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:21.687854+0000","last_undegraded":"2024-09-19T05:45:21.687854+0000","last_fullsized":"2024-09-19T05:45:21.687854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"84'191","reported_seq":283,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189767+0000","last_change":"2024-09-19T05:45:10.627674+0000","last_active":"2024-09-19T05:45:22.189767+0000","last_peered":"2024-09-19T05:45:22.189767+0000","last_clean":"2024-09-19T05:45:22.189767+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:22.189767+0000","last_undegraded":"2024-09-19T05:45:22.189767+0000","last_fullsized":"2024-09-19T05:45:22.189767+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:21.685895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"89'9","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708963+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:45:21.708963+0000","last_peered":"2024-09-19T05:45:21.708963+0000","last_clean":"2024-09-19T05:45:21.708963+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:21.708963+0000","last_undegraded":"2024-09-19T05:45:21.708963+0000","last_fullsized":"2024-09-19T05:45:21.708963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"84'209","reported_seq":380,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716252+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:45:21.716252+0000","last_peered":"2024-09-19T05:45:21.716252+0000","last_clean":"2024-09-19T05:45:21.716252+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:21.716252+0000","last_undegraded":"2024-09-19T05:45:21.716252+0000","last_fullsized":"2024-09-19T05:45:21.716252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688782+0000","last_change":"2024-09-19T05:44:53.065559+0000","last_active":"2024-09-19T05:45:21.688782+0000","last_peered":"2024-09-19T05:45:21.688782+0000","last_clean":"2024-09-19T05:45:21.688782+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:21.688782+0000","last_undegraded":"2024-09-19T05:45:21.688782+0000","last_fullsized":"2024-09-19T05:45:21.688782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:13.889455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935000+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:45:21.935000+0000","last_peered":"2024-09-19T05:45:21.935000+0000","last_clean":"2024-09-19T05:45:21.935000+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:21.935000+0000","last_undegraded":"2024-09-19T05:45:21.935000+0000","last_fullsized":"2024-09-19T05:45:21.935000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709231+0000","last_change":"2024-09-19T05:44:53.059769+0000","last_active":"2024-09-19T05:45:21.709231+0000","last_peered":"2024-09-19T05:45:21.709231+0000","last_clean":"2024-09-19T05:45:21.709231+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:21.709231+0000","last_undegraded":"2024-09-19T05:45:21.709231+0000","last_fullsized":"2024-09-19T05:45:21.709231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:03.909805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"84'153","reported_seq":231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687781+0000","last_change":"2024-09-19T05:44:15.787415+0000","last_active":"2024-09-19T05:45:21.687781+0000","last_peered":"2024-09-19T05:45:21.687781+0000","last_clean":"2024-09-19T05:45:21.687781+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:21.687781+0000","last_undegraded":"2024-09-19T05:45:21.687781+0000","last_fullsized":"2024-09-19T05:45:21.687781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:10.095369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":125,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688186+0000","last_change":"2024-09-19T05:44:53.058979+0000","last_active":"2024-09-19T05:45:21.688186+0000","last_peered":"2024-09-19T05:45:21.688186+0000","last_clean":"2024-09-19T05:45:21.688186+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:21.688186+0000","last_undegraded":"2024-09-19T05:45:21.688186+0000","last_fullsized":"2024-09-19T05:45:21.688186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:04.530725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'159","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686740+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:45:21.686740+0000","last_peered":"2024-09-19T05:45:21.686740+0000","last_clean":"2024-09-19T05:45:21.686740+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:21.686740+0000","last_undegraded":"2024-09-19T05:45:21.686740+0000","last_fullsized":"2024-09-19T05:45:21.686740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773397+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:45:21.773397+0000","last_peered":"2024-09-19T05:45:21.773397+0000","last_clean":"2024-09-19T05:45:21.773397+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:21.773397+0000","last_undegraded":"2024-09-19T05:45:21.773397+0000","last_fullsized":"2024-09-19T05:45:21.773397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"84'168","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716718+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:45:21.716718+0000","last_peered":"2024-09-19T05:45:21.716718+0000","last_clean":"2024-09-19T05:45:21.716718+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:21.716718+0000","last_undegraded":"2024-09-19T05:45:21.716718+0000","last_fullsized":"2024-09-19T05:45:21.716718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189880+0000","last_change":"2024-09-19T05:45:10.627237+0000","last_active":"2024-09-19T05:45:22.189880+0000","last_peered":"2024-09-19T05:45:22.189880+0000","last_clean":"2024-09-19T05:45:22.189880+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:22.189880+0000","last_undegraded":"2024-09-19T05:45:22.189880+0000","last_fullsized":"2024-09-19T05:45:22.189880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:04.886546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716440+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:45:21.716440+0000","last_peered":"2024-09-19T05:45:21.716440+0000","last_clean":"2024-09-19T05:45:21.716440+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:21.716440+0000","last_undegraded":"2024-09-19T05:45:21.716440+0000","last_fullsized":"2024-09-19T05:45:21.716440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"84'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708974+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:45:21.708974+0000","last_peered":"2024-09-19T05:45:21.708974+0000","last_clean":"2024-09-19T05:45:21.708974+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:21.708974+0000","last_undegraded":"2024-09-19T05:45:21.708974+0000","last_fullsized":"2024-09-19T05:45:21.708974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716388+0000","last_change":"2024-09-19T05:44:53.069283+0000","last_active":"2024-09-19T05:45:21.716388+0000","last_peered":"2024-09-19T05:45:21.716388+0000","last_clean":"2024-09-19T05:45:21.716388+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:21.716388+0000","last_undegraded":"2024-09-19T05:45:21.716388+0000","last_fullsized":"2024-09-19T05:45:21.716388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:51.209720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773046+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:45:21.773046+0000","last_peered":"2024-09-19T05:45:21.773046+0000","last_clean":"2024-09-19T05:45:21.773046+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:21.773046+0000","last_undegraded":"2024-09-19T05:45:21.773046+0000","last_fullsized":"2024-09-19T05:45:21.773046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687539+0000","last_change":"2024-09-19T05:44:53.068415+0000","last_active":"2024-09-19T05:45:21.687539+0000","last_peered":"2024-09-19T05:45:21.687539+0000","last_clean":"2024-09-19T05:45:21.687539+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:21.687539+0000","last_undegraded":"2024-09-19T05:45:21.687539+0000","last_fullsized":"2024-09-19T05:45:21.687539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:16.609871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"84'157","reported_seq":310,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687083+0000","last_change":"2024-09-19T05:44:58.820818+0000","last_active":"2024-09-19T05:45:21.687083+0000","last_peered":"2024-09-19T05:45:21.687083+0000","last_clean":"2024-09-19T05:45:21.687083+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:21.687083+0000","last_undegraded":"2024-09-19T05:45:21.687083+0000","last_fullsized":"2024-09-19T05:45:21.687083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:32.427214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716567+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:45:21.716567+0000","last_peered":"2024-09-19T05:45:21.716567+0000","last_clean":"2024-09-19T05:45:21.716567+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:21.716567+0000","last_undegraded":"2024-09-19T05:45:21.716567+0000","last_fullsized":"2024-09-19T05:45:21.716567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"84'170","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.686759+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:45:21.686759+0000","last_peered":"2024-09-19T05:45:21.686759+0000","last_clean":"2024-09-19T05:45:21.686759+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:21.686759+0000","last_undegraded":"2024-09-19T05:45:21.686759+0000","last_fullsized":"2024-09-19T05:45:21.686759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:33.919945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687970+0000","last_change":"2024-09-19T05:44:53.063483+0000","last_active":"2024-09-19T05:45:21.687970+0000","last_peered":"2024-09-19T05:45:21.687970+0000","last_clean":"2024-09-19T05:45:21.687970+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:21.687970+0000","last_undegraded":"2024-09-19T05:45:21.687970+0000","last_fullsized":"2024-09-19T05:45:21.687970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:40.306784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"84'155","reported_seq":327,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935176+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:45:21.935176+0000","last_peered":"2024-09-19T05:45:21.935176+0000","last_clean":"2024-09-19T05:45:21.935176+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:21.935176+0000","last_undegraded":"2024-09-19T05:45:21.935176+0000","last_fullsized":"2024-09-19T05:45:21.935176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716905+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:45:21.716905+0000","last_peered":"2024-09-19T05:45:21.716905+0000","last_clean":"2024-09-19T05:45:21.716905+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:21.716905+0000","last_undegraded":"2024-09-19T05:45:21.716905+0000","last_fullsized":"2024-09-19T05:45:21.716905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"84'153","reported_seq":424,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773336+0000","last_change":"2024-09-19T05:44:55.675882+0000","last_active":"2024-09-19T05:45:21.773336+0000","last_peered":"2024-09-19T05:45:21.773336+0000","last_clean":"2024-09-19T05:45:21.773336+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:21.773336+0000","last_undegraded":"2024-09-19T05:45:21.773336+0000","last_fullsized":"2024-09-19T05:45:21.773336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:04.593886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709321+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:45:21.709321+0000","last_peered":"2024-09-19T05:45:21.709321+0000","last_clean":"2024-09-19T05:45:21.709321+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:21.709321+0000","last_undegraded":"2024-09-19T05:45:21.709321+0000","last_fullsized":"2024-09-19T05:45:21.709321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"84'161","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709277+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:45:21.709277+0000","last_peered":"2024-09-19T05:45:21.709277+0000","last_clean":"2024-09-19T05:45:21.709277+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:21.709277+0000","last_undegraded":"2024-09-19T05:45:21.709277+0000","last_fullsized":"2024-09-19T05:45:21.709277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"85'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716921+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:45:21.716921+0000","last_peered":"2024-09-19T05:45:21.716921+0000","last_clean":"2024-09-19T05:45:21.716921+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:21.716921+0000","last_undegraded":"2024-09-19T05:45:21.716921+0000","last_fullsized":"2024-09-19T05:45:21.716921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"84'176","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709220+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:45:21.709220+0000","last_peered":"2024-09-19T05:45:21.709220+0000","last_clean":"2024-09-19T05:45:21.709220+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:21.709220+0000","last_undegraded":"2024-09-19T05:45:21.709220+0000","last_fullsized":"2024-09-19T05:45:21.709220+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687243+0000","last_change":"2024-09-19T05:42:31.986507+0000","last_active":"2024-09-19T05:45:21.687243+0000","last_peered":"2024-09-19T05:45:21.687243+0000","last_clean":"2024-09-19T05:45:21.687243+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:21.687243+0000","last_undegraded":"2024-09-19T05:45:21.687243+0000","last_fullsized":"2024-09-19T05:45:21.687243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"84'160","reported_seq":331,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716327+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:45:21.716327+0000","last_peered":"2024-09-19T05:45:21.716327+0000","last_clean":"2024-09-19T05:45:21.716327+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:21.716327+0000","last_undegraded":"2024-09-19T05:45:21.716327+0000","last_fullsized":"2024-09-19T05:45:21.716327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709021+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:45:21.709021+0000","last_peered":"2024-09-19T05:45:21.709021+0000","last_clean":"2024-09-19T05:45:21.709021+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:21.709021+0000","last_undegraded":"2024-09-19T05:45:21.709021+0000","last_fullsized":"2024-09-19T05:45:21.709021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"84'164","reported_seq":336,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708902+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:45:21.708902+0000","last_peered":"2024-09-19T05:45:21.708902+0000","last_clean":"2024-09-19T05:45:21.708902+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:21.708902+0000","last_undegraded":"2024-09-19T05:45:21.708902+0000","last_fullsized":"2024-09-19T05:45:21.708902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773238+0000","last_change":"2024-09-19T05:44:53.065594+0000","last_active":"2024-09-19T05:45:21.773238+0000","last_peered":"2024-09-19T05:45:21.773238+0000","last_clean":"2024-09-19T05:45:21.773238+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:21.773238+0000","last_undegraded":"2024-09-19T05:45:21.773238+0000","last_fullsized":"2024-09-19T05:45:21.773238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:47.254673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"84'156","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687718+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:45:21.687718+0000","last_peered":"2024-09-19T05:45:21.687718+0000","last_clean":"2024-09-19T05:45:21.687718+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:21.687718+0000","last_undegraded":"2024-09-19T05:45:21.687718+0000","last_fullsized":"2024-09-19T05:45:21.687718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"59'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716236+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:45:21.716236+0000","last_peered":"2024-09-19T05:45:21.716236+0000","last_clean":"2024-09-19T05:45:21.716236+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:21.716236+0000","last_undegraded":"2024-09-19T05:45:21.716236+0000","last_fullsized":"2024-09-19T05:45:21.716236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"84'149","reported_seq":330,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716425+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:45:21.716425+0000","last_peered":"2024-09-19T05:45:21.716425+0000","last_clean":"2024-09-19T05:45:21.716425+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:21.716425+0000","last_undegraded":"2024-09-19T05:45:21.716425+0000","last_fullsized":"2024-09-19T05:45:21.716425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"84'190","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.715986+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:45:21.715986+0000","last_peered":"2024-09-19T05:45:21.715986+0000","last_clean":"2024-09-19T05:45:21.715986+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:21.715986+0000","last_undegraded":"2024-09-19T05:45:21.715986+0000","last_fullsized":"2024-09-19T05:45:21.715986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":194,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688923+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:45:21.688923+0000","last_peered":"2024-09-19T05:45:21.688923+0000","last_clean":"2024-09-19T05:45:21.688923+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:21.688923+0000","last_undegraded":"2024-09-19T05:45:21.688923+0000","last_fullsized":"2024-09-19T05:45:21.688923+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:59.847187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"84'153","reported_seq":266,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189956+0000","last_change":"2024-09-19T05:45:10.627058+0000","last_active":"2024-09-19T05:45:22.189956+0000","last_peered":"2024-09-19T05:45:22.189956+0000","last_clean":"2024-09-19T05:45:22.189956+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:22.189956+0000","last_undegraded":"2024-09-19T05:45:22.189956+0000","last_fullsized":"2024-09-19T05:45:22.189956+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:52.373727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"70'4","reported_seq":196,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688300+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:45:21.688300+0000","last_peered":"2024-09-19T05:45:21.688300+0000","last_clean":"2024-09-19T05:45:21.688300+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:21.688300+0000","last_undegraded":"2024-09-19T05:45:21.688300+0000","last_fullsized":"2024-09-19T05:45:21.688300+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:52.607339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"84'154","reported_seq":314,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688325+0000","last_change":"2024-09-19T05:44:57.957776+0000","last_active":"2024-09-19T05:45:21.688325+0000","last_peered":"2024-09-19T05:45:21.688325+0000","last_clean":"2024-09-19T05:45:21.688325+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:21.688325+0000","last_undegraded":"2024-09-19T05:45:21.688325+0000","last_fullsized":"2024-09-19T05:45:21.688325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:37.587819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":156,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688211+0000","last_change":"2024-09-19T05:44:53.061701+0000","last_active":"2024-09-19T05:45:21.688211+0000","last_peered":"2024-09-19T05:45:21.688211+0000","last_clean":"2024-09-19T05:45:21.688211+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:21.688211+0000","last_undegraded":"2024-09-19T05:45:21.688211+0000","last_fullsized":"2024-09-19T05:45:21.688211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:26.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"84'176","reported_seq":321,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688312+0000","last_change":"2024-09-19T05:44:57.503018+0000","last_active":"2024-09-19T05:45:21.688312+0000","last_peered":"2024-09-19T05:45:21.688312+0000","last_clean":"2024-09-19T05:45:21.688312+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:21.688312+0000","last_undegraded":"2024-09-19T05:45:21.688312+0000","last_fullsized":"2024-09-19T05:45:21.688312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:44.482643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709387+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:45:21.709387+0000","last_peered":"2024-09-19T05:45:21.709387+0000","last_clean":"2024-09-19T05:45:21.709387+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:21.709387+0000","last_undegraded":"2024-09-19T05:45:21.709387+0000","last_fullsized":"2024-09-19T05:45:21.709387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":10755,"ondisk_log_size":10755,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970373,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3906,"num_write_kb":11500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15940746,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3893,"ondisk_log_size":3893,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":5272,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":5398528,"data_stored":4749255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335296,"statfs":{"total":96636764160,"available":96599343104,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542185,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":4984832,"data_stored":4334380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449096,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46800,"kb_used_data":1724,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325040,"statfs":{"total":96636764160,"available":96588840960,"internally_reserved":0,"allocated":1765376,"data_stored":4919338,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50368,"kb_used_data":5292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321472,"statfs":{"total":96636764160,"available":96585187328,"internally_reserved":0,"allocated":5419008,"data_stored":4755934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40696,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331144,"statfs":{"total":96636764160,"available":96595091456,"internally_reserved":0,"allocated":1212416,"data_stored":560868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40748,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331092,"statfs":{"total":96636764160,"available":96595038208,"internally_reserved":0,"allocated":1261568,"data_stored":1150724,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40700,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331140,"statfs":{"total":96636764160,"available":96595087360,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774691,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:45:27.820 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:45:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.956+0000 7f3de2c4c640 1 Processor -- start 2024-09-19T05:45:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.956+0000 7f3de2c4c640 1 -- start start 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3de2c4c640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ddc0d0ac0 0x7f3ddc0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3de2c4c640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ddc0cb400 con 0x7f3ddc0d06e0 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3de2c4c640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ddc0cb6e0 con 0x7f3ddc15eb50 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3de2c4c640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3ddc0cc970 con 0x7f3ddc0d0ac0 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3de09e8640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3ddc0d06e0 0x7f3ddc154af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54174/0 (socket says 172.21.15.2:54174) 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3de09e8640 1 -- 172.21.15.2:0/3571758184 learned_addr learned my addr 172.21.15.2:0/3571758184 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/3571758184 >> v2:172.21.15.123:3300/0 conn(0x7f3ddc0d0ac0 0x7f3ddc0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996345282 0 0) 0x7f3ddc0cb400 con 0x7f3ddc0d06e0 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dcc003520 con 0x7f3ddc0d06e0 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 111998403 0 0) 0x7f3dcc003520 con 0x7f3ddc0d06e0 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 >> v2:172.21.15.123:3300/0 conn(0x7f3ddc0d0ac0 msgr2=0x7f3ddc0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 --2- 172.21.15.2:0/3571758184 >> v2:172.21.15.123:3300/0 conn(0x7f3ddc0d0ac0 0x7f3ddc0cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 >> v1:172.21.15.123:6789/0 conn(0x7f3ddc15eb50 legacy=0x7f3ddc0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc0ccc50 con 0x7f3ddc0d06e0 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.957+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f3dd0001670 con 0x7f3ddc0d06e0 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 >> v1:172.21.15.2:6789/0 conn(0x7f3ddc0d06e0 legacy=0x7f3ddc154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddb7fe640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dcc003a20 0x7f3dcc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddb7fe640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dcc004800 0x7f3dcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddb7fe640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc0051c0 0x7f3dcc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc0cc970 con 0x7f3dcc004800 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3dcc00c0f0 con 0x7f3dcc003a20 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3dcc00c2f0 con 0x7f3dcc0051c0 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dcc004800 0x7f3dcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3de09e8640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dcc003a20 0x7f3dcc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3de11e9640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc0051c0 0x7f3dcc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddbfff640 1 -- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc0051c0 msgr2=0x7f3dcc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc0051c0 0x7f3dcc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddbfff640 1 -- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dcc003a20 msgr2=0x7f3dcc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:27.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dcc003a20 0x7f3dcc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3ddbfff640 1 -- 172.21.15.2:0/3571758184 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dcc003520 con 0x7f3dcc004800 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3de11e9640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc0051c0 0x7f3dcc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.958+0000 7f3de11e9640 1 -- 172.21.15.2:0/3571758184 reap_dead start 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dcc004800 0x7f3dcc004c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009a90 tx=0x7f3dcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68f60ff04173ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8036070 con 0x7f3dcc004800 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc8031400 con 0x7f3dcc004800 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3ddb7fe640 1 -- 172.21.15.2:0/3571758184 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8009690 con 0x7f3dcc004800 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3de2c4c640 1 -- 172.21.15.2:0/3571758184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dcc004800 msgr2=0x7f3dcc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dcc004800 0x7f3dcc004c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009a90 tx=0x7f3dcc005830 comp rx=0 tx=0).stop 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3de2c4c640 1 -- 172.21.15.2:0/3571758184 shutdown_connections 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/3571758184 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dcc004800 0x7f3dcc004c80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3de2c4c640 1 -- 172.21.15.2:0/3571758184 >> 172.21.15.2:0/3571758184 conn(0x7f3ddc06d740 msgr2=0x7f3ddc073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:27.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3de2c4c640 1 -- 172.21.15.2:0/3571758184 shutdown_connections 2024-09-19T05:45:27.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.959+0000 7f3de2c4c640 1 -- 172.21.15.2:0/3571758184 wait complete. 2024-09-19T05:45:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.961+0000 7f3de2c4c640 1 Processor -- start 2024-09-19T05:45:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.961+0000 7f3de2c4c640 1 -- start start 2024-09-19T05:45:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.961+0000 7f3de2c4c640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc004800 0x7f3ddc1f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.961+0000 7f3de2c4c640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ddc14c750 0x7f3ddc0b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.961+0000 7f3de2c4c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc14ad90 0x7f3ddc14b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.961+0000 7f3de2c4c640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc0c5830 con 0x7f3ddc14c750 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.961+0000 7f3de2c4c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc15efc0 con 0x7f3ddc14ad90 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3de2c4c640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc0cbb40 con 0x7f3dcc004800 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ddc14c750 0x7f3ddc0b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ddc14c750 0x7f3ddc0b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54902/0 (socket says 172.21.15.2:54902) 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 -- 172.21.15.2:0/193245489 learned_addr learned my addr 172.21.15.2:0/193245489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3de09e8640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc004800 0x7f3ddc1f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3de11e9640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc14ad90 0x7f3ddc14b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 -- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc004800 msgr2=0x7f3ddc1f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc004800 0x7f3ddc1f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 -- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc14ad90 msgr2=0x7f3ddc14b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:27.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc14ad90 0x7f3ddc14b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 -- 172.21.15.2:0/193245489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc0b4c00 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3de11e9640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc14ad90 0x7f3ddc14b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3ddbfff640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ddc14c750 0x7f3ddc0b46c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8034040 tx=0x7f3dc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3dd97fa640 1 -- 172.21.15.2:0/193245489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc803f020 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3dd97fa640 1 -- 172.21.15.2:0/193245489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc8037040 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3dd97fa640 1 -- 172.21.15.2:0/193245489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8036070 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc0c8080 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.962+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc0c8320 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.963+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc0cbe70 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.964+0000 7f3dd97fa640 1 -- 172.21.15.2:0/193245489 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3dc8042020 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.964+0000 7f3dd97fa640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3dc006c2b0 0x7f3dc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.964+0000 7f3dd97fa640 1 -- 172.21.15.2:0/193245489 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f3dc80ab640 con 0x7f3ddc14c750 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.964+0000 7f3de09e8640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3dc006c2b0 0x7f3dc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:27.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.965+0000 7f3de09e8640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3dc006c2b0 0x7f3dc006e770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0007c30 tx=0x7f3dd0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:27.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:27.968+0000 7f3dd97fa640 1 -- 172.21.15.2:0/193245489 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f3dc8049030 con 0x7f3ddc14c750 2024-09-19T05:45:28.149 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.147+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ddc0ca8a0 con 0x7f3ddc14c750 2024-09-19T05:45:28.150 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.148+0000 7f3dd97fa640 1 -- 172.21.15.2:0/193245489 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16762 (secure 0 0 0) 0x7f3dc8079d80 con 0x7f3ddc14c750 2024-09-19T05:45:28.151 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:28.151 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":102,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:45:21.682862+0000","last_up_change":"2024-09-19T05:44:52.039950+0000","last_in_change":"2024-09-19T05:43:19.394170+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":78,"up_thru":82,"down_at":68,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6801","nonce":2989037926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6803","nonce":2989037926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6807","nonce":2989037926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6805","nonce":2989037926}]},"public_addr":"172.21.15.123:6801/2989037926","cluster_addr":"172.21.15.123:6803/2989037926","heartbeat_back_addr":"172.21.15.123:6807/2989037926","heartbeat_front_addr":"172.21.15.123:6805/2989037926","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":68},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2a","osds":[5,0]}],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3dc006c2b0 msgr2=0x7f3dc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3dc006c2b0 0x7f3dc006e770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0007c30 tx=0x7f3dd0009040 comp rx=0 tx=0).stop 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ddc14c750 msgr2=0x7f3ddc0b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ddc14c750 0x7f3ddc0b46c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8034040 tx=0x7f3dc8004730 comp rx=0 tx=0).stop 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 shutdown_connections 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f3dc006c2b0 0x7f3dc006e770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ddc14ad90 0x7f3ddc14b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ddc14c750 0x7f3ddc0b46c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:28.153 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 --2- 172.21.15.2:0/193245489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3dcc004800 0x7f3ddc1f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 >> 172.21.15.2:0/193245489 conn(0x7f3ddc06d740 msgr2=0x7f3ddc153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 shutdown_connections 2024-09-19T05:45:28.154 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:28.150+0000 7f3de2c4c640 1 -- 172.21.15.2:0/193245489 wait complete. 2024-09-19T05:45:29.164 INFO:tasks.ceph:Scrubbing osd.0 2024-09-19T05:45:29.165 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:29.285 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.283+0000 7ffa8f465640 1 Processor -- start 2024-09-19T05:45:29.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.283+0000 7ffa8f465640 1 -- start start 2024-09-19T05:45:29.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.284+0000 7ffa8f465640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ffa880cb620 0x7ffa8806d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.284+0000 7ffa8f465640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8815b2c0 con 0x7ffa88154dc0 2024-09-19T05:45:29.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.284+0000 7ffa8f465640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8815b5a0 con 0x7ffa880cbf40 2024-09-19T05:45:29.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.284+0000 7ffa8f465640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ffa8806cdf0 con 0x7ffa880cb620 2024-09-19T05:45:29.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.284+0000 7ffa8ca00640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ffa880cb620 0x7ffa8806d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.287 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.285+0000 7ffa8ca00640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ffa880cb620 0x7ffa8806d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54916/0 (socket says 172.21.15.2:54916) 2024-09-19T05:45:29.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.285+0000 7ffa8ca00640 1 -- 172.21.15.2:0/24541011 learned_addr learned my addr 172.21.15.2:0/24541011 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:29.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.285+0000 7ffa8ca00640 1 -- 172.21.15.2:0/24541011 >> v1:172.21.15.123:6789/0 conn(0x7ffa880cbf40 legacy=0x7ffa881653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.285+0000 7ffa8ca00640 1 -- 172.21.15.2:0/24541011 >> v1:172.21.15.2:6789/0 conn(0x7ffa88154dc0 legacy=0x7ffa8815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.285+0000 7ffa87fff640 1 -- 172.21.15.2:0/24541011 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507231581 0 0) 0x7ffa8815b2c0 con 0x7ffa88154dc0 2024-09-19T05:45:29.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.285+0000 7ffa8ca00640 1 -- 172.21.15.2:0/24541011 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8815b880 con 0x7ffa880cb620 2024-09-19T05:45:29.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.286+0000 7ffa8ca00640 1 --2- 172.21.15.2:0/24541011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa880cb620 0x7ffa8806d280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009870 tx=0x7ffa7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867fb020b80be066 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.286+0000 7ffa87fff640 1 -- 172.21.15.2:0/24541011 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c037070 con 0x7ffa880cb620 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.286+0000 7ffa87fff640 1 -- 172.21.15.2:0/24541011 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7c032440 con 0x7ffa880cb620 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.286+0000 7ffa87fff640 1 -- 172.21.15.2:0/24541011 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c032810 con 0x7ffa880cb620 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.286+0000 7ffa8f465640 1 -- 172.21.15.2:0/24541011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa880cb620 msgr2=0x7ffa8806d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.286+0000 7ffa8f465640 1 --2- 172.21.15.2:0/24541011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa880cb620 0x7ffa8806d280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009870 tx=0x7ffa7c02a270 comp rx=0 tx=0).stop 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.287+0000 7ffa8f465640 1 -- 172.21.15.2:0/24541011 shutdown_connections 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.287+0000 7ffa8f465640 1 --2- 172.21.15.2:0/24541011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa880cb620 0x7ffa8806d280 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.287+0000 7ffa8f465640 1 -- 172.21.15.2:0/24541011 >> 172.21.15.2:0/24541011 conn(0x7ffa88067950 msgr2=0x7ffa88067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.287+0000 7ffa8f465640 1 -- 172.21.15.2:0/24541011 shutdown_connections 2024-09-19T05:45:29.290 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.287+0000 7ffa8f465640 1 -- 172.21.15.2:0/24541011 wait complete. 2024-09-19T05:45:29.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.288+0000 7ffa8f465640 1 Processor -- start 2024-09-19T05:45:29.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.288+0000 7ffa8f465640 1 -- start start 2024-09-19T05:45:29.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8f465640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa880cb620 0x7ffa881e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8f465640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa880cbf40 0x7ffa8814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8f465640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa88154dc0 0x7ffa881512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.293 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8f465640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8806d7c0 con 0x7ffa88154dc0 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8f465640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa880b34e0 con 0x7ffa880cbf40 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8f465640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ffa881551a0 con 0x7ffa880cb620 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa88154dc0 0x7ffa881512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa88154dc0 0x7ffa881512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54928/0 (socket says 172.21.15.2:54928) 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 -- 172.21.15.2:0/3796840289 learned_addr learned my addr 172.21.15.2:0/3796840289 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8ca00640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa880cbf40 0x7ffa8814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8d201640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa880cb620 0x7ffa881e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 -- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa880cb620 msgr2=0x7ffa881e1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa880cb620 0x7ffa881e1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 -- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa880cbf40 msgr2=0x7ffa8814ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.294 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa880cbf40 0x7ffa8814ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 -- 172.21.15.2:0/3796840289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa881f0db0 con 0x7ffa88154dc0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8d201640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa880cb620 0x7ffa881e1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa8da02640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa88154dc0 0x7ffa881512f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7d0 tx=0x7ffa7800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.289+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78003b70 con 0x7ffa88154dc0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.290+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa78003d30 con 0x7ffa88154dc0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.290+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800aa90 con 0x7ffa88154dc0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.290+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8814f4c0 con 0x7ffa88154dc0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.290+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa880b38c0 con 0x7ffa88154dc0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.291+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa881f0b60 con 0x7ffa88154dc0 2024-09-19T05:45:29.295 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.291+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffa7800ac30 con 0x7ffa88154dc0 2024-09-19T05:45:29.296 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa85ffb640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ffa7406c2b0 0x7ffa7406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7ffa7808b340 con 0x7ffa88154dc0 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa8d201640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ffa7406c2b0 0x7ffa7406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa85ffb640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ffa74074570 0x7ffa740769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa78010e50 con 0x7ffa74074570 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7ffa78092050 con 0x7ffa88154dc0 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa8ca00640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ffa74074570 0x7ffa740769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.292+0000 7ffa8d201640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ffa7406c2b0 0x7ffa7406e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000f60 tx=0x7ffa80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.293+0000 7ffa8ca00640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ffa74074570 0x7ffa740769d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.297 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.293+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== osd.0 v2:172.21.15.2:6808/288510134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa78010e50 con 0x7ffa74074570 2024-09-19T05:45:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.306+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 --> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffa881f1090 con 0x7ffa74074570 2024-09-19T05:45:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.306+0000 7ffa85ffb640 1 -- 172.21.15.2:0/3796840289 <== osd.0 v2:172.21.15.2:6808/288510134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffa881f1090 con 0x7ffa74074570 2024-09-19T05:45:29.309 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:29.309 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:29.309 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:29.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.306+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ffa74074570 msgr2=0x7ffa740769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.306+0000 7ffa8f465640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:6808/288510134,v1:172.21.15.2:6809/288510134] conn(0x7ffa74074570 0x7ffa740769d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ffa7406c2b0 msgr2=0x7ffa7406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ffa7406c2b0 0x7ffa7406e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa80000f60 tx=0x7ffa80009040 comp rx=0 tx=0).stop 2024-09-19T05:45:29.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa88154dc0 msgr2=0x7ffa881512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.310 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 --2- 172.21.15.2:0/3796840289 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa88154dc0 0x7ffa881512f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7d0 tx=0x7ffa7800cc90 comp rx=0 tx=0).stop 2024-09-19T05:45:29.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8da02640 1 -- 172.21.15.2:0/3796840289 reap_dead start 2024-09-19T05:45:29.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 shutdown_connections 2024-09-19T05:45:29.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 >> 172.21.15.2:0/3796840289 conn(0x7ffa88067950 msgr2=0x7ffa88151dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:29.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 shutdown_connections 2024-09-19T05:45:29.311 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.307+0000 7ffa8f465640 1 -- 172.21.15.2:0/3796840289 wait complete. 2024-09-19T05:45:29.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-19T05:45:29.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.421+0000 7f2252c8d640 1 Processor -- start 2024-09-19T05:45:29.423 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.421+0000 7f2252c8d640 1 -- start start 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f2252c8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c155000 0x7f224c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f2252c8d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f224c0caf40 0x7f224c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f2252c8d640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c0ccbc0 con 0x7f224c154c20 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f2252c8d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f224c0cc600 con 0x7f224c0caf40 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f2252c8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0cc8e0 con 0x7f224c155000 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f2250a29640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f224c154c20 0x7f224c073340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54184/0 (socket says 172.21.15.2:54184) 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f2250a29640 1 -- 172.21.15.2:0/3604733195 learned_addr learned my addr 172.21.15.2:0/3604733195 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f225122a640 1 --2- 172.21.15.2:0/3604733195 >> v2:172.21.15.2:3300/0 conn(0x7f224c0caf40 0x7f224c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f224bfff640 1 --2- 172.21.15.2:0/3604733195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c155000 0x7f224c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.425 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f224b7fe640 1 -- 172.21.15.2:0/3604733195 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461432982 0 0) 0x7f224c0ccbc0 con 0x7f224c154c20 2024-09-19T05:45:29.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f224b7fe640 1 -- 172.21.15.2:0/3604733195 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f223c003520 con 0x7f224c154c20 2024-09-19T05:45:29.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f225122a640 1 -- 172.21.15.2:0/3604733195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c155000 msgr2=0x7f224c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:29.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f225122a640 1 --2- 172.21.15.2:0/3604733195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c155000 0x7f224c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f225122a640 1 -- 172.21.15.2:0/3604733195 >> v1:172.21.15.2:6789/0 conn(0x7f224c154c20 legacy=0x7f224c073340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f225122a640 1 -- 172.21.15.2:0/3604733195 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c0cbc60 con 0x7f224c0caf40 2024-09-19T05:45:29.426 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.422+0000 7f224bfff640 1 --2- 172.21.15.2:0/3604733195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c155000 0x7f224c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:29.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.423+0000 7f225122a640 1 --2- 172.21.15.2:0/3604733195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c0caf40 0x7f224c15aae0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2244009d90 tx=0x7f224402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208b3a63a7206ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.423+0000 7f224b7fe640 1 -- 172.21.15.2:0/3604733195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244007740 con 0x7f224c0caf40 2024-09-19T05:45:29.427 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.423+0000 7f224b7fe640 1 -- 172.21.15.2:0/3604733195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22440078e0 con 0x7f224c0caf40 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.423+0000 7f224b7fe640 1 -- 172.21.15.2:0/3604733195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244007cb0 con 0x7f224c0caf40 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 -- 172.21.15.2:0/3604733195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c0caf40 msgr2=0x7f224c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 --2- 172.21.15.2:0/3604733195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c0caf40 0x7f224c15aae0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2244009d90 tx=0x7f224402a660 comp rx=0 tx=0).stop 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 -- 172.21.15.2:0/3604733195 shutdown_connections 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 --2- 172.21.15.2:0/3604733195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c0caf40 0x7f224c15aae0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 --2- 172.21.15.2:0/3604733195 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c155000 0x7f224c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 -- 172.21.15.2:0/3604733195 >> 172.21.15.2:0/3604733195 conn(0x7f224c06d0e0 msgr2=0x7f224c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 -- 172.21.15.2:0/3604733195 shutdown_connections 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.424+0000 7f2252c8d640 1 -- 172.21.15.2:0/3604733195 wait complete. 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.425+0000 7f2252c8d640 1 Processor -- start 2024-09-19T05:45:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.425+0000 7f2252c8d640 1 -- start start 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f2252c8d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c0caf40 0x7f224c1e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f2252c8d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c154c20 0x7f224c153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f2252c8d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f224c155000 0x7f224c0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f2252c8d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f224c073a50 con 0x7f224c154c20 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f2252c8d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f224c0cb430 con 0x7f224c155000 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f2252c8d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f224c073bd0 con 0x7f224c0caf40 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c154c20 0x7f224c153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c154c20 0x7f224c153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54952/0 (socket says 172.21.15.2:54952) 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 -- 172.21.15.2:0/1458255906 learned_addr learned my addr 172.21.15.2:0/1458255906 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f2250a29640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c0caf40 0x7f224c1e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f225122a640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f224c155000 0x7f224c0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 -- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c0caf40 msgr2=0x7f224c1e0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c0caf40 0x7f224c1e0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 -- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f224c155000 msgr2=0x7f224c0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f224c155000 0x7f224c0c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.426+0000 7f224bfff640 1 -- 172.21.15.2:0/1458255906 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1e3890 con 0x7f224c154c20 2024-09-19T05:45:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.427+0000 7f224bfff640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c154c20 0x7f224c153c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f223800d800 tx=0x7f223800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.427+0000 7f22497fa640 1 -- 172.21.15.2:0/1458255906 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238005990 con 0x7f224c154c20 2024-09-19T05:45:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.427+0000 7f22497fa640 1 -- 172.21.15.2:0/1458255906 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2238005b30 con 0x7f224c154c20 2024-09-19T05:45:29.432 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.427+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c1e3b50 con 0x7f224c154c20 2024-09-19T05:45:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.427+0000 7f22497fa640 1 -- 172.21.15.2:0/1458255906 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380096a0 con 0x7f224c154c20 2024-09-19T05:45:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.427+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1e5b00 con 0x7f224c154c20 2024-09-19T05:45:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.428+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c0c5770 con 0x7f224c154c20 2024-09-19T05:45:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.428+0000 7f22497fa640 1 -- 172.21.15.2:0/1458255906 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2238009840 con 0x7f224c154c20 2024-09-19T05:45:29.433 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.429+0000 7f22497fa640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f223006c2b0 0x7f223006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.429+0000 7f22497fa640 1 -- 172.21.15.2:0/1458255906 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f223808b7b0 con 0x7f224c154c20 2024-09-19T05:45:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.429+0000 7f2250a29640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f223006c2b0 0x7f223006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.434 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.430+0000 7f2250a29640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f223006c2b0 0x7f223006e770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2240005880 tx=0x7f2240009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.435 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.433+0000 7f22497fa640 1 -- 172.21.15.2:0/1458255906 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f2238059e20 con 0x7f224c154c20 2024-09-19T05:45:29.612 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.610+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f224c0b32c0 con 0x7f223006c2b0 2024-09-19T05:45:29.613 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.611+0000 7f22497fa640 1 -- 172.21.15.2:0/1458255906 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f224c0b32c0 con 0x7f223006c2b0 2024-09-19T05:45:29.613 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 0 to deep-scrub 2024-09-19T05:45:29.615 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f223006c2b0 msgr2=0x7f223006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f223006c2b0 0x7f223006e770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2240005880 tx=0x7f2240009040 comp rx=0 tx=0).stop 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c154c20 msgr2=0x7f224c153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c154c20 0x7f224c153c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f223800d800 tx=0x7f223800dcc0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 shutdown_connections 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f223006c2b0 0x7f223006e770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f224c155000 0x7f224c0c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f224c154c20 0x7f224c153c80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 --2- 172.21.15.2:0/1458255906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f224c0caf40 0x7f224c1e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 >> 172.21.15.2:0/1458255906 conn(0x7f224c06d0e0 msgr2=0x7f224c0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 shutdown_connections 2024-09-19T05:45:29.616 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.613+0000 7f2252c8d640 1 -- 172.21.15.2:0/1458255906 wait complete. 2024-09-19T05:45:29.626 INFO:tasks.ceph:Scrubbing osd.1 2024-09-19T05:45:29.626 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:29.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.732+0000 7fd886b9f640 1 Processor -- start 2024-09-19T05:45:29.734 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.732+0000 7fd886b9f640 1 -- start start 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.732+0000 7fd886b9f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 0x7fd8800cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.732+0000 7fd886b9f640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd8800cc940 0x7fd8800d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.732+0000 7fd886b9f640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd88015aef0 con 0x7fd8800d09a0 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd886b9f640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd88015ba70 con 0x7fd8800cc940 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd886b9f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd88015b1d0 con 0x7fd880154e60 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd87ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd8800cc940 0x7fd8800d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd87ffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd8800cc940 0x7fd8800d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54968/0 (socket says 172.21.15.2:54968) 2024-09-19T05:45:29.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd87ffff640 1 -- 172.21.15.2:0/472364858 learned_addr learned my addr 172.21.15.2:0/472364858 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd88493b640 1 --2- 172.21.15.2:0/472364858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 0x7fd8800cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd87ffff640 1 -- 172.21.15.2:0/472364858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 msgr2=0x7fd8800cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd87ffff640 1 --2- 172.21.15.2:0/472364858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 0x7fd8800cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd87ffff640 1 -- 172.21.15.2:0/472364858 >> v1:172.21.15.123:6789/0 conn(0x7fd8800d09a0 legacy=0x7fd88015e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.733+0000 7fd87ffff640 1 -- 172.21.15.2:0/472364858 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88015b4b0 con 0x7fd8800cc940 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd87ffff640 1 --2- 172.21.15.2:0/472364858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800cc940 0x7fd8800d0460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd870009e50 tx=0x7fd87002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1a06e05b0ea3bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd87f7fe640 1 -- 172.21.15.2:0/472364858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870036070 con 0x7fd8800cc940 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd87f7fe640 1 -- 172.21.15.2:0/472364858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd870031600 con 0x7fd8800cc940 2024-09-19T05:45:29.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd87f7fe640 1 -- 172.21.15.2:0/472364858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700319d0 con 0x7fd8800cc940 2024-09-19T05:45:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 -- 172.21.15.2:0/472364858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800cc940 msgr2=0x7fd8800d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 --2- 172.21.15.2:0/472364858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800cc940 0x7fd8800d0460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd870009e50 tx=0x7fd87002a750 comp rx=0 tx=0).stop 2024-09-19T05:45:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 -- 172.21.15.2:0/472364858 shutdown_connections 2024-09-19T05:45:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 --2- 172.21.15.2:0/472364858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800cc940 0x7fd8800d0460 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 --2- 172.21.15.2:0/472364858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 0x7fd8800cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 -- 172.21.15.2:0/472364858 >> 172.21.15.2:0/472364858 conn(0x7fd88006dc50 msgr2=0x7fd880073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:29.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 -- 172.21.15.2:0/472364858 shutdown_connections 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.734+0000 7fd886b9f640 1 -- 172.21.15.2:0/472364858 wait complete. 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 Processor -- start 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 -- start start 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800cc940 0x7fd88014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800d09a0 0x7fd88014dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 0x7fd8801f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd880155240 con 0x7fd8800d09a0 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd88015e130 con 0x7fd8800cc940 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.736+0000 7fd886b9f640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd8800cba10 con 0x7fd880154e60 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800d09a0 0x7fd88014dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800d09a0 0x7fd88014dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54972/0 (socket says 172.21.15.2:54972) 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 -- 172.21.15.2:0/4224306071 learned_addr learned my addr 172.21.15.2:0/4224306071 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:29.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd88493b640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800cc940 0x7fd88014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd88513c640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 0x7fd8801f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 -- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 msgr2=0x7fd8801f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd880154e60 0x7fd8801f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 -- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800cc940 msgr2=0x7fd88014d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8800cc940 0x7fd88014d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 -- 172.21.15.2:0/4224306071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd880152a80 con 0x7fd8800d09a0 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87ffff640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800d09a0 0x7fd88014dac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd870034040 tx=0x7fd870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.744 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870036050 con 0x7fd8800d09a0 2024-09-19T05:45:29.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd870042070 con 0x7fd8800d09a0 2024-09-19T05:45:29.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88014c920 con 0x7fd8800d09a0 2024-09-19T05:45:29.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870009510 con 0x7fd8800d09a0 2024-09-19T05:45:29.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.737+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd880157b20 con 0x7fd8800d09a0 2024-09-19T05:45:29.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.738+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd88015ab20 con 0x7fd8800d09a0 2024-09-19T05:45:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.739+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd870007c50 con 0x7fd8800d09a0 2024-09-19T05:45:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.739+0000 7fd87d7fa640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd86006c2b0 0x7fd86006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.739+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fd8700ac2f0 con 0x7fd8800d09a0 2024-09-19T05:45:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.740+0000 7fd87d7fa640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd8600745d0 0x7fd860076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.740+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd87004ae80 con 0x7fd8600745d0 2024-09-19T05:45:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.740+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fd8700b3050 con 0x7fd8800d09a0 2024-09-19T05:45:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.740+0000 7fd88493b640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd86006c2b0 0x7fd86006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.740+0000 7fd88513c640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd8600745d0 0x7fd860076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.747 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.740+0000 7fd88513c640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd8600745d0 0x7fd860076a30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.740+0000 7fd88493b640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd86006c2b0 0x7fd86006e770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd8780079a0 tx=0x7fd878008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.748 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.741+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== osd.1 v2:172.21.15.2:6816/2074594312 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd87004ae80 con 0x7fd8600745d0 2024-09-19T05:45:29.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.753+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 --> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd8800cc0a0 con 0x7fd8600745d0 2024-09-19T05:45:29.755 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.753+0000 7fd87d7fa640 1 -- 172.21.15.2:0/4224306071 <== osd.1 v2:172.21.15.2:6816/2074594312 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd8800cc0a0 con 0x7fd8600745d0 2024-09-19T05:45:29.756 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:29.756 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:29.756 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd8600745d0 msgr2=0x7fd860076a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:6816/2074594312,v1:172.21.15.2:6817/2074594312] conn(0x7fd8600745d0 0x7fd860076a30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd86006c2b0 msgr2=0x7fd86006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd86006c2b0 0x7fd86006e770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd8780079a0 tx=0x7fd878008040 comp rx=0 tx=0).stop 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800d09a0 msgr2=0x7fd88014dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 --2- 172.21.15.2:0/4224306071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8800d09a0 0x7fd88014dac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd870034040 tx=0x7fd870004a40 comp rx=0 tx=0).stop 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd88513c640 1 -- 172.21.15.2:0/4224306071 reap_dead start 2024-09-19T05:45:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 shutdown_connections 2024-09-19T05:45:29.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 >> 172.21.15.2:0/4224306071 conn(0x7fd88006dc50 msgr2=0x7fd88015a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:29.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 shutdown_connections 2024-09-19T05:45:29.758 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.754+0000 7fd886b9f640 1 -- 172.21.15.2:0/4224306071 wait complete. 2024-09-19T05:45:29.766 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-19T05:45:29.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 Processor -- start 2024-09-19T05:45:29.870 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 -- start start 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa474154c20 0x7fa4740cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa474155000 0x7fa474073340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4740caf40 0x7fa474073880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fa4740ccbc0 con 0x7fa474155000 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa4740cc600 con 0x7fa474154c20 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.868+0000 7fa47a98d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa4740cc8e0 con 0x7fa4740caf40 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa474155000 0x7fa474073340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa474155000 0x7fa474073340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54984/0 (socket says 172.21.15.2:54984) 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 -- 172.21.15.2:0/100635579 learned_addr learned my addr 172.21.15.2:0/100635579 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 -- 172.21.15.2:0/100635579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4740caf40 msgr2=0x7fa474073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa478f2a640 1 --2- 172.21.15.2:0/100635579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4740caf40 0x7fa474073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa473fff640 1 --2- 172.21.15.2:0/100635579 >> v2:172.21.15.123:3300/0 conn(0x7fa474154c20 0x7fa4740cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 --2- 172.21.15.2:0/100635579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4740caf40 0x7fa474073880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 -- 172.21.15.2:0/100635579 >> v2:172.21.15.123:3300/0 conn(0x7fa474154c20 msgr2=0x7fa4740cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 --2- 172.21.15.2:0/100635579 >> v2:172.21.15.123:3300/0 conn(0x7fa474154c20 0x7fa4740cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.869+0000 7fa4737fe640 1 -- 172.21.15.2:0/100635579 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4740cbc60 con 0x7fa474155000 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa4737fe640 1 --2- 172.21.15.2:0/100635579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa474155000 0x7fa474073340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa474073ed0 tx=0x7fa46802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34efd23a062120f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa472ffd640 1 -- 172.21.15.2:0/100635579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468004030 con 0x7fa474155000 2024-09-19T05:45:29.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa472ffd640 1 -- 172.21.15.2:0/100635579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4680041d0 con 0x7fa474155000 2024-09-19T05:45:29.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa472ffd640 1 -- 172.21.15.2:0/100635579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468003610 con 0x7fa474155000 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 -- 172.21.15.2:0/100635579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa474155000 msgr2=0x7fa474073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 --2- 172.21.15.2:0/100635579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa474155000 0x7fa474073340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa474073ed0 tx=0x7fa46802a1c0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 -- 172.21.15.2:0/100635579 shutdown_connections 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 --2- 172.21.15.2:0/100635579 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa4740caf40 0x7fa474073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 --2- 172.21.15.2:0/100635579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa474155000 0x7fa474073340 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 --2- 172.21.15.2:0/100635579 >> v2:172.21.15.123:3300/0 conn(0x7fa474154c20 0x7fa4740cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 -- 172.21.15.2:0/100635579 >> 172.21.15.2:0/100635579 conn(0x7fa47406d0e0 msgr2=0x7fa47406d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 -- 172.21.15.2:0/100635579 shutdown_connections 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.870+0000 7fa47a98d640 1 -- 172.21.15.2:0/100635579 wait complete. 2024-09-19T05:45:29.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 Processor -- start 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 -- start start 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4740caf40 0x7fa4741e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa474154c20 0x7fa4741f0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa474155000 0x7fa4741f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740c5770 con 0x7fa4740caf40 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740cb430 con 0x7fa474154c20 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.872+0000 7fa47a98d640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa4740c58f0 con 0x7fa474155000 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4740caf40 0x7fa4741e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4740caf40 0x7fa4741e5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54988/0 (socket says 172.21.15.2:54988) 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 -- 172.21.15.2:0/1108325360 learned_addr learned my addr 172.21.15.2:0/1108325360 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:29.878 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa4737fe640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa474154c20 0x7fa4741f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa478f2a640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa474155000 0x7fa4741f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 -- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa474155000 msgr2=0x7fa4741f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa474155000 0x7fa4741f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 -- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa474154c20 msgr2=0x7fa4741f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa474154c20 0x7fa4741f0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 -- 172.21.15.2:0/1108325360 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4741e2d90 con 0x7fa4740caf40 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa473fff640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4740caf40 0x7fa4741e5180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c820 tx=0x7fa46400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa470ff9640 1 -- 172.21.15.2:0/1108325360 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464015070 con 0x7fa4740caf40 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa470ff9640 1 -- 172.21.15.2:0/1108325360 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa464004020 con 0x7fa4740caf40 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4741e3050 con 0x7fa4740caf40 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa470ff9640 1 -- 172.21.15.2:0/1108325360 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4640043f0 con 0x7fa4740caf40 2024-09-19T05:45:29.879 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.873+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4740cf8e0 con 0x7fa4740caf40 2024-09-19T05:45:29.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.874+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4740cfdc0 con 0x7fa4740caf40 2024-09-19T05:45:29.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.875+0000 7fa470ff9640 1 -- 172.21.15.2:0/1108325360 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa464007830 con 0x7fa4740caf40 2024-09-19T05:45:29.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.875+0000 7fa470ff9640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa46006c2e0 0x7fa46006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:29.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.875+0000 7fa470ff9640 1 -- 172.21.15.2:0/1108325360 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fa46408b180 con 0x7fa4740caf40 2024-09-19T05:45:29.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.875+0000 7fa4737fe640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa46006c2e0 0x7fa46006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:29.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.876+0000 7fa4737fe640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa46006c2e0 0x7fa46006e7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa468033000 tx=0x7fa46802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:29.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:29.880+0000 7fa470ff9640 1 -- 172.21.15.2:0/1108325360 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fa464059240 con 0x7fa4740caf40 2024-09-19T05:45:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.055+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa4740cd9e0 con 0x7fa46006c2e0 2024-09-19T05:45:30.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.055+0000 7fa470ff9640 1 -- 172.21.15.2:0/1108325360 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa4740cd9e0 con 0x7fa46006c2e0 2024-09-19T05:45:30.058 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 1 to deep-scrub 2024-09-19T05:45:30.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa46006c2e0 msgr2=0x7fa46006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa46006c2e0 0x7fa46006e7a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa468033000 tx=0x7fa46802e000 comp rx=0 tx=0).stop 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4740caf40 msgr2=0x7fa4741e5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4740caf40 0x7fa4741e5180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa46400c820 tx=0x7fa46400cce0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 shutdown_connections 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa46006c2e0 0x7fa46006e7a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa474154c20 0x7fa4741f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa474155000 0x7fa4741f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 --2- 172.21.15.2:0/1108325360 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4740caf40 0x7fa4741e5180 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 >> 172.21.15.2:0/1108325360 conn(0x7fa47406d0e0 msgr2=0x7fa474153d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 shutdown_connections 2024-09-19T05:45:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.058+0000 7fa47a98d640 1 -- 172.21.15.2:0/1108325360 wait complete. 2024-09-19T05:45:30.071 INFO:tasks.ceph:Scrubbing osd.2 2024-09-19T05:45:30.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:30.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.175+0000 7f7bc7d59640 1 Processor -- start 2024-09-19T05:45:30.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.175+0000 7f7bc7d59640 1 -- start start 2024-09-19T05:45:30.179 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc7d59640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb830 0x7f7bc015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc7d59640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7bc01548d0 0x7f7bc0154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc7d59640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc00c58d0 con 0x7f7bc00cb450 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc7d59640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc015b9f0 con 0x7f7bc01548d0 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc7d59640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc00cca30 con 0x7f7bc00cb830 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc5af5640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f7bc00cb450 0x7f7bc00cbf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54186/0 (socket says 172.21.15.2:54186) 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc5af5640 1 -- 172.21.15.2:0/3521635652 learned_addr learned my addr 172.21.15.2:0/3521635652 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc62f6640 1 --2- 172.21.15.2:0/3521635652 >> v2:172.21.15.2:3300/0 conn(0x7f7bc01548d0 0x7f7bc0154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.176+0000 7f7bc52f4640 1 --2- 172.21.15.2:0/3521635652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb830 0x7f7bc015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.177+0000 7f7bc4af3640 1 -- 172.21.15.2:0/3521635652 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280845216 0 0) 0x7f7bc00c58d0 con 0x7f7bc00cb450 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.177+0000 7f7bc4af3640 1 -- 172.21.15.2:0/3521635652 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ba8003520 con 0x7f7bc00cb450 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.177+0000 7f7bc62f6640 1 -- 172.21.15.2:0/3521635652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb830 msgr2=0x7f7bc015b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.177+0000 7f7bc62f6640 1 --2- 172.21.15.2:0/3521635652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb830 0x7f7bc015b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.180 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.177+0000 7f7bc62f6640 1 -- 172.21.15.2:0/3521635652 >> v1:172.21.15.2:6789/0 conn(0x7f7bc00cb450 legacy=0x7f7bc00cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.177+0000 7f7bc62f6640 1 -- 172.21.15.2:0/3521635652 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc00ccd10 con 0x7f7bc01548d0 2024-09-19T05:45:30.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.177+0000 7f7bc62f6640 1 --2- 172.21.15.2:0/3521635652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc0154d60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009d90 tx=0x7f7bb002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ec0c8148a344c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc4af3640 1 -- 172.21.15.2:0/3521635652 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0007740 con 0x7f7bc01548d0 2024-09-19T05:45:30.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc4af3640 1 -- 172.21.15.2:0/3521635652 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb00078e0 con 0x7f7bc01548d0 2024-09-19T05:45:30.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc4af3640 1 -- 172.21.15.2:0/3521635652 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0007cb0 con 0x7f7bc01548d0 2024-09-19T05:45:30.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 -- 172.21.15.2:0/3521635652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 msgr2=0x7f7bc0154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 --2- 172.21.15.2:0/3521635652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc0154d60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009d90 tx=0x7f7bb002a6f0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 -- 172.21.15.2:0/3521635652 shutdown_connections 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 --2- 172.21.15.2:0/3521635652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc0154d60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 --2- 172.21.15.2:0/3521635652 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb830 0x7f7bc015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 -- 172.21.15.2:0/3521635652 >> 172.21.15.2:0/3521635652 conn(0x7f7bc0067950 msgr2=0x7f7bc0067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 -- 172.21.15.2:0/3521635652 shutdown_connections 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.178+0000 7f7bc7d59640 1 -- 172.21.15.2:0/3521635652 wait complete. 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.179+0000 7f7bc7d59640 1 Processor -- start 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.179+0000 7f7bc7d59640 1 -- start start 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc7d59640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb450 0x7f7bc01e5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc7d59640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc00cb830 0x7f7bc00cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc7d59640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc01f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc7d59640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc00cd5b0 con 0x7f7bc01548d0 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc7d59640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc00cd2d0 con 0x7f7bc00cb830 2024-09-19T05:45:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc7d59640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc006ca00 con 0x7f7bc00cb450 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc62f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc01f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc62f6640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc01f0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55006/0 (socket says 172.21.15.2:55006) 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc62f6640 1 -- 172.21.15.2:0/4138034969 learned_addr learned my addr 172.21.15.2:0/4138034969 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc5af5640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb450 0x7f7bc01e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.180+0000 7f7bc52f4640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc00cb830 0x7f7bc00cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc62f6640 1 -- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb450 msgr2=0x7f7bc01e5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc62f6640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb450 0x7f7bc01e5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc62f6640 1 -- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc00cb830 msgr2=0x7f7bc00cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc62f6640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bc00cb830 0x7f7bc00cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc62f6640 1 -- 172.21.15.2:0/4138034969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc01e1ac0 con 0x7f7bc01548d0 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc5af5640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7bc00cb450 0x7f7bc01e5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc62f6640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc01f0b20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009eb0 tx=0x7f7bb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb003f070 con 0x7f7bc01548d0 2024-09-19T05:45:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb0031bc0 con 0x7f7bc01548d0 2024-09-19T05:45:30.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc01f4090 con 0x7f7bc01548d0 2024-09-19T05:45:30.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0009450 con 0x7f7bc01548d0 2024-09-19T05:45:30.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.181+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc01f4650 con 0x7f7bc01548d0 2024-09-19T05:45:30.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.182+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7bc00b3750 con 0x7f7bc01548d0 2024-09-19T05:45:30.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7bb003b410 con 0x7f7bc01548d0 2024-09-19T05:45:30.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bbe7fc640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7bac06c2e0 0x7f7bac06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.187 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f7bb00ac930 con 0x7f7bc01548d0 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bbe7fc640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7bac0745a0 0x7f7bac076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7bb003de50 con 0x7f7bac0745a0 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f7bb00b12c0 con 0x7f7bc01548d0 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bc52f4640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7bac0745a0 0x7f7bac076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.183+0000 7f7bc5af5640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7bac06c2e0 0x7f7bac06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.184+0000 7f7bc52f4640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7bac0745a0 0x7f7bac076a00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.184+0000 7f7bc5af5640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7bac06c2e0 0x7f7bac06e7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005880 tx=0x7f7bb8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.184+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== osd.2 v2:172.21.15.2:6824/2644191985 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7bb003de50 con 0x7f7bac0745a0 2024-09-19T05:45:30.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 --> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7bc015b4b0 con 0x7f7bac0745a0 2024-09-19T05:45:30.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bbe7fc640 1 -- 172.21.15.2:0/4138034969 <== osd.2 v2:172.21.15.2:6824/2644191985 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7bc015b4b0 con 0x7f7bac0745a0 2024-09-19T05:45:30.199 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:30.199 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:30.199 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:30.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7bac0745a0 msgr2=0x7f7bac076a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:6824/2644191985,v1:172.21.15.2:6825/2644191985] conn(0x7f7bac0745a0 0x7f7bac076a00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7bac06c2e0 msgr2=0x7f7bac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7bac06c2e0 0x7f7bac06e7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005880 tx=0x7f7bb8009040 comp rx=0 tx=0).stop 2024-09-19T05:45:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 msgr2=0x7f7bc01f0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 --2- 172.21.15.2:0/4138034969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7bc01548d0 0x7f7bc01f0b20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009eb0 tx=0x7f7bb0004a40 comp rx=0 tx=0).stop 2024-09-19T05:45:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc62f6640 1 -- 172.21.15.2:0/4138034969 reap_dead start 2024-09-19T05:45:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 shutdown_connections 2024-09-19T05:45:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 >> 172.21.15.2:0/4138034969 conn(0x7f7bc0067950 msgr2=0x7f7bc00d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 shutdown_connections 2024-09-19T05:45:30.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.197+0000 7f7bc7d59640 1 -- 172.21.15.2:0/4138034969 wait complete. 2024-09-19T05:45:30.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-19T05:45:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.311+0000 7fecaae47640 1 Processor -- start 2024-09-19T05:45:30.313 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.311+0000 7fecaae47640 1 -- start start 2024-09-19T05:45:30.314 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fecaae47640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feca40d0c90 0x7feca415b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fecaae47640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40c55d0 0x7feca40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fecaae47640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca40cc930 con 0x7feca40d08b0 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fecaae47640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7feca40cc010 con 0x7feca40d0c90 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fecaae47640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feca40ccbd0 con 0x7feca40c55d0 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fec9bfff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feca40d0c90 0x7feca415b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fec9bfff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feca40d0c90 0x7feca415b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55010/0 (socket says 172.21.15.2:55010) 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fec9bfff640 1 -- 172.21.15.2:0/4178775198 learned_addr learned my addr 172.21.15.2:0/4178775198 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7feca93e4640 1 --2- 172.21.15.2:0/4178775198 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40c55d0 0x7feca40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fec9bfff640 1 -- 172.21.15.2:0/4178775198 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40c55d0 msgr2=0x7feca40cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:30.315 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fec9bfff640 1 --2- 172.21.15.2:0/4178775198 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40c55d0 0x7feca40cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fec9bfff640 1 -- 172.21.15.2:0/4178775198 >> v1:172.21.15.123:6789/0 conn(0x7feca40d08b0 legacy=0x7feca40cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.316 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.312+0000 7fec9bfff640 1 -- 172.21.15.2:0/4178775198 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40cceb0 con 0x7feca40d0c90 2024-09-19T05:45:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.313+0000 7fec9bfff640 1 --2- 172.21.15.2:0/4178775198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca415b660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fec94009870 tx=0x7fec9402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fdadb34d3a794d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.313+0000 7fec9b7fe640 1 -- 172.21.15.2:0/4178775198 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94037070 con 0x7feca40d0c90 2024-09-19T05:45:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.313+0000 7fec9b7fe640 1 -- 172.21.15.2:0/4178775198 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9402ae50 con 0x7feca40d0c90 2024-09-19T05:45:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.313+0000 7fec9b7fe640 1 -- 172.21.15.2:0/4178775198 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94032860 con 0x7feca40d0c90 2024-09-19T05:45:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 -- 172.21.15.2:0/4178775198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 msgr2=0x7feca415b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 --2- 172.21.15.2:0/4178775198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca415b660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fec94009870 tx=0x7fec9402a300 comp rx=0 tx=0).stop 2024-09-19T05:45:30.320 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 -- 172.21.15.2:0/4178775198 shutdown_connections 2024-09-19T05:45:30.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 --2- 172.21.15.2:0/4178775198 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40c55d0 0x7feca40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 --2- 172.21.15.2:0/4178775198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca415b660 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 -- 172.21.15.2:0/4178775198 >> 172.21.15.2:0/4178775198 conn(0x7feca406d940 msgr2=0x7feca4073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 -- 172.21.15.2:0/4178775198 shutdown_connections 2024-09-19T05:45:30.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.314+0000 7fecaae47640 1 -- 172.21.15.2:0/4178775198 wait complete. 2024-09-19T05:45:30.321 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.315+0000 7fecaae47640 1 Processor -- start 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.315+0000 7fecaae47640 1 -- start start 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fecaae47640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feca40c55d0 0x7feca414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fecaae47640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40d08b0 0x7feca40ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fecaae47640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca41e4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fecaae47640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7feca40cbb70 con 0x7feca40d0c90 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fecaae47640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feca40cd470 con 0x7feca40c55d0 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fecaae47640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feca40cbcf0 con 0x7feca40d08b0 2024-09-19T05:45:30.322 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca41e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca41e4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55022/0 (socket says 172.21.15.2:55022) 2024-09-19T05:45:30.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 -- 172.21.15.2:0/3957774745 learned_addr learned my addr 172.21.15.2:0/3957774745 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:30.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fec9bfff640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40d08b0 0x7feca40ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca8be3640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feca40c55d0 0x7feca414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 -- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40d08b0 msgr2=0x7feca40ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40d08b0 0x7feca40ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.323 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 -- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feca40c55d0 msgr2=0x7feca414dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feca40c55d0 0x7feca414dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 -- 172.21.15.2:0/3957774745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40c8c80 con 0x7feca40d0c90 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7fec9bfff640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40d08b0 0x7feca40ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca8be3640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feca40c55d0 0x7feca414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.316+0000 7feca93e4640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca41e4540 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00efc0 tx=0x7fec9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.317+0000 7fec997fa640 1 -- 172.21.15.2:0/3957774745 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c018020 con 0x7feca40d0c90 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.317+0000 7fec997fa640 1 -- 172.21.15.2:0/3957774745 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9c014480 con 0x7feca40d0c90 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.317+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca40c8f60 con 0x7feca40d0c90 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.317+0000 7fec997fa640 1 -- 172.21.15.2:0/3957774745 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c014870 con 0x7feca40d0c90 2024-09-19T05:45:30.324 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.317+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca40c9800 con 0x7feca40d0c90 2024-09-19T05:45:30.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.318+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca40c9350 con 0x7feca40d0c90 2024-09-19T05:45:30.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.318+0000 7fec997fa640 1 -- 172.21.15.2:0/3957774745 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fec9c005d10 con 0x7feca40d0c90 2024-09-19T05:45:30.325 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.319+0000 7fec997fa640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fec9006c2e0 0x7fec9006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.319+0000 7fec997fa640 1 -- 172.21.15.2:0/3957774745 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fec9c08be20 con 0x7feca40d0c90 2024-09-19T05:45:30.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.319+0000 7feca8be3640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fec9006c2e0 0x7fec9006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.319+0000 7feca8be3640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fec9006c2e0 0x7fec9006e7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feca0000f60 tx=0x7feca0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.326 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.323+0000 7fec997fa640 1 -- 172.21.15.2:0/3957774745 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fec9c022070 con 0x7feca40d0c90 2024-09-19T05:45:30.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.500+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7feca40cc190 con 0x7fec9006c2e0 2024-09-19T05:45:30.502 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.500+0000 7fec997fa640 1 -- 172.21.15.2:0/3957774745 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7feca40cc190 con 0x7fec9006c2e0 2024-09-19T05:45:30.502 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 2 to deep-scrub 2024-09-19T05:45:30.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fec9006c2e0 msgr2=0x7fec9006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fec9006c2e0 0x7fec9006e7a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feca0000f60 tx=0x7feca0009040 comp rx=0 tx=0).stop 2024-09-19T05:45:30.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 msgr2=0x7feca41e4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.505 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca41e4540 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00efc0 tx=0x7fec9c00c490 comp rx=0 tx=0).stop 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 shutdown_connections 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fec9006c2e0 0x7fec9006e7a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feca40d0c90 0x7feca41e4540 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feca40d08b0 0x7feca40ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 --2- 172.21.15.2:0/3957774745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feca40c55d0 0x7feca414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 >> 172.21.15.2:0/3957774745 conn(0x7feca406d940 msgr2=0x7feca4150440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 shutdown_connections 2024-09-19T05:45:30.506 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.503+0000 7fecaae47640 1 -- 172.21.15.2:0/3957774745 wait complete. 2024-09-19T05:45:30.517 INFO:tasks.ceph:Scrubbing osd.3 2024-09-19T05:45:30.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.621+0000 7fa802998640 1 Processor -- start 2024-09-19T05:45:30.623 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.621+0000 7fa802998640 1 -- start start 2024-09-19T05:45:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa802998640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb830 0x7fa7fc15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa802998640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa7fc1548d0 0x7fa7fc154d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa802998640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc0c58d0 con 0x7fa7fc0cb450 2024-09-19T05:45:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa802998640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc15b9f0 con 0x7fa7fc1548d0 2024-09-19T05:45:30.625 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa802998640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc0cca30 con 0x7fa7fc0cb830 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa800f35640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa7fc1548d0 0x7fa7fc154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa800f35640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa7fc1548d0 0x7fa7fc154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55036/0 (socket says 172.21.15.2:55036) 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa800f35640 1 -- 172.21.15.2:0/2373281660 learned_addr learned my addr 172.21.15.2:0/2373281660 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa7fb7fe640 1 --2- 172.21.15.2:0/2373281660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb830 0x7fa7fc15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa800f35640 1 -- 172.21.15.2:0/2373281660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb830 msgr2=0x7fa7fc15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa800f35640 1 --2- 172.21.15.2:0/2373281660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb830 0x7fa7fc15b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa800f35640 1 -- 172.21.15.2:0/2373281660 >> v1:172.21.15.123:6789/0 conn(0x7fa7fc0cb450 legacy=0x7fa7fc0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa800f35640 1 -- 172.21.15.2:0/2373281660 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0ccd10 con 0x7fa7fc1548d0 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.622+0000 7fa7fb7fe640 1 --2- 172.21.15.2:0/2373281660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb830 0x7fa7fc15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.623+0000 7fa800f35640 1 --2- 172.21.15.2:0/2373281660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc1548d0 0x7fa7fc154d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009e50 tx=0x7fa7f402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1619f3cc2b4276f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.623+0000 7fa7faffd640 1 -- 172.21.15.2:0/2373281660 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4036070 con 0x7fa7fc1548d0 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.623+0000 7fa7faffd640 1 -- 172.21.15.2:0/2373281660 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4031600 con 0x7fa7fc1548d0 2024-09-19T05:45:30.626 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.623+0000 7fa7faffd640 1 -- 172.21.15.2:0/2373281660 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40319f0 con 0x7fa7fc1548d0 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 -- 172.21.15.2:0/2373281660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc1548d0 msgr2=0x7fa7fc154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 --2- 172.21.15.2:0/2373281660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc1548d0 0x7fa7fc154d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009e50 tx=0x7fa7f402a7c0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 -- 172.21.15.2:0/2373281660 shutdown_connections 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 --2- 172.21.15.2:0/2373281660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc1548d0 0x7fa7fc154d60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 --2- 172.21.15.2:0/2373281660 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb830 0x7fa7fc15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 -- 172.21.15.2:0/2373281660 >> 172.21.15.2:0/2373281660 conn(0x7fa7fc067950 msgr2=0x7fa7fc067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 -- 172.21.15.2:0/2373281660 shutdown_connections 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.624+0000 7fa802998640 1 -- 172.21.15.2:0/2373281660 wait complete. 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.625+0000 7fa802998640 1 Processor -- start 2024-09-19T05:45:30.628 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.625+0000 7fa802998640 1 -- start start 2024-09-19T05:45:30.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa802998640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb450 0x7fa7fc1e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa802998640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc0cb830 0x7fa7fc158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.630 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa802998640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1548d0 0x7fa7fc14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa802998640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0cd5b0 con 0x7fa7fc0cb830 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa802998640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0cd2d0 con 0x7fa7fc1548d0 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa802998640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc06ca00 con 0x7fa7fc0cb450 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc0cb830 0x7fa7fc158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc0cb830 0x7fa7fc158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55040/0 (socket says 172.21.15.2:55040) 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 -- 172.21.15.2:0/3834093235 learned_addr learned my addr 172.21.15.2:0/3834093235 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fbfff640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb450 0x7fa7fc1e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa800f35640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1548d0 0x7fa7fc14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 -- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb450 msgr2=0x7fa7fc1e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa7fc0cb450 0x7fa7fc1e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 -- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1548d0 msgr2=0x7fa7fc14d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7fc1548d0 0x7fa7fc14d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.631 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.626+0000 7fa7fb7fe640 1 -- 172.21.15.2:0/3834093235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc158bd0 con 0x7fa7fc0cb830 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.627+0000 7fa7fb7fe640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc0cb830 0x7fa7fc158690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e810 tx=0x7fa7f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.627+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0017070 con 0x7fa7fc0cb830 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.627+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f000c8d0 con 0x7fa7fc0cb830 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.627+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc0d0100 con 0x7fa7fc0cb830 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.627+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000f040 con 0x7fa7fc0cb830 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.627+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1f47d0 con 0x7fa7fc0cb830 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.628+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7fc1f4ab0 con 0x7fa7fc0cb830 2024-09-19T05:45:30.632 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.628+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa7f001a070 con 0x7fa7fc0cb830 2024-09-19T05:45:30.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa7f8ff9640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa7e806c2b0 0x7fa7e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.633 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fa7f008af50 con 0x7fa7fc0cb830 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa7fbfff640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa7e806c2b0 0x7fa7e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa7f8ff9640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa7e8074570 0x7fa7e80769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7e80770a0 con 0x7fa7e8074570 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fa7f008d5f0 con 0x7fa7fc0cb830 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa800f35640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa7e8074570 0x7fa7e80769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa7fbfff640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa7e806c2b0 0x7fa7e806e770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec001780 tx=0x7fa7ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.629+0000 7fa800f35640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa7e8074570 0x7fa7e80769d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.634 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.630+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== osd.3 v2:172.21.15.2:6800/3985782072 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7e80770a0 con 0x7fa7e8074570 2024-09-19T05:45:30.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 --> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa7fc15b4b0 con 0x7fa7e8074570 2024-09-19T05:45:30.645 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa7f8ff9640 1 -- 172.21.15.2:0/3834093235 <== osd.3 v2:172.21.15.2:6800/3985782072 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa7fc15b4b0 con 0x7fa7e8074570 2024-09-19T05:45:30.645 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:30.645 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:30.645 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa7e8074570 msgr2=0x7fa7e80769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:6800/3985782072,v1:172.21.15.2:6801/3985782072] conn(0x7fa7e8074570 0x7fa7e80769d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.646 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa7e806c2b0 msgr2=0x7fa7e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fa7e806c2b0 0x7fa7e806e770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec001780 tx=0x7fa7ec009040 comp rx=0 tx=0).stop 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc0cb830 msgr2=0x7fa7fc158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 --2- 172.21.15.2:0/3834093235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7fc0cb830 0x7fa7fc158690 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000e810 tx=0x7fa7f000ecd0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa800f35640 1 -- 172.21.15.2:0/3834093235 reap_dead start 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 shutdown_connections 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 >> 172.21.15.2:0/3834093235 conn(0x7fa7fc067950 msgr2=0x7fa7fc0b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 shutdown_connections 2024-09-19T05:45:30.647 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.643+0000 7fa802998640 1 -- 172.21.15.2:0/3834093235 wait complete. 2024-09-19T05:45:30.656 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-19T05:45:30.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.757+0000 7f1474853640 1 Processor -- start 2024-09-19T05:45:30.759 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.757+0000 7f1474853640 1 -- start start 2024-09-19T05:45:30.760 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1474853640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f14700cc3f0 0x7f147015af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1474853640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f147015b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1474853640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1470155130 con 0x7f1470154d50 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1474853640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f14700cbc60 con 0x7f14700cc3f0 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1474853640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f14700cb050 con 0x7f14700cbec0 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1467fff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f14700cc3f0 0x7f147015af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f146e59c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f1470154d50 0x7f1470073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54198/0 (socket says 172.21.15.2:54198) 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f146e59c640 1 -- 172.21.15.2:0/3647209760 learned_addr learned my addr 172.21.15.2:0/3647209760 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f146ed9d640 1 --2- 172.21.15.2:0/3647209760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f147015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1467fff640 1 -- 172.21.15.2:0/3647209760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 msgr2=0x7f147015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1467fff640 1 --2- 172.21.15.2:0/3647209760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f147015b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1467fff640 1 -- 172.21.15.2:0/3647209760 >> v1:172.21.15.2:6789/0 conn(0x7f1470154d50 legacy=0x7f1470073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f1467fff640 1 -- 172.21.15.2:0/3647209760 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700cb330 con 0x7f14700cc3f0 2024-09-19T05:45:30.762 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.758+0000 7f146dd9b640 1 -- 172.21.15.2:0/3647209760 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4053921340 0 0) 0x7f1470155130 con 0x7f1470154d50 2024-09-19T05:45:30.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.759+0000 7f1467fff640 1 --2- 172.21.15.2:0/3647209760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14700cc3f0 0x7f147015af50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f14580097a0 tx=0x7f145802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fca2de905f0d8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.759+0000 7f146dd9b640 1 -- 172.21.15.2:0/3647209760 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458037070 con 0x7f14700cc3f0 2024-09-19T05:45:30.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.759+0000 7f146dd9b640 1 -- 172.21.15.2:0/3647209760 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1458032440 con 0x7f14700cc3f0 2024-09-19T05:45:30.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.759+0000 7f146dd9b640 1 -- 172.21.15.2:0/3647209760 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458032810 con 0x7f14700cc3f0 2024-09-19T05:45:30.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 -- 172.21.15.2:0/3647209760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14700cc3f0 msgr2=0x7f147015af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 --2- 172.21.15.2:0/3647209760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14700cc3f0 0x7f147015af50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f14580097a0 tx=0x7f145802a210 comp rx=0 tx=0).stop 2024-09-19T05:45:30.763 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 -- 172.21.15.2:0/3647209760 shutdown_connections 2024-09-19T05:45:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 --2- 172.21.15.2:0/3647209760 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f147015b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 --2- 172.21.15.2:0/3647209760 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14700cc3f0 0x7f147015af50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 -- 172.21.15.2:0/3647209760 >> 172.21.15.2:0/3647209760 conn(0x7f147006d250 msgr2=0x7f147006d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 -- 172.21.15.2:0/3647209760 shutdown_connections 2024-09-19T05:45:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.760+0000 7f1474853640 1 -- 172.21.15.2:0/3647209760 wait complete. 2024-09-19T05:45:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.761+0000 7f1474853640 1 Processor -- start 2024-09-19T05:45:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.761+0000 7f1474853640 1 -- start start 2024-09-19T05:45:30.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1474853640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f14701f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1474853640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14700cc3f0 0x7f1470143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1474853640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1470154d50 0x7f1470141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.766 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1474853640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f147015aa00 con 0x7f1470154d50 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1474853640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14700c5c40 con 0x7f14700cc3f0 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1474853640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1470073aa0 con 0x7f14700cbec0 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1470154d50 0x7f1470141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1470154d50 0x7f1470141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55068/0 (socket says 172.21.15.2:55068) 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 -- 172.21.15.2:0/609966701 learned_addr learned my addr 172.21.15.2:0/609966701 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146e59c640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f14701f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1467fff640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14700cc3f0 0x7f1470143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 -- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 msgr2=0x7f14701f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f14701f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 -- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14700cc3f0 msgr2=0x7f1470143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14700cc3f0 0x7f1470143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 -- 172.21.15.2:0/609966701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14700b57d0 con 0x7f1470154d50 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f1467fff640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14700cc3f0 0x7f1470143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:30.767 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146e59c640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f14701f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.762+0000 7f146ed9d640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1470154d50 0x7f1470141e60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1468007e60 tx=0x7f146800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.763+0000 7f1466ffd640 1 -- 172.21.15.2:0/609966701 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468014420 con 0x7f1470154d50 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.763+0000 7f1466ffd640 1 -- 172.21.15.2:0/609966701 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1468019070 con 0x7f1470154d50 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.763+0000 7f1466ffd640 1 -- 172.21.15.2:0/609966701 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468014820 con 0x7f1470154d50 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.763+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1470148c00 con 0x7f1470154d50 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.763+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14700b5a70 con 0x7f1470154d50 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.764+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14700c69f0 con 0x7f1470154d50 2024-09-19T05:45:30.768 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.764+0000 7f1466ffd640 1 -- 172.21.15.2:0/609966701 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f14680149c0 con 0x7f1470154d50 2024-09-19T05:45:30.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.765+0000 7f1466ffd640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f145006c2b0 0x7f145006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:30.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.765+0000 7f1466ffd640 1 -- 172.21.15.2:0/609966701 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f146808bd60 con 0x7f1470154d50 2024-09-19T05:45:30.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.765+0000 7f146e59c640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f145006c2b0 0x7f145006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:30.769 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.765+0000 7f146e59c640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f145006c2b0 0x7f145006e770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1460000f60 tx=0x7f1460007c10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:30.771 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.769+0000 7f1466ffd640 1 -- 172.21.15.2:0/609966701 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f146805a3e0 con 0x7f1470154d50 2024-09-19T05:45:30.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.946+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f14700cd750 con 0x7f145006c2b0 2024-09-19T05:45:30.948 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.946+0000 7f1466ffd640 1 -- 172.21.15.2:0/609966701 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f14700cd750 con 0x7f145006c2b0 2024-09-19T05:45:30.949 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 3 to deep-scrub 2024-09-19T05:45:30.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f145006c2b0 msgr2=0x7f145006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.951 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f145006c2b0 0x7f145006e770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1460000f60 tx=0x7f1460007c10 comp rx=0 tx=0).stop 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1470154d50 msgr2=0x7f1470141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1470154d50 0x7f1470141e60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1468007e60 tx=0x7f146800e4b0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 shutdown_connections 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f145006c2b0 0x7f145006e770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14700cc3f0 0x7f1470143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1470154d50 0x7f1470141e60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 --2- 172.21.15.2:0/609966701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f14700cbec0 0x7f14701f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 >> 172.21.15.2:0/609966701 conn(0x7f147006d250 msgr2=0x7f147015ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 shutdown_connections 2024-09-19T05:45:30.952 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:30.949+0000 7f1474853640 1 -- 172.21.15.2:0/609966701 wait complete. 2024-09-19T05:45:30.963 INFO:tasks.ceph:Scrubbing osd.4 2024-09-19T05:45:30.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:31.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.067+0000 7f16f6ef1640 1 Processor -- start 2024-09-19T05:45:31.069 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.067+0000 7f16f6ef1640 1 -- start start 2024-09-19T05:45:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16f6ef1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f0154e60 0x7f16f00cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16f6ef1640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f015aef0 con 0x7f16f015e650 2024-09-19T05:45:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16f6ef1640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16f015b1d0 con 0x7f16f00cc940 2024-09-19T05:45:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16f6ef1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f16f015ba70 con 0x7f16f0154e60 2024-09-19T05:45:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16f548e640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f16f015e650 0x7f16f01653e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54214/0 (socket says 172.21.15.2:54214) 2024-09-19T05:45:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16f548e640 1 -- 172.21.15.2:0/348424591 learned_addr learned my addr 172.21.15.2:0/348424591 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:31.073 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16f4c8d640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f0154e60 0x7f16f00cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568521342 0 0) 0x7f16f015aef0 con 0x7f16f015e650 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16e0003520 con 0x7f16f015e650 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923493712 0 0) 0x7f16f015b1d0 con 0x7f16f00cc940 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16f015aef0 con 0x7f16f00cc940 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 22277635 0 0) 0x7f16e0003520 con 0x7f16f015e650 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f0154e60 msgr2=0x7f16f00cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f0154e60 0x7f16f00cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 >> v1:172.21.15.123:6789/0 conn(0x7f16f00cc940 legacy=0x7f16f00d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.068+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f015b4b0 con 0x7f16f015e650 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f16e80019f0 con 0x7f16f015e650 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 >> v1:172.21.15.2:6789/0 conn(0x7f16f015e650 legacy=0x7f16f01653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16e0003be0 0x7f16e0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.074 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16e0004800 0x7f16e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e00051c0 0x7f16e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16f015ba70 con 0x7f16e0004800 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16e000bfe0 con 0x7f16e00051c0 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f16e000c1e0 con 0x7f16e0003be0 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16f4c8d640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f0154e60 0x7f16f00cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16effff640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16e0004800 0x7f16e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16f548e640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e00051c0 0x7f16e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16f4c8d640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16e0003be0 0x7f16e0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16effff640 1 -- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16e0003be0 msgr2=0x7f16e0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16effff640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16e0003be0 0x7f16e0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16effff640 1 -- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e00051c0 msgr2=0x7f16e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16effff640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e00051c0 0x7f16e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16effff640 1 -- 172.21.15.2:0/348424591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e0003520 con 0x7f16e0004800 2024-09-19T05:45:31.075 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.069+0000 7f16f548e640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e00051c0 0x7f16e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16effff640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16e0004800 0x7f16e0004c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f16f00cc700 tx=0x7f16e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8203b11978dc134 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0014e0 con 0x7f16e0004800 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc0052f0 con 0x7f16e0004800 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16ef7fe640 1 -- 172.21.15.2:0/348424591 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc02ac50 con 0x7f16e0004800 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16f6ef1640 1 -- 172.21.15.2:0/348424591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16e0004800 msgr2=0x7f16e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16f6ef1640 1 --2- 172.21.15.2:0/348424591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16e0004800 0x7f16e0004c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f16f00cc700 tx=0x7f16e0005830 comp rx=0 tx=0).stop 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16f548e640 1 -- 172.21.15.2:0/348424591 reap_dead start 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16f6ef1640 1 -- 172.21.15.2:0/348424591 shutdown_connections 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16f6ef1640 1 -- 172.21.15.2:0/348424591 >> 172.21.15.2:0/348424591 conn(0x7f16f006dc50 msgr2=0x7f16f0073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16f6ef1640 1 -- 172.21.15.2:0/348424591 shutdown_connections 2024-09-19T05:45:31.076 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.070+0000 7f16f6ef1640 1 -- 172.21.15.2:0/348424591 wait complete. 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.072+0000 7f16f6ef1640 1 Processor -- start 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.072+0000 7f16f6ef1640 1 -- start start 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.072+0000 7f16f6ef1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0004800 0x7f16f0157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f6ef1640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f015db60 0x7f16f015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f6ef1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00cf570 0x7f16f014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f6ef1640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0155240 con 0x7f16f00cf570 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f6ef1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16f015e130 con 0x7f16e0004800 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f6ef1640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f16f015ab20 con 0x7f16f015db60 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00cf570 0x7f16f014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00cf570 0x7f16f014e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55098/0 (socket says 172.21.15.2:55098) 2024-09-19T05:45:31.080 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 -- 172.21.15.2:0/1261909398 learned_addr learned my addr 172.21.15.2:0/1261909398 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f4c8d640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0004800 0x7f16f0157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16effff640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f015db60 0x7f16f015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 -- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f015db60 msgr2=0x7f16f015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f015db60 0x7f16f015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 -- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0004800 msgr2=0x7f16f0157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0004800 0x7f16f0157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 -- 172.21.15.2:0/1261909398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f01f4240 con 0x7f16f00cf570 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16effff640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16f015db60 0x7f16f015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f548e640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00cf570 0x7f16f014e0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16f015ea30 tx=0x7f16e8007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e800e980 con 0x7f16f00cf570 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16e800eb40 con 0x7f16f00cf570 2024-09-19T05:45:31.081 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f01f4510 con 0x7f16f00cf570 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8014430 con 0x7f16f00cf570 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.073+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f00ce7b0 con 0x7f16f00cf570 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.074+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16f014eee0 con 0x7f16f00cf570 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.075+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f16e80145d0 con 0x7f16f00cf570 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.075+0000 7f16ed7fa640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f16d406c2b0 0x7f16d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f16e808b170 con 0x7f16f00cf570 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16ed7fa640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f16d4074570 0x7f16d40769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16f4c8d640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f16d406c2b0 0x7f16d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16d40770a0 con 0x7f16d4074570 2024-09-19T05:45:31.082 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f16e8092050 con 0x7f16f00cf570 2024-09-19T05:45:31.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16effff640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f16d4074570 0x7f16d40769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16f4c8d640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f16d406c2b0 0x7f16d406e770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16f008dd40 tx=0x7f16e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.076+0000 7f16effff640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f16d4074570 0x7f16d40769d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.083 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.077+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== osd.4 v2:172.21.15.123:6816/2654957182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16d40770a0 con 0x7f16d4074570 2024-09-19T05:45:31.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.090+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 --> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f16f015eb00 con 0x7f16d4074570 2024-09-19T05:45:31.092 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.090+0000 7f16ed7fa640 1 -- 172.21.15.2:0/1261909398 <== osd.4 v2:172.21.15.123:6816/2654957182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f16f015eb00 con 0x7f16d4074570 2024-09-19T05:45:31.092 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:31.093 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:31.093 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:31.093 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.090+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f16d4074570 msgr2=0x7f16d40769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6816/2654957182,v1:172.21.15.123:6817/2654957182] conn(0x7f16d4074570 0x7f16d40769d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f16d406c2b0 msgr2=0x7f16d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f16d406c2b0 0x7f16d406e770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16f008dd40 tx=0x7f16e4008040 comp rx=0 tx=0).stop 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00cf570 msgr2=0x7f16f014e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 --2- 172.21.15.2:0/1261909398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00cf570 0x7f16f014e0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16f015ea30 tx=0x7f16e8007b00 comp rx=0 tx=0).stop 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f548e640 1 -- 172.21.15.2:0/1261909398 reap_dead start 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 shutdown_connections 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 >> 172.21.15.2:0/1261909398 conn(0x7f16f006dc50 msgr2=0x7f16f0159bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 shutdown_connections 2024-09-19T05:45:31.094 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.091+0000 7f16f6ef1640 1 -- 172.21.15.2:0/1261909398 wait complete. 2024-09-19T05:45:31.103 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-19T05:45:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.204+0000 7f7d72282640 1 Processor -- start 2024-09-19T05:45:31.207 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d72282640 1 -- start start 2024-09-19T05:45:31.210 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d72282640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d6c067d10 0x7f7d6c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d72282640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 0x7f7d6c0c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d72282640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c0b4930 con 0x7f7d6c0680f0 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d72282640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6c0b4650 con 0x7f7d6c067d10 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d72282640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c0c90c0 con 0x7f7d6c15cb90 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d6b7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f7d6c0680f0 0x7f7d6c15c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:54226/0 (socket says 172.21.15.2:54226) 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.205+0000 7f7d6b7fe640 1 -- 172.21.15.2:0/1875334270 learned_addr learned my addr 172.21.15.2:0/1875334270 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2483210311 0 0) 0x7f7d6c0b4930 con 0x7f7d6c0680f0 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d54003520 con 0x7f7d6c0680f0 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1875334270 >> v2:172.21.15.123:3300/0 conn(0x7f7d6c067d10 0x7f7d6c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d7081f640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 0x7f7d6c0c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.211 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3014681822 0 0) 0x7f7d54003520 con 0x7f7d6c0680f0 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 msgr2=0x7f7d6c0c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 0x7f7d6c0c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 >> v2:172.21.15.123:3300/0 conn(0x7f7d6c067d10 msgr2=0x7f7d6c0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 --2- 172.21.15.2:0/1875334270 >> v2:172.21.15.123:3300/0 conn(0x7f7d6c067d10 0x7f7d6c0b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0c93a0 con 0x7f7d6c0680f0 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f7d60001690 con 0x7f7d6c0680f0 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 >> v1:172.21.15.2:6789/0 conn(0x7f7d6c0680f0 legacy=0x7f7d6c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d540039b0 0x7f7d54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.212 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d54004800 0x7f7d54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 0x7f7d540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b4650 con 0x7f7d540039b0 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0c90c0 con 0x7f7d54004800 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5400b9c0 con 0x7f7d6c15cb90 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d540039b0 0x7f7d54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6b7fe640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d54004800 0x7f7d54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d7081f640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 0x7f7d540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6bfff640 1 -- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 msgr2=0x7f7d540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c15cb90 0x7f7d540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6bfff640 1 -- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d54004800 msgr2=0x7f7d54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.206+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d54004800 0x7f7d54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.213 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.207+0000 7f7d6bfff640 1 -- 172.21.15.2:0/1875334270 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d54003520 con 0x7f7d540039b0 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.207+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d540039b0 0x7f7d54003dd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ac0 tx=0x7f7d54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2201aecb30ef0ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c036070 con 0x7f7d540039b0 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c031400 con 0x7f7d540039b0 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d6affd640 1 -- 172.21.15.2:0/1875334270 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c009690 con 0x7f7d540039b0 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d72282640 1 -- 172.21.15.2:0/1875334270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d540039b0 msgr2=0x7f7d54003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d72282640 1 --2- 172.21.15.2:0/1875334270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d540039b0 0x7f7d54003dd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ac0 tx=0x7f7d54005830 comp rx=0 tx=0).stop 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d7081f640 1 -- 172.21.15.2:0/1875334270 reap_dead start 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d72282640 1 -- 172.21.15.2:0/1875334270 shutdown_connections 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d72282640 1 -- 172.21.15.2:0/1875334270 >> 172.21.15.2:0/1875334270 conn(0x7f7d6c070fc0 msgr2=0x7f7d6c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d72282640 1 -- 172.21.15.2:0/1875334270 shutdown_connections 2024-09-19T05:45:31.214 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.208+0000 7f7d72282640 1 -- 172.21.15.2:0/1875334270 wait complete. 2024-09-19T05:45:31.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.209+0000 7f7d72282640 1 Processor -- start 2024-09-19T05:45:31.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d72282640 1 -- start start 2024-09-19T05:45:31.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d72282640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d540039b0 0x7f7d6c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d72282640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d6c14ee10 0x7f7d6c146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.217 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d72282640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c146bc0 0x7f7d6c1f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d72282640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b41e0 con 0x7f7d6c14ee10 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d72282640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b4060 con 0x7f7d540039b0 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d72282640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b4360 con 0x7f7d6c146bc0 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d6c14ee10 0x7f7d6c146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d6b7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d6c14ee10 0x7f7d6c146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55114/0 (socket says 172.21.15.2:55114) 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d6b7fe640 1 -- 172.21.15.2:0/1708380776 learned_addr learned my addr 172.21.15.2:0/1708380776 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d540039b0 0x7f7d6c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.210+0000 7f7d7081f640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c146bc0 0x7f7d6c1f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d6b7fe640 1 -- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c146bc0 msgr2=0x7f7d6c1f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d6b7fe640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c146bc0 0x7f7d6c1f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d6b7fe640 1 -- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d540039b0 msgr2=0x7f7d6c1513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d6b7fe640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d540039b0 0x7f7d6c1513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.218 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d6b7fe640 1 -- 172.21.15.2:0/1708380776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0b32f0 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d540039b0 0x7f7d6c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d6b7fe640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d6c14ee10 0x7f7d6c146680 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d60007340 tx=0x7f7d60007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d68ff9640 1 -- 172.21.15.2:0/1708380776 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000f960 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d68ff9640 1 -- 172.21.15.2:0/1708380776 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d60018070 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c14f920 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d68ff9640 1 -- 172.21.15.2:0/1708380776 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000fd60 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.211+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c148240 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.212+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c0b3a90 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.213+0000 7f7d68ff9640 1 -- 172.21.15.2:0/1708380776 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7d6001d460 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.213+0000 7f7d68ff9640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d5806c2b0 0x7f7d5806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.213+0000 7f7d68ff9640 1 -- 172.21.15.2:0/1708380776 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f7d6008b400 con 0x7f7d6c14ee10 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.213+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d5806c2b0 0x7f7d5806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.219 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.214+0000 7f7d6bfff640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d5806c2b0 0x7f7d5806e770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c033000 tx=0x7f7d5c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.220 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.217+0000 7f7d68ff9640 1 -- 172.21.15.2:0/1708380776 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f7d60059550 con 0x7f7d6c14ee10 2024-09-19T05:45:31.397 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.395+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f7d6c145490 con 0x7f7d5806c2b0 2024-09-19T05:45:31.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.395+0000 7f7d68ff9640 1 -- 172.21.15.2:0/1708380776 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7d6c145490 con 0x7f7d5806c2b0 2024-09-19T05:45:31.398 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 4 to deep-scrub 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d5806c2b0 msgr2=0x7f7d5806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d5806c2b0 0x7f7d5806e770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c033000 tx=0x7f7d5c008030 comp rx=0 tx=0).stop 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d6c14ee10 msgr2=0x7f7d6c146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d6c14ee10 0x7f7d6c146680 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7d60007340 tx=0x7f7d60007800 comp rx=0 tx=0).stop 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 shutdown_connections 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f7d5806c2b0 0x7f7d5806e770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d6c146bc0 0x7f7d6c1f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d6c14ee10 0x7f7d6c146680 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 --2- 172.21.15.2:0/1708380776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d540039b0 0x7f7d6c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 >> 172.21.15.2:0/1708380776 conn(0x7f7d6c070fc0 msgr2=0x7f7d6c0b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.397+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 shutdown_connections 2024-09-19T05:45:31.401 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.398+0000 7f7d72282640 1 -- 172.21.15.2:0/1708380776 wait complete. 2024-09-19T05:45:31.411 INFO:tasks.ceph:Scrubbing osd.5 2024-09-19T05:45:31.411 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:31.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.514+0000 7fd62fc1e640 1 Processor -- start 2024-09-19T05:45:31.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.515+0000 7fd62fc1e640 1 -- start start 2024-09-19T05:45:31.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.515+0000 7fd62fc1e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd6280c96a0 0x7fd628070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.515+0000 7fd62fc1e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62806e920 con 0x7fd62815a390 2024-09-19T05:45:31.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.515+0000 7fd62fc1e640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd62806ebe0 con 0x7fd62815ad40 2024-09-19T05:45:31.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.515+0000 7fd62fc1e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fd62806cd20 con 0x7fd6280c96a0 2024-09-19T05:45:31.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62d9ba640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd62815a390 0x7fd628070220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55518/0 (socket says 172.21.15.2:55518) 2024-09-19T05:45:31.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62d9ba640 1 -- 172.21.15.2:0/2702955503 learned_addr learned my addr 172.21.15.2:0/2702955503 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:31.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62d1b9640 1 --2- 172.21.15.2:0/2702955503 >> v2:172.21.15.2:3300/0 conn(0x7fd6280c96a0 0x7fd628070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62c9b8640 1 -- 172.21.15.2:0/2702955503 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192204040 0 0) 0x7fd62806e920 con 0x7fd62815a390 2024-09-19T05:45:31.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62c9b8640 1 -- 172.21.15.2:0/2702955503 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd614003520 con 0x7fd62815a390 2024-09-19T05:45:31.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62d1b9640 1 -- 172.21.15.2:0/2702955503 >> v1:172.21.15.123:6789/0 conn(0x7fd62815ad40 legacy=0x7fd62815b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62d1b9640 1 -- 172.21.15.2:0/2702955503 >> v1:172.21.15.2:6789/0 conn(0x7fd62815a390 legacy=0x7fd628070220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.516+0000 7fd62d1b9640 1 -- 172.21.15.2:0/2702955503 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62806cf70 con 0x7fd6280c96a0 2024-09-19T05:45:31.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62d1b9640 1 --2- 172.21.15.2:0/2702955503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6280c96a0 0x7fd628070fa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd6180097a0 tx=0x7fd61802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974b005f1ce11f31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62c9b8640 1 -- 172.21.15.2:0/2702955503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618037070 con 0x7fd6280c96a0 2024-09-19T05:45:31.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62c9b8640 1 -- 172.21.15.2:0/2702955503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd618032440 con 0x7fd6280c96a0 2024-09-19T05:45:31.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62c9b8640 1 -- 172.21.15.2:0/2702955503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618032810 con 0x7fd6280c96a0 2024-09-19T05:45:31.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62fc1e640 1 -- 172.21.15.2:0/2702955503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6280c96a0 msgr2=0x7fd628070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62fc1e640 1 --2- 172.21.15.2:0/2702955503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6280c96a0 0x7fd628070fa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd6180097a0 tx=0x7fd61802a230 comp rx=0 tx=0).stop 2024-09-19T05:45:31.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62fc1e640 1 -- 172.21.15.2:0/2702955503 shutdown_connections 2024-09-19T05:45:31.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62fc1e640 1 --2- 172.21.15.2:0/2702955503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6280c96a0 0x7fd628070fa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62fc1e640 1 -- 172.21.15.2:0/2702955503 >> 172.21.15.2:0/2702955503 conn(0x7fd62806c030 msgr2=0x7fd628067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62fc1e640 1 -- 172.21.15.2:0/2702955503 shutdown_connections 2024-09-19T05:45:31.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.517+0000 7fd62fc1e640 1 -- 172.21.15.2:0/2702955503 wait complete. 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 Processor -- start 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 -- start start 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6280c96a0 0x7fd6281e6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd62815a390 0x7fd6281e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd62815ad40 0x7fd6281f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fd6280b0c70 con 0x7fd62815a390 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6280b12c0 con 0x7fd62815ad40 2024-09-19T05:45:31.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62fc1e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd6280b0f50 con 0x7fd6280c96a0 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62d1b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd62815a390 0x7fd6281e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62d1b9640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd62815a390 0x7fd6281e7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60852/0 (socket says 172.21.15.2:60852) 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.519+0000 7fd62d1b9640 1 -- 172.21.15.2:0/3031161489 learned_addr learned my addr 172.21.15.2:0/3031161489 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62e1bb640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd62815ad40 0x7fd6281f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62d9ba640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6280c96a0 0x7fd6281e6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62d1b9640 1 -- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6280c96a0 msgr2=0x7fd6281e6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62d1b9640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd6280c96a0 0x7fd6281e6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62d1b9640 1 -- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd62815ad40 msgr2=0x7fd6281f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62d1b9640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd62815ad40 0x7fd6281f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62d1b9640 1 -- 172.21.15.2:0/3031161489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281e6dd0 con 0x7fd62815a390 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62d1b9640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd62815a390 0x7fd6281e7db0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd6180098c0 tx=0x7fd6180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618037040 con 0x7fd62815a390 2024-09-19T05:45:31.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd618030500 con 0x7fd62815a390 2024-09-19T05:45:31.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6280aec20 con 0x7fd62815a390 2024-09-19T05:45:31.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180308d0 con 0x7fd62815a390 2024-09-19T05:45:31.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.520+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281f4200 con 0x7fd62815a390 2024-09-19T05:45:31.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.521+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd6281f3cd0 con 0x7fd62815a390 2024-09-19T05:45:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.522+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd618044020 con 0x7fd62815a390 2024-09-19T05:45:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.522+0000 7fd61e7fc640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd60c06c2b0 0x7fd60c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.522+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fd6180ab520 con 0x7fd62815a390 2024-09-19T05:45:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.522+0000 7fd62d9ba640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd60c06c2b0 0x7fd60c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.522+0000 7fd61e7fc640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fd60c074570 0x7fd60c0769d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.522+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd618032440 con 0x7fd60c074570 2024-09-19T05:45:31.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.522+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fd6180b2050 con 0x7fd62815a390 2024-09-19T05:45:31.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.523+0000 7fd62e1bb640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fd60c074570 0x7fd60c0769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.523+0000 7fd62d9ba640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd60c06c2b0 0x7fd60c06e770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd6240011e0 tx=0x7fd624007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.523+0000 7fd62e1bb640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fd60c074570 0x7fd60c0769d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.527 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.524+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== osd.5 v2:172.21.15.123:6800/2989037926 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd618032440 con 0x7fd60c074570 2024-09-19T05:45:31.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.536+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 --> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd62815a770 con 0x7fd60c074570 2024-09-19T05:45:31.539 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.537+0000 7fd61e7fc640 1 -- 172.21.15.2:0/3031161489 <== osd.5 v2:172.21.15.123:6800/2989037926 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd62815a770 con 0x7fd60c074570 2024-09-19T05:45:31.539 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:31.540 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:31.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.537+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fd60c074570 msgr2=0x7fd60c0769d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.537+0000 7fd62fc1e640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6800/2989037926,v1:172.21.15.123:6801/2989037926] conn(0x7fd60c074570 0x7fd60c0769d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.540 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.537+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd60c06c2b0 msgr2=0x7fd60c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.537+0000 7fd62fc1e640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fd60c06c2b0 0x7fd60c06e770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd6240011e0 tx=0x7fd624007d30 comp rx=0 tx=0).stop 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.538+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd62815a390 msgr2=0x7fd6281e7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.538+0000 7fd62fc1e640 1 --2- 172.21.15.2:0/3031161489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd62815a390 0x7fd6281e7db0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd6180098c0 tx=0x7fd6180046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.538+0000 7fd62e1bb640 1 -- 172.21.15.2:0/3031161489 reap_dead start 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.538+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 shutdown_connections 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.538+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 >> 172.21.15.2:0/3031161489 conn(0x7fd62806c030 msgr2=0x7fd628067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.538+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 shutdown_connections 2024-09-19T05:45:31.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.538+0000 7fd62fc1e640 1 -- 172.21.15.2:0/3031161489 wait complete. 2024-09-19T05:45:31.550 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-19T05:45:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.651+0000 7fe4822dd640 1 Processor -- start 2024-09-19T05:45:31.653 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.651+0000 7fe4822dd640 1 -- start start 2024-09-19T05:45:31.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.651+0000 7fe4822dd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15cc40 0x7fe47c0c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.651+0000 7fe4822dd640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe47c152460 0x7fe47c1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.651+0000 7fe4822dd640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe47c0c3a60 con 0x7fe47c15c150 2024-09-19T05:45:31.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.651+0000 7fe4822dd640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fe47c152df0 con 0x7fe47c152460 2024-09-19T05:45:31.654 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.651+0000 7fe4822dd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c0c3d40 con 0x7fe47c15cc40 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe48087a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe47c152460 0x7fe47c1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe48087a640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe47c152460 0x7fe47c1528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60866/0 (socket says 172.21.15.2:60866) 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe48087a640 1 -- 172.21.15.2:0/1133764715 learned_addr learned my addr 172.21.15.2:0/1133764715 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe47b7fe640 1 --2- 172.21.15.2:0/1133764715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15cc40 0x7fe47c0c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe48087a640 1 -- 172.21.15.2:0/1133764715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15cc40 msgr2=0x7fe47c0c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe48087a640 1 --2- 172.21.15.2:0/1133764715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15cc40 0x7fe47c0c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe48087a640 1 -- 172.21.15.2:0/1133764715 >> v1:172.21.15.123:6789/0 conn(0x7fe47c15c150 legacy=0x7fe47c15c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.655 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.652+0000 7fe48087a640 1 -- 172.21.15.2:0/1133764715 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c0c4c00 con 0x7fe47c152460 2024-09-19T05:45:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe48087a640 1 --2- 172.21.15.2:0/1133764715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c1528b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe474009e50 tx=0x7fe47402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db4e414a7010d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe47affd640 1 -- 172.21.15.2:0/1133764715 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474036070 con 0x7fe47c152460 2024-09-19T05:45:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe47affd640 1 -- 172.21.15.2:0/1133764715 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe474031600 con 0x7fe47c152460 2024-09-19T05:45:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe47affd640 1 -- 172.21.15.2:0/1133764715 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740319d0 con 0x7fe47c152460 2024-09-19T05:45:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe4822dd640 1 -- 172.21.15.2:0/1133764715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 msgr2=0x7fe47c1528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.656 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe4822dd640 1 --2- 172.21.15.2:0/1133764715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c1528b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe474009e50 tx=0x7fe47402a710 comp rx=0 tx=0).stop 2024-09-19T05:45:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe4822dd640 1 -- 172.21.15.2:0/1133764715 shutdown_connections 2024-09-19T05:45:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe4822dd640 1 --2- 172.21.15.2:0/1133764715 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c1528b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe4822dd640 1 --2- 172.21.15.2:0/1133764715 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15cc40 0x7fe47c0c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.653+0000 7fe4822dd640 1 -- 172.21.15.2:0/1133764715 >> 172.21.15.2:0/1133764715 conn(0x7fe47c06bc10 msgr2=0x7fe47c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.654+0000 7fe4822dd640 1 -- 172.21.15.2:0/1133764715 shutdown_connections 2024-09-19T05:45:31.657 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.654+0000 7fe4822dd640 1 -- 172.21.15.2:0/1133764715 wait complete. 2024-09-19T05:45:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.655+0000 7fe4822dd640 1 Processor -- start 2024-09-19T05:45:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.655+0000 7fe4822dd640 1 -- start start 2024-09-19T05:45:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.655+0000 7fe4822dd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.655+0000 7fe4822dd640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15c150 0x7fe47c1f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe4822dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c072bf0 0x7fe47c1f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.659 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe4822dd640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c0c51c0 con 0x7fe47c152460 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe4822dd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c06efa0 con 0x7fe47c072bf0 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe4822dd640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c0c7570 con 0x7fe47c15c150 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c0cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60874/0 (socket says 172.21.15.2:60874) 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 -- 172.21.15.2:0/3085623034 learned_addr learned my addr 172.21.15.2:0/3085623034 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe48087a640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c072bf0 0x7fe47c1f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47b7fe640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15c150 0x7fe47c1f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 -- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15c150 msgr2=0x7fe47c1f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15c150 0x7fe47c1f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 -- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c072bf0 msgr2=0x7fe47c1f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c072bf0 0x7fe47c1f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 -- 172.21.15.2:0/3085623034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c073120 con 0x7fe47c152460 2024-09-19T05:45:31.660 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe47bfff640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c0cc6a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c430 tx=0x7fe46c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe478ff9640 1 -- 172.21.15.2:0/3085623034 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0052d0 con 0x7fe47c152460 2024-09-19T05:45:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe478ff9640 1 -- 172.21.15.2:0/3085623034 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c005490 con 0x7fe47c152460 2024-09-19T05:45:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47c1f4bd0 con 0x7fe47c152460 2024-09-19T05:45:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe478ff9640 1 -- 172.21.15.2:0/3085623034 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c007260 con 0x7fe47c152460 2024-09-19T05:45:31.661 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.656+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe47c15b450 con 0x7fe47c152460 2024-09-19T05:45:31.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.657+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47c0c67f0 con 0x7fe47c152460 2024-09-19T05:45:31.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.658+0000 7fe478ff9640 1 -- 172.21.15.2:0/3085623034 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe46c007400 con 0x7fe47c152460 2024-09-19T05:45:31.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.658+0000 7fe478ff9640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe46806c2b0 0x7fe46806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.658+0000 7fe478ff9640 1 -- 172.21.15.2:0/3085623034 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fe46c08a910 con 0x7fe47c152460 2024-09-19T05:45:31.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.658+0000 7fe47b7fe640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe46806c2b0 0x7fe46806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.662 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.659+0000 7fe47b7fe640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe46806c2b0 0x7fe46806e770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe47c1ee630 tx=0x7fe470008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.665 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.663+0000 7fe478ff9640 1 -- 172.21.15.2:0/3085623034 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fe47c0c67f0 con 0x7fe47c152460 2024-09-19T05:45:31.841 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.839+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe47c1528b0 con 0x7fe46806c2b0 2024-09-19T05:45:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.840+0000 7fe478ff9640 1 -- 172.21.15.2:0/3085623034 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe47c1528b0 con 0x7fe46806c2b0 2024-09-19T05:45:31.842 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 5 to deep-scrub 2024-09-19T05:45:31.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe46806c2b0 msgr2=0x7fe46806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe46806c2b0 0x7fe46806e770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe47c1ee630 tx=0x7fe470008040 comp rx=0 tx=0).stop 2024-09-19T05:45:31.844 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 msgr2=0x7fe47c0cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c0cc6a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c430 tx=0x7fe46c00c8f0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 shutdown_connections 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fe46806c2b0 0x7fe46806e770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe47c072bf0 0x7fe47c1f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe47c15c150 0x7fe47c1f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 --2- 172.21.15.2:0/3085623034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe47c152460 0x7fe47c0cc6a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 >> 172.21.15.2:0/3085623034 conn(0x7fe47c06bc10 msgr2=0x7fe47c1508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 shutdown_connections 2024-09-19T05:45:31.845 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.842+0000 7fe4822dd640 1 -- 172.21.15.2:0/3085623034 wait complete. 2024-09-19T05:45:31.855 INFO:tasks.ceph:Scrubbing osd.6 2024-09-19T05:45:31.855 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:31.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.959+0000 7f0597f81640 1 Processor -- start 2024-09-19T05:45:31.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.960+0000 7f0597f81640 1 -- start start 2024-09-19T05:45:31.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.960+0000 7f0597f81640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0590154e60 0x7f05900cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.960+0000 7f0597f81640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f059015aef0 con 0x7f05900cc940 2024-09-19T05:45:31.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.960+0000 7f0597f81640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f059015b1d0 con 0x7f059015e650 2024-09-19T05:45:31.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.960+0000 7f0597f81640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f059015ba70 con 0x7f0590154e60 2024-09-19T05:45:31.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.961+0000 7f0595d1d640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0590154e60 0x7f05900cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.961+0000 7f059551c640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f05900cc940 0x7f05900d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55520/0 (socket says 172.21.15.2:55520) 2024-09-19T05:45:31.963 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.961+0000 7f059551c640 1 -- 172.21.15.2:0/2887562929 learned_addr learned my addr 172.21.15.2:0/2887562929 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:31.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.961+0000 7f0595d1d640 1 -- 172.21.15.2:0/2887562929 >> v1:172.21.15.123:6789/0 conn(0x7f059015e650 legacy=0x7f05901653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:31.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.961+0000 7f0595d1d640 1 -- 172.21.15.2:0/2887562929 >> v1:172.21.15.2:6789/0 conn(0x7f05900cc940 legacy=0x7f05900d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.964 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.961+0000 7f0595d1d640 1 -- 172.21.15.2:0/2887562929 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059015b4b0 con 0x7f0590154e60 2024-09-19T05:45:31.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0595d1d640 1 --2- 172.21.15.2:0/2887562929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0590154e60 0x7f05900cb4d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f058c0097a0 tx=0x7f058c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7fe67f0fedf95f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0594d1b640 1 -- 172.21.15.2:0/2887562929 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c037070 con 0x7f0590154e60 2024-09-19T05:45:31.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0594d1b640 1 -- 172.21.15.2:0/2887562929 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f058c032440 con 0x7f0590154e60 2024-09-19T05:45:31.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0594d1b640 1 -- 172.21.15.2:0/2887562929 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c032810 con 0x7f0590154e60 2024-09-19T05:45:31.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0597f81640 1 -- 172.21.15.2:0/2887562929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0590154e60 msgr2=0x7f05900cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.965 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0597f81640 1 --2- 172.21.15.2:0/2887562929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0590154e60 0x7f05900cb4d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f058c0097a0 tx=0x7f058c02a270 comp rx=0 tx=0).stop 2024-09-19T05:45:31.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0597f81640 1 -- 172.21.15.2:0/2887562929 shutdown_connections 2024-09-19T05:45:31.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0597f81640 1 --2- 172.21.15.2:0/2887562929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0590154e60 0x7f05900cb4d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.962+0000 7f0597f81640 1 -- 172.21.15.2:0/2887562929 >> 172.21.15.2:0/2887562929 conn(0x7f059006dc50 msgr2=0x7f0590073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.963+0000 7f0597f81640 1 -- 172.21.15.2:0/2887562929 shutdown_connections 2024-09-19T05:45:31.966 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.963+0000 7f0597f81640 1 -- 172.21.15.2:0/2887562929 wait complete. 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 Processor -- start 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 -- start start 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05900cc940 0x7f059015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590154e60 0x7f05900c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059015e650 0x7f05900c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f0590155240 con 0x7f05900cc940 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f059015e130 con 0x7f0590154e60 2024-09-19T05:45:31.969 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.964+0000 7f0597f81640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f05900cba10 con 0x7f059015e650 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05900cc940 0x7f059015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05900cc940 0x7f059015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60890/0 (socket says 172.21.15.2:60890) 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 -- 172.21.15.2:0/2750145477 learned_addr learned my addr 172.21.15.2:0/2750145477 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f059551c640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590154e60 0x7f05900c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f059651e640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059015e650 0x7f05900c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 -- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059015e650 msgr2=0x7f05900c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f059015e650 0x7f05900c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 -- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590154e60 msgr2=0x7f05900c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590154e60 0x7f05900c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 -- 172.21.15.2:0/2750145477 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05901f4100 con 0x7f05900cc940 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f059551c640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590154e60 0x7f05900c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0595d1d640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05900cc940 0x7f059015cf40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f058c000c00 tx=0x7f058c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c037040 con 0x7f05900cc940 2024-09-19T05:45:31.970 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f058c042070 con 0x7f05900cc940 2024-09-19T05:45:31.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05901f0cb0 con 0x7f05900cc940 2024-09-19T05:45:31.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c030a70 con 0x7f05900cc940 2024-09-19T05:45:31.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.965+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05901f1270 con 0x7f05900cc940 2024-09-19T05:45:31.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.966+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f059015ab20 con 0x7f05900cc940 2024-09-19T05:45:31.971 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.967+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f058c030c10 con 0x7f05900cc940 2024-09-19T05:45:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.967+0000 7f05867fc640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f057406c2b0 0x7f057406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.967+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f058c0abea0 con 0x7f05900cc940 2024-09-19T05:45:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.967+0000 7f059551c640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f057406c2b0 0x7f057406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.967+0000 7f05867fc640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f05740745d0 0x7f0574076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.967+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f058c049e70 con 0x7f05740745d0 2024-09-19T05:45:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.967+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f058c0ae5a0 con 0x7f05900cc940 2024-09-19T05:45:31.972 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.968+0000 7f059651e640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f05740745d0 0x7f0574076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:31.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.968+0000 7f059551c640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f057406c2b0 0x7f057406e770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f05900ca760 tx=0x7f0580009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.968+0000 7f059651e640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f05740745d0 0x7f0574076a30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:31.973 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.969+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== osd.6 v2:172.21.15.123:6824/593730548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f058c049e70 con 0x7f05740745d0 2024-09-19T05:45:31.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.982+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 --> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f05900cc0a0 con 0x7f05740745d0 2024-09-19T05:45:31.984 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.982+0000 7f05867fc640 1 -- 172.21.15.2:0/2750145477 <== osd.6 v2:172.21.15.123:6824/593730548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f05900cc0a0 con 0x7f05740745d0 2024-09-19T05:45:31.985 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:31.985 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:31.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f05740745d0 msgr2=0x7f0574076a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6824/593730548,v1:172.21.15.123:6825/593730548] conn(0x7f05740745d0 0x7f0574076a30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f057406c2b0 msgr2=0x7f057406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f057406c2b0 0x7f057406e770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f05900ca760 tx=0x7f0580009040 comp rx=0 tx=0).stop 2024-09-19T05:45:31.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05900cc940 msgr2=0x7f059015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:31.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 --2- 172.21.15.2:0/2750145477 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05900cc940 0x7f059015cf40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f058c000c00 tx=0x7f058c0046c0 comp rx=0 tx=0).stop 2024-09-19T05:45:31.986 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f059651e640 1 -- 172.21.15.2:0/2750145477 reap_dead start 2024-09-19T05:45:31.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 shutdown_connections 2024-09-19T05:45:31.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 >> 172.21.15.2:0/2750145477 conn(0x7f059006dc50 msgr2=0x7f0590152680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:31.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 shutdown_connections 2024-09-19T05:45:31.987 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:31.983+0000 7f0597f81640 1 -- 172.21.15.2:0/2750145477 wait complete. 2024-09-19T05:45:31.995 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-19T05:45:32.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.096+0000 7f75c562e640 1 Processor -- start 2024-09-19T05:45:32.099 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.096+0000 7f75c562e640 1 -- start start 2024-09-19T05:45:32.100 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.097+0000 7f75c562e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75c00c9000 0x7f75c00ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.097+0000 7f75c562e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0071110 0x7f75c00ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.097+0000 7f75c562e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c00ca3a0 con 0x7f75c0152b80 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.097+0000 7f75c562e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f75c01598e0 con 0x7f75c00c9000 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.097+0000 7f75c562e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f75c00c94f0 con 0x7f75c0071110 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.097+0000 7f75beffd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f75c0152b80 0x7f75c01591d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55526/0 (socket says 172.21.15.2:55526) 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.097+0000 7f75beffd640 1 -- 172.21.15.2:0/2682756147 learned_addr learned my addr 172.21.15.2:0/2682756147 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bf7fe640 1 --2- 172.21.15.2:0/2682756147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0071110 0x7f75c00ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75be7fc640 1 --2- 172.21.15.2:0/2682756147 >> v2:172.21.15.123:3300/0 conn(0x7f75c00c9000 0x7f75c00ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bdffb640 1 -- 172.21.15.2:0/2682756147 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630972436 0 0) 0x7f75c00ca3a0 con 0x7f75c0152b80 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bdffb640 1 -- 172.21.15.2:0/2682756147 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75ac003520 con 0x7f75c0152b80 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bf7fe640 1 -- 172.21.15.2:0/2682756147 >> v2:172.21.15.123:3300/0 conn(0x7f75c00c9000 msgr2=0x7f75c00ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bf7fe640 1 --2- 172.21.15.2:0/2682756147 >> v2:172.21.15.123:3300/0 conn(0x7f75c00c9000 0x7f75c00ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bf7fe640 1 -- 172.21.15.2:0/2682756147 >> v1:172.21.15.2:6789/0 conn(0x7f75c0152b80 legacy=0x7f75c01591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.101 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bdffb640 1 -- 172.21.15.2:0/2682756147 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3351977076 0 0) 0x7f75ac003520 con 0x7f75c0152b80 2024-09-19T05:45:32.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.098+0000 7f75bf7fe640 1 -- 172.21.15.2:0/2682756147 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c00c3750 con 0x7f75c0071110 2024-09-19T05:45:32.102 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75bf7fe640 1 --2- 172.21.15.2:0/2682756147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0071110 0x7f75c00ceab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75b80097a0 tx=0x7f75b802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14dad58982876d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75bdffb640 1 -- 172.21.15.2:0/2682756147 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8037070 con 0x7f75c0071110 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75bdffb640 1 -- 172.21.15.2:0/2682756147 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b802aba0 con 0x7f75c0071110 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75bdffb640 1 -- 172.21.15.2:0/2682756147 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0001a10 con 0x7f75c0071110 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 -- 172.21.15.2:0/2682756147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0071110 msgr2=0x7f75c00ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 --2- 172.21.15.2:0/2682756147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0071110 0x7f75c00ceab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75b80097a0 tx=0x7f75b802a110 comp rx=0 tx=0).stop 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 -- 172.21.15.2:0/2682756147 shutdown_connections 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 --2- 172.21.15.2:0/2682756147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0071110 0x7f75c00ceab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 --2- 172.21.15.2:0/2682756147 >> v2:172.21.15.123:3300/0 conn(0x7f75c00c9000 0x7f75c00ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 -- 172.21.15.2:0/2682756147 >> 172.21.15.2:0/2682756147 conn(0x7f75c006c240 msgr2=0x7f75c0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:32.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 -- 172.21.15.2:0/2682756147 shutdown_connections 2024-09-19T05:45:32.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.099+0000 7f75c562e640 1 -- 172.21.15.2:0/2682756147 wait complete. 2024-09-19T05:45:32.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 Processor -- start 2024-09-19T05:45:32.104 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 -- start start 2024-09-19T05:45:32.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c0071110 0x7f75c00c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c00c9000 0x7f75c01f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0152b80 0x7f75c0153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f75c01589d0 con 0x7f75c00c9000 2024-09-19T05:45:32.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75c00c3d10 con 0x7f75c0071110 2024-09-19T05:45:32.106 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.101+0000 7f75c562e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f75c0158cb0 con 0x7f75c0152b80 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c00c9000 0x7f75c01f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c00c9000 0x7f75c01f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60904/0 (socket says 172.21.15.2:60904) 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 -- 172.21.15.2:0/1053471925 learned_addr learned my addr 172.21.15.2:0/1053471925 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75bf7fe640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0152b80 0x7f75c0153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75beffd640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c0071110 0x7f75c00c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 -- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0152b80 msgr2=0x7f75c0153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0152b80 0x7f75c0153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 -- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c0071110 msgr2=0x7f75c00c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c0071110 0x7f75c00c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 -- 172.21.15.2:0/1053471925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c01f24c0 con 0x7f75c00c9000 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75bf7fe640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0152b80 0x7f75c0153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75be7fc640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c00c9000 0x7f75c01f1c00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75a800e870 tx=0x7f75a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75b77fe640 1 -- 172.21.15.2:0/1053471925 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8017070 con 0x7f75c00c9000 2024-09-19T05:45:32.107 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75b77fe640 1 -- 172.21.15.2:0/1053471925 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a800c940 con 0x7f75c00c9000 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c01f27a0 con 0x7f75c00c9000 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75b77fe640 1 -- 172.21.15.2:0/1053471925 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a800cd30 con 0x7f75c00c9000 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.102+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c01554a0 con 0x7f75c00c9000 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.103+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c0156500 con 0x7f75c00c9000 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.104+0000 7f75b77fe640 1 -- 172.21.15.2:0/1053471925 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f75a801a070 con 0x7f75c00c9000 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.104+0000 7f75b77fe640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f75a006c2b0 0x7f75a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.104+0000 7f75b77fe640 1 -- 172.21.15.2:0/1053471925 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f75a808b700 con 0x7f75c00c9000 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.104+0000 7f75beffd640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f75a006c2b0 0x7f75a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.108 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.105+0000 7f75beffd640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f75a006c2b0 0x7f75a006e770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005fd0 tx=0x7f75b0007e10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.111 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.109+0000 7f75b77fe640 1 -- 172.21.15.2:0/1053471925 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f75c0156500 con 0x7f75c00c9000 2024-09-19T05:45:32.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.283+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f75c0152f60 con 0x7f75a006c2b0 2024-09-19T05:45:32.286 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.284+0000 7f75b77fe640 1 -- 172.21.15.2:0/1053471925 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f75c0152f60 con 0x7f75a006c2b0 2024-09-19T05:45:32.286 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 6 to deep-scrub 2024-09-19T05:45:32.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f75a006c2b0 msgr2=0x7f75a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f75a006c2b0 0x7f75a006e770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f75b0005fd0 tx=0x7f75b0007e10 comp rx=0 tx=0).stop 2024-09-19T05:45:32.288 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c00c9000 msgr2=0x7f75c01f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c00c9000 0x7f75c01f1c00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f75a800e870 tx=0x7f75a800ed30 comp rx=0 tx=0).stop 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 shutdown_connections 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f75a006c2b0 0x7f75a006e770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c0152b80 0x7f75c0153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75c00c9000 0x7f75c01f1c00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 --2- 172.21.15.2:0/1053471925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c0071110 0x7f75c00c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 >> 172.21.15.2:0/1053471925 conn(0x7f75c006c240 msgr2=0x7f75c0071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 shutdown_connections 2024-09-19T05:45:32.289 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.286+0000 7f75c562e640 1 -- 172.21.15.2:0/1053471925 wait complete. 2024-09-19T05:45:32.299 INFO:tasks.ceph:Scrubbing osd.7 2024-09-19T05:45:32.300 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-19T05:45:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.404+0000 7f9362b11640 1 Processor -- start 2024-09-19T05:45:32.406 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.404+0000 7f9362b11640 1 -- start start 2024-09-19T05:45:32.409 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.404+0000 7f9362b11640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 0x7f935c15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.404+0000 7f9362b11640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c15ad20 con 0x7f935c067dc0 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.404+0000 7f9362b11640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c0c77f0 con 0x7f935c0c32b0 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.404+0000 7f9362b11640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f935c0c75a0 con 0x7f935c154f40 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935bfff640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f935c067dc0 0x7f935c072d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55528/0 (socket says 172.21.15.2:55528) 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935bfff640 1 -- 172.21.15.2:0/2377555813 learned_addr learned my addr 172.21.15.2:0/2377555813 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f93608ad640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 0x7f935c15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 66968476 0 0) 0x7f935c15ad20 con 0x7f935c067dc0 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f934c003520 con 0x7f935c067dc0 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 140498869 0 0) 0x7f934c003520 con 0x7f935c067dc0 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 msgr2=0x7f935c15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 0x7f935c15b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.410 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 >> v1:172.21.15.123:6789/0 conn(0x7f935c0c32b0 legacy=0x7f935c1654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c15afc0 con 0x7f935c067dc0 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f9348001670 con 0x7f935c067dc0 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 >> v1:172.21.15.2:6789/0 conn(0x7f935c067dc0 legacy=0x7f935c072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f934c003a50 0x7f934c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 0x7f934c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f934c004d40 0x7f934c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f935c0c75a0 con 0x7f934c003a50 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f934c00baf0 con 0x7f934c004d40 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f934c00bcf0 con 0x7f935c154f40 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f93608ad640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f934c003a50 0x7f934c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.405+0000 7f935bfff640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 0x7f934c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f93610ae640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f934c004d40 0x7f934c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f93608ad640 1 -- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 msgr2=0x7f934c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f93608ad640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 0x7f934c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f93608ad640 1 -- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f934c004d40 msgr2=0x7f934c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f93608ad640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f934c004d40 0x7f934c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f93608ad640 1 -- 172.21.15.2:0/2377555813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c003520 con 0x7f934c003a50 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f935bfff640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f935c154f40 0x7f934c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f93608ad640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f934c003a50 0x7f934c003e90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9350009ac0 tx=0x7f934c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902b471fd1ea4ead server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.406+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350036070 con 0x7f934c003a50 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9350031400 con 0x7f934c003a50 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f935b7fe640 1 -- 172.21.15.2:0/2377555813 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350009690 con 0x7f934c003a50 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f9362b11640 1 -- 172.21.15.2:0/2377555813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f934c003a50 msgr2=0x7f934c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f9362b11640 1 --2- 172.21.15.2:0/2377555813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f934c003a50 0x7f934c003e90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9350009ac0 tx=0x7f934c005830 comp rx=0 tx=0).stop 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f93610ae640 1 -- 172.21.15.2:0/2377555813 reap_dead start 2024-09-19T05:45:32.412 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f9362b11640 1 -- 172.21.15.2:0/2377555813 shutdown_connections 2024-09-19T05:45:32.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f9362b11640 1 -- 172.21.15.2:0/2377555813 >> 172.21.15.2:0/2377555813 conn(0x7f935c0c96a0 msgr2=0x7f935c0c9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:32.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f9362b11640 1 -- 172.21.15.2:0/2377555813 shutdown_connections 2024-09-19T05:45:32.413 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.407+0000 7f9362b11640 1 -- 172.21.15.2:0/2377555813 wait complete. 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.408+0000 7f9362b11640 1 Processor -- start 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.408+0000 7f9362b11640 1 -- start start 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f9362b11640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f934c003a50 0x7f935c0c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f9362b11640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c0c6b20 0x7f935c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f9362b11640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f935c1f1570 0x7f935c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f9362b11640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f935c0c7ad0 con 0x7f935c1f1570 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f9362b11640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f935c15b330 con 0x7f935c0c6b20 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f9362b11640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f935c073490 con 0x7f934c003a50 2024-09-19T05:45:32.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f93610ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f935c1f1570 0x7f935c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f93610ae640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f935c1f1570 0x7f935c158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60922/0 (socket says 172.21.15.2:60922) 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f93610ae640 1 -- 172.21.15.2:0/3307263681 learned_addr learned my addr 172.21.15.2:0/3307263681 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.409+0000 7f93608ad640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f934c003a50 0x7f935c0c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f935bfff640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c0c6b20 0x7f935c0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93610ae640 1 -- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f934c003a50 msgr2=0x7f935c0c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93610ae640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f934c003a50 0x7f935c0c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93610ae640 1 -- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c0c6b20 msgr2=0x7f935c0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93610ae640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c0c6b20 0x7f935c0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93610ae640 1 -- 172.21.15.2:0/3307263681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c151b40 con 0x7f935c1f1570 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f935bfff640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f935c0c6b20 0x7f935c0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93610ae640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f935c1f1570 0x7f935c158870 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f935c0caa70 tx=0x7f935400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354003970 con 0x7f935c1f1570 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9354018070 con 0x7f935c1f1570 2024-09-19T05:45:32.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354003d70 con 0x7f935c1f1570 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c0c4110 con 0x7f935c1f1570 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.410+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c0c46d0 con 0x7f935c1f1570 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.411+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f935c0b3070 con 0x7f935c1f1570 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.412+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f93540050d0 con 0x7f935c1f1570 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.412+0000 7f93597fa640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f934006c2b0 0x7f934006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.412+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f935408af60 con 0x7f935c1f1570 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.412+0000 7f93608ad640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f934006c2b0 0x7f934006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.412+0000 7f93597fa640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f93400744a0 0x7f9340076900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.412+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f935400bf30 con 0x7f93400744a0 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.412+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f935408d600 con 0x7f935c1f1570 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.413+0000 7f935bfff640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f93400744a0 0x7f9340076900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.413+0000 7f93608ad640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f934006c2b0 0x7f934006e770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9350033000 tx=0x7f9350008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.413+0000 7f935bfff640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f93400744a0 0x7f9340076900 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.418 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.414+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== osd.7 v2:172.21.15.123:6808/1657801827 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f935400bf30 con 0x7f93400744a0 2024-09-19T05:45:32.428 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.426+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 --> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f935c1520c0 con 0x7f93400744a0 2024-09-19T05:45:32.429 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.427+0000 7f93597fa640 1 -- 172.21.15.2:0/3307263681 <== osd.7 v2:172.21.15.123:6808/1657801827 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f935c1520c0 con 0x7f93400744a0 2024-09-19T05:45:32.429 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-19T05:45:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T05:45:32.430 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-19T05:45:32.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.427+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f93400744a0 msgr2=0x7f9340076900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.430 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.427+0000 7f9362b11640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6808/1657801827,v1:172.21.15.123:6809/1657801827] conn(0x7f93400744a0 0x7f9340076900 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.427+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f934006c2b0 msgr2=0x7f934006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.427+0000 7f9362b11640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f934006c2b0 0x7f934006e770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9350033000 tx=0x7f9350008030 comp rx=0 tx=0).stop 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.428+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f935c1f1570 msgr2=0x7f935c158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.428+0000 7f9362b11640 1 --2- 172.21.15.2:0/3307263681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f935c1f1570 0x7f935c158870 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f935c0caa70 tx=0x7f935400eb90 comp rx=0 tx=0).stop 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.428+0000 7f93610ae640 1 -- 172.21.15.2:0/3307263681 reap_dead start 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.428+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 shutdown_connections 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.428+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 >> 172.21.15.2:0/3307263681 conn(0x7f935c0c96a0 msgr2=0x7f935c06cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.428+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 shutdown_connections 2024-09-19T05:45:32.431 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.428+0000 7f9362b11640 1 -- 172.21.15.2:0/3307263681 wait complete. 2024-09-19T05:45:32.440 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-19T05:45:32.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.541+0000 7f948665e640 1 Processor -- start 2024-09-19T05:45:32.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.541+0000 7f948665e640 1 -- start start 2024-09-19T05:45:32.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f948665e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f948015adb0 0x7f948015b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f948665e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 0x7f9480072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f948665e640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9480070500 con 0x7f948015b6d0 2024-09-19T05:45:32.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f948665e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9480071f80 con 0x7f948015adb0 2024-09-19T05:45:32.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f948665e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9480072820 con 0x7f9480155030 2024-09-19T05:45:32.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947f7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f948015b6d0 0x7f9480154920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55534/0 (socket says 172.21.15.2:55534) 2024-09-19T05:45:32.546 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947f7fe640 1 -- 172.21.15.2:0/4058024822 learned_addr learned my addr 172.21.15.2:0/4058024822 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f9484bfb640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 0x7f9480072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947ffff640 1 --2- 172.21.15.2:0/4058024822 >> v2:172.21.15.123:3300/0 conn(0x7f948015adb0 0x7f948015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2272358937 0 0) 0x7f9480070500 con 0x7f948015b6d0 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9468003520 con 0x7f948015b6d0 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 148169802 0 0) 0x7f9468003520 con 0x7f948015b6d0 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 msgr2=0x7f9480072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 0x7f9480072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 >> v2:172.21.15.123:3300/0 conn(0x7f948015adb0 msgr2=0x7f948015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 --2- 172.21.15.2:0/4058024822 >> v2:172.21.15.123:3300/0 conn(0x7f948015adb0 0x7f948015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94800707e0 con 0x7f948015b6d0 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f9474001670 con 0x7f948015b6d0 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947ffff640 1 --2- 172.21.15.2:0/4058024822 >> v2:172.21.15.123:3300/0 conn(0x7f948015adb0 0x7f948015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 >> v1:172.21.15.2:6789/0 conn(0x7f948015b6d0 legacy=0x7f9480154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.547 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 0x7f94680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.542+0000 7f947effd640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9468003f30 0x7f9468004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947effd640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468004d40 0x7f94680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f9480071f80 con 0x7f9468003f30 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9480072820 con 0x7f9468004d40 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f946800b9f0 con 0x7f9480155030 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947f7fe640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9468003f30 0x7f9468004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947ffff640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 0x7f94680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f9484bfb640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468004d40 0x7f94680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947f7fe640 1 -- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 msgr2=0x7f94680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947f7fe640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9480155030 0x7f94680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947f7fe640 1 -- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468004d40 msgr2=0x7f94680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947f7fe640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468004d40 0x7f94680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.543+0000 7f947f7fe640 1 -- 172.21.15.2:0/4058024822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9468003520 con 0x7f9468003f30 2024-09-19T05:45:32.548 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f947f7fe640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9468003f30 0x7f9468004800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f947400c240 tx=0x7f9468005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac7b603afe27c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947402e070 con 0x7f9468003f30 2024-09-19T05:45:32.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9474005af0 con 0x7f9468003f30 2024-09-19T05:45:32.549 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f947effd640 1 -- 172.21.15.2:0/4058024822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474007d30 con 0x7f9468003f30 2024-09-19T05:45:32.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f948665e640 1 -- 172.21.15.2:0/4058024822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9468003f30 msgr2=0x7f9468004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f948665e640 1 --2- 172.21.15.2:0/4058024822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9468003f30 0x7f9468004800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f947400c240 tx=0x7f9468005830 comp rx=0 tx=0).stop 2024-09-19T05:45:32.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f9484bfb640 1 -- 172.21.15.2:0/4058024822 reap_dead start 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f948665e640 1 -- 172.21.15.2:0/4058024822 shutdown_connections 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f948665e640 1 -- 172.21.15.2:0/4058024822 >> 172.21.15.2:0/4058024822 conn(0x7f94800cc880 msgr2=0x7f94800ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f948665e640 1 -- 172.21.15.2:0/4058024822 shutdown_connections 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.544+0000 7f948665e640 1 -- 172.21.15.2:0/4058024822 wait complete. 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.546+0000 7f948665e640 1 Processor -- start 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.546+0000 7f948665e640 1 -- start start 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.546+0000 7f948665e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468003f30 0x7f94800c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.546+0000 7f948665e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94801f49d0 0x7f94801532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.546+0000 7f948665e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f948014ff40 0x7f94800c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.546+0000 7f948665e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f948006edb0 con 0x7f948014ff40 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.546+0000 7f948665e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94800724b0 con 0x7f9468003f30 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f948665e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f94800700e0 con 0x7f94801f49d0 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f948014ff40 0x7f94800c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f948014ff40 0x7f94800c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60946/0 (socket says 172.21.15.2:60946) 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 -- 172.21.15.2:0/124360315 learned_addr learned my addr 172.21.15.2:0/124360315 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f947f7fe640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94801f49d0 0x7f94801532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f947ffff640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468003f30 0x7f94800c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 -- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94801f49d0 msgr2=0x7f94801532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94801f49d0 0x7f94801532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 -- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468003f30 msgr2=0x7f94800c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468003f30 0x7f94800c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 -- 172.21.15.2:0/124360315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801f4ec0 con 0x7f948014ff40 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f947f7fe640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94801f49d0 0x7f94801532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f947ffff640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468003f30 0x7f94800c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f9484bfb640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f948014ff40 0x7f94800c4fe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9478009fd0 tx=0x7f947800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f947cff9640 1 -- 172.21.15.2:0/124360315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478019070 con 0x7f948014ff40 2024-09-19T05:45:32.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f947cff9640 1 -- 172.21.15.2:0/124360315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9478011e90 con 0x7f948014ff40 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801f0bf0 con 0x7f948014ff40 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f947cff9640 1 -- 172.21.15.2:0/124360315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478003bd0 con 0x7f948014ff40 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.547+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801f1170 con 0x7f948014ff40 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.548+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94800cd4a0 con 0x7f948014ff40 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.549+0000 7f947cff9640 1 -- 172.21.15.2:0/124360315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f947801c070 con 0x7f948014ff40 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.549+0000 7f947cff9640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f946c06c2e0 0x7f946c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.549+0000 7f947cff9640 1 -- 172.21.15.2:0/124360315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f947808bc60 con 0x7f948014ff40 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.549+0000 7f947ffff640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f946c06c2e0 0x7f946c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.550+0000 7f947ffff640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f946c06c2e0 0x7f946c06e7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9470009c40 tx=0x7f9470008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.555 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.553+0000 7f947cff9640 1 -- 172.21.15.2:0/124360315 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f9478010040 con 0x7f948014ff40 2024-09-19T05:45:32.732 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.730+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f94800c3690 con 0x7f946c06c2e0 2024-09-19T05:45:32.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.731+0000 7f947cff9640 1 -- 172.21.15.2:0/124360315 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f94800c3690 con 0x7f946c06c2e0 2024-09-19T05:45:32.733 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 7 to deep-scrub 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f946c06c2e0 msgr2=0x7f946c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f946c06c2e0 0x7f946c06e7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9470009c40 tx=0x7f9470008040 comp rx=0 tx=0).stop 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f948014ff40 msgr2=0x7f94800c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f948014ff40 0x7f94800c4fe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9478009fd0 tx=0x7f947800f580 comp rx=0 tx=0).stop 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 shutdown_connections 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f946c06c2e0 0x7f946c06e7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f948014ff40 0x7f94800c4fe0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f94801f49d0 0x7f94801532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 --2- 172.21.15.2:0/124360315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9468003f30 0x7f94800c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 >> 172.21.15.2:0/124360315 conn(0x7f94800cc880 msgr2=0x7f9480148ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 shutdown_connections 2024-09-19T05:45:32.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.733+0000 7f948665e640 1 -- 172.21.15.2:0/124360315 wait complete. 2024-09-19T05:45:32.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:45:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.848+0000 7f8de58e4640 1 Processor -- start 2024-09-19T05:45:32.850 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.848+0000 7f8de58e4640 1 -- start start 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.848+0000 7f8de58e4640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8de00c5620 0x7f8de00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8de58e4640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de0151bb0 con 0x7f8de00c9000 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8de58e4640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de015ac60 con 0x7f8de00cbcc0 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8de58e4640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8de015b870 con 0x7f8de00c5620 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8ddf7fe640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8de00c9000 0x7f8de015e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:55540/0 (socket says 172.21.15.2:55540) 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/1968008304 learned_addr learned my addr 172.21.15.2:0/1968008304 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8ddeffd640 1 --2- 172.21.15.2:0/1968008304 >> v2:172.21.15.123:3300/0 conn(0x7f8de00c5620 0x7f8de00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531027317 0 0) 0x7f8de0151bb0 con 0x7f8de00c9000 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dc8003520 con 0x7f8de00c9000 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3499252177 0 0) 0x7f8dc8003520 con 0x7f8de00c9000 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 >> v2:172.21.15.123:3300/0 conn(0x7f8de00c5620 msgr2=0x7f8de00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.854 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 --2- 172.21.15.2:0/1968008304 >> v2:172.21.15.123:3300/0 conn(0x7f8de00c5620 0x7f8de00c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 >> v1:172.21.15.123:6789/0 conn(0x7f8de00cbcc0 legacy=0x7f8de00ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de015af40 con 0x7f8de00c9000 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f8dd80019f0 con 0x7f8de00c9000 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 >> v1:172.21.15.2:6789/0 conn(0x7f8de00c9000 legacy=0x7f8de015e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dc8003a20 0x7f8dc8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc8004800 0x7f8dc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80051c0 0x7f8dc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8de015b870 con 0x7f8dc80051c0 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc800c0d0 con 0x7f8dc8004800 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc800c2d0 con 0x7f8dc8003a20 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.849+0000 7f8ddf7fe640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80051c0 0x7f8dc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddeffd640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dc8003a20 0x7f8dc8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8dde7fc640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc8004800 0x7f8dc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dc8003a20 msgr2=0x7f8dc8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddf7fe640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8dc8003a20 0x7f8dc8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc8004800 msgr2=0x7f8dc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddf7fe640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc8004800 0x7f8dc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/1968008304 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc8003520 con 0x7f8dc80051c0 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/1968008304 reap_dead start 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.850+0000 7f8ddf7fe640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80051c0 0x7f8dc800b9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800a860 tx=0x7f8dc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c83850aab32a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800a480 con 0x7f8dc80051c0 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd802f070 con 0x7f8dc80051c0 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8dddffb640 1 -- 172.21.15.2:0/1968008304 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd802bcc0 con 0x7f8dc80051c0 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8de58e4640 1 -- 172.21.15.2:0/1968008304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80051c0 msgr2=0x7f8dc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8de58e4640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80051c0 0x7f8dc800b9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800a860 tx=0x7f8dc8005830 comp rx=0 tx=0).stop 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8de58e4640 1 -- 172.21.15.2:0/1968008304 shutdown_connections 2024-09-19T05:45:32.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8de58e4640 1 --2- 172.21.15.2:0/1968008304 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80051c0 0x7f8dc800b9c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8de58e4640 1 -- 172.21.15.2:0/1968008304 >> 172.21.15.2:0/1968008304 conn(0x7f8de0070f90 msgr2=0x7f8de006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:32.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8de58e4640 1 -- 172.21.15.2:0/1968008304 shutdown_connections 2024-09-19T05:45:32.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.851+0000 7f8de58e4640 1 -- 172.21.15.2:0/1968008304 wait complete. 2024-09-19T05:45:32.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.852+0000 7f8de58e4640 1 Processor -- start 2024-09-19T05:45:32.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8de58e4640 1 -- start start 2024-09-19T05:45:32.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8de58e4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc80051c0 0x7f8de006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8de58e4640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de0159880 0x7f8de01f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8de58e4640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de015d9c0 0x7f8de01f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8de58e4640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f8de015a910 con 0x7f8de015d9c0 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8de58e4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8de015b500 con 0x7f8dc80051c0 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8de58e4640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8de00c9830 con 0x7f8de0159880 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.853+0000 7f8ddf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de015d9c0 0x7f8de01f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de015d9c0 0x7f8de01f11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60966/0 (socket says 172.21.15.2:60966) 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/2250447737 learned_addr learned my addr 172.21.15.2:0/2250447737 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddeffd640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc80051c0 0x7f8de006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8dde7fc640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de0159880 0x7f8de01f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de0159880 msgr2=0x7f8de01f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de0159880 0x7f8de01f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc80051c0 msgr2=0x7f8de006d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc80051c0 0x7f8de006d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 -- 172.21.15.2:0/2250447737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de014fce0 con 0x7f8de015d9c0 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddeffd640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc80051c0 0x7f8de006d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8ddf7fe640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de015d9c0 0x7f8de01f11d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8dd802b530 tx=0x7f8de015ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8dc7fff640 1 -- 172.21.15.2:0/2250447737 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8044070 con 0x7f8de015d9c0 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8dc7fff640 1 -- 172.21.15.2:0/2250447737 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd802bdc0 con 0x7f8de015d9c0 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de014ffa0 con 0x7f8de015d9c0 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8dc7fff640 1 -- 172.21.15.2:0/2250447737 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd800a480 con 0x7f8de015d9c0 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.854+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de015a230 con 0x7f8de015d9c0 2024-09-19T05:45:32.861 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.855+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de00ca810 con 0x7f8de015d9c0 2024-09-19T05:45:32.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.856+0000 7f8dc7fff640 1 -- 172.21.15.2:0/2250447737 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8dd8047070 con 0x7f8de015d9c0 2024-09-19T05:45:32.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.856+0000 7f8dc7fff640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c2b0 0x7f8dcc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:32.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.857+0000 7f8dc7fff640 1 -- 172.21.15.2:0/2250447737 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f8dd80ab170 con 0x7f8de015d9c0 2024-09-19T05:45:32.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.857+0000 7f8ddeffd640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c2b0 0x7f8dcc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:32.862 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.857+0000 7f8ddeffd640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c2b0 0x7f8dcc06e770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009b40 tx=0x7f8dd0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:32.863 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:32.861+0000 7f8dc7fff640 1 -- 172.21.15.2:0/2250447737 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f8dd8079880 con 0x7f8de015d9c0 2024-09-19T05:45:33.036 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.034+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8de00c3820 con 0x7f8dcc06c2b0 2024-09-19T05:45:33.042 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.041+0000 7f8dc7fff640 1 -- 172.21.15.2:0/2250447737 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+253298 (secure 0 0 0) 0x7f8de00c3820 con 0x7f8dcc06c2b0 2024-09-19T05:45:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:33.047 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:45:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c2b0 msgr2=0x7f8dcc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c2b0 0x7f8dcc06e770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009b40 tx=0x7f8dd0008040 comp rx=0 tx=0).stop 2024-09-19T05:45:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de015d9c0 msgr2=0x7f8de01f11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de015d9c0 0x7f8de01f11d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8dd802b530 tx=0x7f8de015ae30 comp rx=0 tx=0).stop 2024-09-19T05:45:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 shutdown_connections 2024-09-19T05:45:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f8dcc06c2b0 0x7f8dcc06e770 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:33.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8de015d9c0 0x7f8de01f11d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8de0159880 0x7f8de01f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 --2- 172.21.15.2:0/2250447737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8dc80051c0 0x7f8de006d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 >> 172.21.15.2:0/2250447737 conn(0x7f8de0070f90 msgr2=0x7f8de00c47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 shutdown_connections 2024-09-19T05:45:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:33.043+0000 7f8de58e4640 1 -- 172.21.15.2:0/2250447737 wait complete. 2024-09-19T05:45:33.057 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-09-19T05:45:31.856908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560990,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13285,"num_write_kb":25179742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15268,"ondisk_log_size":15268,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342232,"kb_used_data":21976,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632488,"statfs":{"total":773094113280,"available":772743667712,"internally_reserved":0,"allocated":22503424,"data_stored":21604268,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.171248"},"pg_stats":[{"pgid":"3.3f","version":"102'167","reported_seq":340,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.568120+0000","last_change":"2024-09-19T05:42:32.947221+0000","last_active":"2024-09-19T05:45:26.568120+0000","last_peered":"2024-09-19T05:45:26.568120+0000","last_clean":"2024-09-19T05:45:26.568120+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:26.568120+0000","last_undegraded":"2024-09-19T05:45:26.568120+0000","last_fullsized":"2024-09-19T05:45:26.568120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"102'152","reported_seq":305,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.553720+0000","last_change":"2024-09-19T05:44:54.646789+0000","last_active":"2024-09-19T05:45:26.553720+0000","last_peered":"2024-09-19T05:45:26.553720+0000","last_clean":"2024-09-19T05:45:26.553720+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:26.553720+0000","last_undegraded":"2024-09-19T05:45:26.553720+0000","last_fullsized":"2024-09-19T05:45:26.553720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:11.761535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"102'178","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.553564+0000","last_change":"2024-09-19T05:44:53.475804+0000","last_active":"2024-09-19T05:45:26.553564+0000","last_peered":"2024-09-19T05:45:26.553564+0000","last_clean":"2024-09-19T05:45:26.553564+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:26.553564+0000","last_undegraded":"2024-09-19T05:45:26.553564+0000","last_fullsized":"2024-09-19T05:45:26.553564+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:07:57.009039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"102'193","reported_seq":365,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.547675+0000","last_change":"2024-09-19T05:42:32.937324+0000","last_active":"2024-09-19T05:45:26.547675+0000","last_peered":"2024-09-19T05:45:26.547675+0000","last_clean":"2024-09-19T05:45:26.547675+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:26.547675+0000","last_undegraded":"2024-09-19T05:45:26.547675+0000","last_fullsized":"2024-09-19T05:45:26.547675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.090391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"102'174","reported_seq":443,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.561436+0000","last_change":"2024-09-19T05:44:53.953498+0000","last_active":"2024-09-19T05:45:26.561436+0000","last_peered":"2024-09-19T05:45:26.561436+0000","last_clean":"2024-09-19T05:45:26.561436+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:26.561436+0000","last_undegraded":"2024-09-19T05:45:26.561436+0000","last_fullsized":"2024-09-19T05:45:26.561436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"102'162","reported_seq":334,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.547541+0000","last_change":"2024-09-19T05:42:32.936483+0000","last_active":"2024-09-19T05:45:26.547541+0000","last_peered":"2024-09-19T05:45:26.547541+0000","last_clean":"2024-09-19T05:45:26.547541+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:26.547541+0000","last_undegraded":"2024-09-19T05:45:26.547541+0000","last_fullsized":"2024-09-19T05:45:26.547541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:09.462341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"102'188","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.564903+0000","last_change":"2024-09-19T05:42:32.939559+0000","last_active":"2024-09-19T05:45:26.564903+0000","last_peered":"2024-09-19T05:45:26.564903+0000","last_clean":"2024-09-19T05:45:26.564903+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:26.564903+0000","last_undegraded":"2024-09-19T05:45:26.564903+0000","last_fullsized":"2024-09-19T05:45:26.564903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"102'204","reported_seq":366,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.557584+0000","last_change":"2024-09-19T05:44:56.567156+0000","last_active":"2024-09-19T05:45:26.557584+0000","last_peered":"2024-09-19T05:45:26.557584+0000","last_clean":"2024-09-19T05:45:26.557584+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:26.557584+0000","last_undegraded":"2024-09-19T05:45:26.557584+0000","last_fullsized":"2024-09-19T05:45:26.557584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:17.811359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"102'178","reported_seq":291,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.538066+0000","last_change":"2024-09-19T05:45:10.627715+0000","last_active":"2024-09-19T05:45:26.538066+0000","last_peered":"2024-09-19T05:45:26.538066+0000","last_clean":"2024-09-19T05:45:26.538066+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:26.538066+0000","last_undegraded":"2024-09-19T05:45:26.538066+0000","last_fullsized":"2024-09-19T05:45:26.538066+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:38.174880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"102'190","reported_seq":363,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.567925+0000","last_change":"2024-09-19T05:42:32.934678+0000","last_active":"2024-09-19T05:45:26.567925+0000","last_peered":"2024-09-19T05:45:26.567925+0000","last_clean":"2024-09-19T05:45:26.567925+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:26.567925+0000","last_undegraded":"2024-09-19T05:45:26.567925+0000","last_fullsized":"2024-09-19T05:45:26.567925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"102'214","reported_seq":387,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.568213+0000","last_change":"2024-09-19T05:42:32.946174+0000","last_active":"2024-09-19T05:45:26.568213+0000","last_peered":"2024-09-19T05:45:26.568213+0000","last_clean":"2024-09-19T05:45:26.568213+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:26.568213+0000","last_undegraded":"2024-09-19T05:45:26.568213+0000","last_fullsized":"2024-09-19T05:45:26.568213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"102'166","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.567794+0000","last_change":"2024-09-19T05:43:19.551369+0000","last_active":"2024-09-19T05:45:26.567794+0000","last_peered":"2024-09-19T05:45:26.567794+0000","last_clean":"2024-09-19T05:45:26.567794+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:26.567794+0000","last_undegraded":"2024-09-19T05:45:26.567794+0000","last_fullsized":"2024-09-19T05:45:26.567794+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"102'177","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.562384+0000","last_change":"2024-09-19T05:42:32.943260+0000","last_active":"2024-09-19T05:45:26.562384+0000","last_peered":"2024-09-19T05:45:26.562384+0000","last_clean":"2024-09-19T05:45:26.562384+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:26.562384+0000","last_undegraded":"2024-09-19T05:45:26.562384+0000","last_fullsized":"2024-09-19T05:45:26.562384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:17.915074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"102'172","reported_seq":344,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.542578+0000","last_change":"2024-09-19T05:42:32.933366+0000","last_active":"2024-09-19T05:45:26.542578+0000","last_peered":"2024-09-19T05:45:26.542578+0000","last_clean":"2024-09-19T05:45:26.542578+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:26.542578+0000","last_undegraded":"2024-09-19T05:45:26.542578+0000","last_fullsized":"2024-09-19T05:45:26.542578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:09.158454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"102'177","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.553340+0000","last_change":"2024-09-19T05:44:53.930699+0000","last_active":"2024-09-19T05:45:26.553340+0000","last_peered":"2024-09-19T05:45:26.553340+0000","last_clean":"2024-09-19T05:45:26.553340+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:26.553340+0000","last_undegraded":"2024-09-19T05:45:26.553340+0000","last_fullsized":"2024-09-19T05:45:26.553340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:53:02.486476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"102'162","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.556507+0000","last_change":"2024-09-19T05:42:32.939603+0000","last_active":"2024-09-19T05:45:26.556507+0000","last_peered":"2024-09-19T05:45:26.556507+0000","last_clean":"2024-09-19T05:45:26.556507+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:26.556507+0000","last_undegraded":"2024-09-19T05:45:26.556507+0000","last_fullsized":"2024-09-19T05:45:26.556507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"102'176","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.564637+0000","last_change":"2024-09-19T05:42:32.942153+0000","last_active":"2024-09-19T05:45:26.564637+0000","last_peered":"2024-09-19T05:45:26.564637+0000","last_clean":"2024-09-19T05:45:26.564637+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:26.564637+0000","last_undegraded":"2024-09-19T05:45:26.564637+0000","last_fullsized":"2024-09-19T05:45:26.564637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:08.449161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"102'179","reported_seq":352,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.550906+0000","last_change":"2024-09-19T05:42:32.933374+0000","last_active":"2024-09-19T05:45:26.550906+0000","last_peered":"2024-09-19T05:45:26.550906+0000","last_clean":"2024-09-19T05:45:26.550906+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:26.550906+0000","last_undegraded":"2024-09-19T05:45:26.550906+0000","last_fullsized":"2024-09-19T05:45:26.550906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:01.370141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"102'205","reported_seq":376,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.552465+0000","last_change":"2024-09-19T05:42:32.942695+0000","last_active":"2024-09-19T05:45:26.552465+0000","last_peered":"2024-09-19T05:45:26.552465+0000","last_clean":"2024-09-19T05:45:26.552465+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:26.552465+0000","last_undegraded":"2024-09-19T05:45:26.552465+0000","last_fullsized":"2024-09-19T05:45:26.552465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:08.527311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"102'179","reported_seq":362,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.568053+0000","last_change":"2024-09-19T05:43:19.551782+0000","last_active":"2024-09-19T05:45:26.568053+0000","last_peered":"2024-09-19T05:45:26.568053+0000","last_clean":"2024-09-19T05:45:26.568053+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:26.568053+0000","last_undegraded":"2024-09-19T05:45:26.568053+0000","last_fullsized":"2024-09-19T05:45:26.568053+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"102'154","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.548850+0000","last_change":"2024-09-19T05:42:32.933068+0000","last_active":"2024-09-19T05:45:26.548850+0000","last_peered":"2024-09-19T05:45:26.548850+0000","last_clean":"2024-09-19T05:45:26.548850+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:26.548850+0000","last_undegraded":"2024-09-19T05:45:26.548850+0000","last_fullsized":"2024-09-19T05:45:26.548850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:21:35.077079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"102'199","reported_seq":256,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.556618+0000","last_change":"2024-09-19T05:44:55.559370+0000","last_active":"2024-09-19T05:45:26.556618+0000","last_peered":"2024-09-19T05:45:26.556618+0000","last_clean":"2024-09-19T05:45:26.556618+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:26.556618+0000","last_undegraded":"2024-09-19T05:45:26.556618+0000","last_fullsized":"2024-09-19T05:45:26.556618+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:25.838393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"102'150","reported_seq":303,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.556397+0000","last_change":"2024-09-19T05:44:56.134769+0000","last_active":"2024-09-19T05:45:26.556397+0000","last_peered":"2024-09-19T05:45:26.556397+0000","last_clean":"2024-09-19T05:45:26.556397+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:26.556397+0000","last_undegraded":"2024-09-19T05:45:26.556397+0000","last_fullsized":"2024-09-19T05:45:26.556397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:06.255089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"102'207","reported_seq":498,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.554133+0000","last_change":"2024-09-19T05:44:56.565770+0000","last_active":"2024-09-19T05:45:26.554133+0000","last_peered":"2024-09-19T05:45:26.554133+0000","last_clean":"2024-09-19T05:45:26.554133+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:26.554133+0000","last_undegraded":"2024-09-19T05:45:26.554133+0000","last_fullsized":"2024-09-19T05:45:26.554133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:03.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"102'160","reported_seq":321,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.545481+0000","last_change":"2024-09-19T05:44:56.582314+0000","last_active":"2024-09-19T05:45:26.545481+0000","last_peered":"2024-09-19T05:45:26.545481+0000","last_clean":"2024-09-19T05:45:26.545481+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:26.545481+0000","last_undegraded":"2024-09-19T05:45:26.545481+0000","last_fullsized":"2024-09-19T05:45:26.545481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:41.223871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"102'217","reported_seq":389,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.552609+0000","last_change":"2024-09-19T05:42:32.944389+0000","last_active":"2024-09-19T05:45:26.552609+0000","last_peered":"2024-09-19T05:45:26.552609+0000","last_clean":"2024-09-19T05:45:26.552609+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:26.552609+0000","last_undegraded":"2024-09-19T05:45:26.552609+0000","last_fullsized":"2024-09-19T05:45:26.552609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:32.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"102'166","reported_seq":338,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.543750+0000","last_change":"2024-09-19T05:42:32.933274+0000","last_active":"2024-09-19T05:45:26.543750+0000","last_peered":"2024-09-19T05:45:26.543750+0000","last_clean":"2024-09-19T05:45:26.543750+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:26.543750+0000","last_undegraded":"2024-09-19T05:45:26.543750+0000","last_fullsized":"2024-09-19T05:45:26.543750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"102'152","reported_seq":324,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.562473+0000","last_change":"2024-09-19T05:42:32.943264+0000","last_active":"2024-09-19T05:45:26.562473+0000","last_peered":"2024-09-19T05:45:26.562473+0000","last_clean":"2024-09-19T05:45:26.562473+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:26.562473+0000","last_undegraded":"2024-09-19T05:45:26.562473+0000","last_fullsized":"2024-09-19T05:45:26.562473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"102'134","reported_seq":288,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.552669+0000","last_change":"2024-09-19T05:44:54.977580+0000","last_active":"2024-09-19T05:45:26.552669+0000","last_peered":"2024-09-19T05:45:26.552669+0000","last_clean":"2024-09-19T05:45:26.552669+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:26.552669+0000","last_undegraded":"2024-09-19T05:45:26.552669+0000","last_fullsized":"2024-09-19T05:45:26.552669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:12.165530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"102'153","reported_seq":416,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.554123+0000","last_change":"2024-09-19T05:44:53.510921+0000","last_active":"2024-09-19T05:45:26.554123+0000","last_peered":"2024-09-19T05:45:26.554123+0000","last_clean":"2024-09-19T05:45:26.554123+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:26.554123+0000","last_undegraded":"2024-09-19T05:45:26.554123+0000","last_fullsized":"2024-09-19T05:45:26.554123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:39.530204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"102'158","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.541234+0000","last_change":"2024-09-19T05:43:19.548711+0000","last_active":"2024-09-19T05:45:26.541234+0000","last_peered":"2024-09-19T05:45:26.541234+0000","last_clean":"2024-09-19T05:45:26.541234+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:26.541234+0000","last_undegraded":"2024-09-19T05:45:26.541234+0000","last_fullsized":"2024-09-19T05:45:26.541234+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:32.263788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"102'180","reported_seq":352,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.543359+0000","last_change":"2024-09-19T05:42:32.934726+0000","last_active":"2024-09-19T05:45:26.543359+0000","last_peered":"2024-09-19T05:45:26.543359+0000","last_clean":"2024-09-19T05:45:26.543359+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:26.543359+0000","last_undegraded":"2024-09-19T05:45:26.543359+0000","last_fullsized":"2024-09-19T05:45:26.543359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:13.158126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"102'149","reported_seq":247,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.545568+0000","last_change":"2024-09-19T05:45:10.624968+0000","last_active":"2024-09-19T05:45:26.545568+0000","last_peered":"2024-09-19T05:45:26.545568+0000","last_clean":"2024-09-19T05:45:26.545568+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:26.545568+0000","last_undegraded":"2024-09-19T05:45:26.545568+0000","last_fullsized":"2024-09-19T05:45:26.545568+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:53:59.539763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"102'1811","reported_seq":1887,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.513334+0000","last_change":"2024-09-19T05:44:55.560257+0000","last_active":"2024-09-19T05:45:26.513334+0000","last_peered":"2024-09-19T05:45:26.513334+0000","last_clean":"2024-09-19T05:45:26.513334+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:26.513334+0000","last_undegraded":"2024-09-19T05:45:26.513334+0000","last_fullsized":"2024-09-19T05:45:26.513334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1811,"log_dups_size":0,"ondisk_log_size":1811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:25.647547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774343,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":5432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"102'162","reported_seq":334,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.563687+0000","last_change":"2024-09-19T05:42:32.935696+0000","last_active":"2024-09-19T05:45:26.563687+0000","last_peered":"2024-09-19T05:45:26.563687+0000","last_clean":"2024-09-19T05:45:26.563687+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:26.563687+0000","last_undegraded":"2024-09-19T05:45:26.563687+0000","last_fullsized":"2024-09-19T05:45:26.563687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:18.241258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716460+0000","last_change":"2024-09-19T05:42:31.996189+0000","last_active":"2024-09-19T05:45:21.716460+0000","last_peered":"2024-09-19T05:45:21.716460+0000","last_clean":"2024-09-19T05:45:21.716460+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:21.716460+0000","last_undegraded":"2024-09-19T05:45:21.716460+0000","last_fullsized":"2024-09-19T05:45:21.716460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:16.912760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773054+0000","last_change":"2024-09-19T05:42:31.991795+0000","last_active":"2024-09-19T05:45:21.773054+0000","last_peered":"2024-09-19T05:45:21.773054+0000","last_clean":"2024-09-19T05:45:21.773054+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:21.773054+0000","last_undegraded":"2024-09-19T05:45:21.773054+0000","last_fullsized":"2024-09-19T05:45:21.773054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"102'190","reported_seq":345,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.554501+0000","last_change":"2024-09-19T05:44:59.989289+0000","last_active":"2024-09-19T05:45:26.554501+0000","last_peered":"2024-09-19T05:45:26.554501+0000","last_clean":"2024-09-19T05:45:26.554501+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:26.554501+0000","last_undegraded":"2024-09-19T05:45:26.554501+0000","last_fullsized":"2024-09-19T05:45:26.554501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:52.052457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716243+0000","last_change":"2024-09-19T05:43:19.543746+0000","last_active":"2024-09-19T05:45:21.716243+0000","last_peered":"2024-09-19T05:45:21.716243+0000","last_clean":"2024-09-19T05:45:21.716243+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:21.716243+0000","last_undegraded":"2024-09-19T05:45:21.716243+0000","last_fullsized":"2024-09-19T05:45:21.716243+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:16.764095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"102'187","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.545364+0000","last_change":"2024-09-19T05:43:19.549632+0000","last_active":"2024-09-19T05:45:26.545364+0000","last_peered":"2024-09-19T05:45:26.545364+0000","last_clean":"2024-09-19T05:45:26.545364+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:26.545364+0000","last_undegraded":"2024-09-19T05:45:26.545364+0000","last_fullsized":"2024-09-19T05:45:26.545364+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:51.370035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":77,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.190091+0000","last_change":"2024-09-19T05:45:10.622931+0000","last_active":"2024-09-19T05:45:22.190091+0000","last_peered":"2024-09-19T05:45:22.190091+0000","last_clean":"2024-09-19T05:45:22.190091+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:22.190091+0000","last_undegraded":"2024-09-19T05:45:22.190091+0000","last_fullsized":"2024-09-19T05:45:22.190091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:34:06.182707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"102'251","reported_seq":423,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.557363+0000","last_change":"2024-09-19T05:42:32.944142+0000","last_active":"2024-09-19T05:45:26.557363+0000","last_peered":"2024-09-19T05:45:26.557363+0000","last_clean":"2024-09-19T05:45:26.557363+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:26.557363+0000","last_undegraded":"2024-09-19T05:45:26.557363+0000","last_fullsized":"2024-09-19T05:45:26.557363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:35:52.333110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":185,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688537+0000","last_change":"2024-09-19T05:43:19.552125+0000","last_active":"2024-09-19T05:45:21.688537+0000","last_peered":"2024-09-19T05:45:21.688537+0000","last_clean":"2024-09-19T05:45:21.688537+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:21.688537+0000","last_undegraded":"2024-09-19T05:45:21.688537+0000","last_fullsized":"2024-09-19T05:45:21.688537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"102'160","reported_seq":309,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.556173+0000","last_change":"2024-09-19T05:44:59.517543+0000","last_active":"2024-09-19T05:45:26.556173+0000","last_peered":"2024-09-19T05:45:26.556173+0000","last_clean":"2024-09-19T05:45:26.556173+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:26.556173+0000","last_undegraded":"2024-09-19T05:45:26.556173+0000","last_fullsized":"2024-09-19T05:45:26.556173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.453343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"89'10","reported_seq":185,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687787+0000","last_change":"2024-09-19T05:42:31.985783+0000","last_active":"2024-09-19T05:45:21.687787+0000","last_peered":"2024-09-19T05:45:21.687787+0000","last_clean":"2024-09-19T05:45:21.687787+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:21.687787+0000","last_undegraded":"2024-09-19T05:45:21.687787+0000","last_fullsized":"2024-09-19T05:45:21.687787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:51.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'168","reported_seq":339,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.557209+0000","last_change":"2024-09-19T05:42:32.939566+0000","last_active":"2024-09-19T05:45:26.557209+0000","last_peered":"2024-09-19T05:45:26.557209+0000","last_clean":"2024-09-19T05:45:26.557209+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:26.557209+0000","last_undegraded":"2024-09-19T05:45:26.557209+0000","last_fullsized":"2024-09-19T05:45:26.557209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:43:04.474569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"94'2017","reported_seq":2191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.935286+0000","last_change":"2024-09-19T05:42:31.988464+0000","last_active":"2024-09-19T05:45:21.935286+0000","last_peered":"2024-09-19T05:45:21.935286+0000","last_clean":"2024-09-19T05:45:21.935286+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:21.935286+0000","last_undegraded":"2024-09-19T05:45:21.935286+0000","last_fullsized":"2024-09-19T05:45:21.935286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2017,"log_dups_size":0,"ondisk_log_size":2017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:47:00.457496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"102'176","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.560758+0000","last_change":"2024-09-19T05:42:32.933454+0000","last_active":"2024-09-19T05:45:26.560758+0000","last_peered":"2024-09-19T05:45:26.560758+0000","last_clean":"2024-09-19T05:45:26.560758+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:26.560758+0000","last_undegraded":"2024-09-19T05:45:26.560758+0000","last_fullsized":"2024-09-19T05:45:26.560758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":177,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687838+0000","last_change":"2024-09-19T05:42:31.989517+0000","last_active":"2024-09-19T05:45:21.687838+0000","last_peered":"2024-09-19T05:45:21.687838+0000","last_clean":"2024-09-19T05:45:21.687838+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:21.687838+0000","last_undegraded":"2024-09-19T05:45:21.687838+0000","last_fullsized":"2024-09-19T05:45:21.687838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"102'177","reported_seq":349,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.546253+0000","last_change":"2024-09-19T05:42:32.942299+0000","last_active":"2024-09-19T05:45:26.546253+0000","last_peered":"2024-09-19T05:45:26.546253+0000","last_clean":"2024-09-19T05:45:26.546253+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:26.546253+0000","last_undegraded":"2024-09-19T05:45:26.546253+0000","last_fullsized":"2024-09-19T05:45:26.546253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:04.147657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716531+0000","last_change":"2024-09-19T05:42:31.987502+0000","last_active":"2024-09-19T05:45:21.716531+0000","last_peered":"2024-09-19T05:45:21.716531+0000","last_clean":"2024-09-19T05:45:21.716531+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:21.716531+0000","last_undegraded":"2024-09-19T05:45:21.716531+0000","last_fullsized":"2024-09-19T05:45:21.716531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:03.145343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"102'201","reported_seq":372,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.566194+0000","last_change":"2024-09-19T05:42:32.933486+0000","last_active":"2024-09-19T05:45:26.566194+0000","last_peered":"2024-09-19T05:45:26.566194+0000","last_clean":"2024-09-19T05:45:26.566194+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:26.566194+0000","last_undegraded":"2024-09-19T05:45:26.566194+0000","last_fullsized":"2024-09-19T05:45:26.566194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:35.471702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":175,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687906+0000","last_change":"2024-09-19T05:42:31.996162+0000","last_active":"2024-09-19T05:45:21.687906+0000","last_peered":"2024-09-19T05:45:21.687906+0000","last_clean":"2024-09-19T05:45:21.687906+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:21.687906+0000","last_undegraded":"2024-09-19T05:45:21.687906+0000","last_fullsized":"2024-09-19T05:45:21.687906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"102'197","reported_seq":289,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.541965+0000","last_change":"2024-09-19T05:45:10.627674+0000","last_active":"2024-09-19T05:45:26.541965+0000","last_peered":"2024-09-19T05:45:26.541965+0000","last_clean":"2024-09-19T05:45:26.541965+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:26.541965+0000","last_undegraded":"2024-09-19T05:45:26.541965+0000","last_fullsized":"2024-09-19T05:45:26.541965+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:21.685895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"89'9","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.708963+0000","last_change":"2024-09-19T05:42:31.984445+0000","last_active":"2024-09-19T05:45:21.708963+0000","last_peered":"2024-09-19T05:45:21.708963+0000","last_clean":"2024-09-19T05:45:21.708963+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:21.708963+0000","last_undegraded":"2024-09-19T05:45:21.708963+0000","last_fullsized":"2024-09-19T05:45:21.708963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:42.688989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"102'215","reported_seq":386,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.554457+0000","last_change":"2024-09-19T05:42:32.945732+0000","last_active":"2024-09-19T05:45:26.554457+0000","last_peered":"2024-09-19T05:45:26.554457+0000","last_clean":"2024-09-19T05:45:26.554457+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:26.554457+0000","last_undegraded":"2024-09-19T05:45:26.554457+0000","last_fullsized":"2024-09-19T05:45:26.554457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:51.344868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688832+0000","last_change":"2024-09-19T05:44:53.065559+0000","last_active":"2024-09-19T05:45:21.688832+0000","last_peered":"2024-09-19T05:45:21.688832+0000","last_clean":"2024-09-19T05:45:21.688832+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:21.688832+0000","last_undegraded":"2024-09-19T05:45:21.688832+0000","last_fullsized":"2024-09-19T05:45:21.688832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"102'175","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.531178+0000","last_change":"2024-09-19T05:42:32.931566+0000","last_active":"2024-09-19T05:45:26.531178+0000","last_peered":"2024-09-19T05:45:26.531178+0000","last_clean":"2024-09-19T05:45:26.531178+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:26.531178+0000","last_undegraded":"2024-09-19T05:45:26.531178+0000","last_fullsized":"2024-09-19T05:45:26.531178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:17.815664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709231+0000","last_change":"2024-09-19T05:44:53.059769+0000","last_active":"2024-09-19T05:45:21.709231+0000","last_peered":"2024-09-19T05:45:21.709231+0000","last_clean":"2024-09-19T05:45:21.709231+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:21.709231+0000","last_undegraded":"2024-09-19T05:45:21.709231+0000","last_fullsized":"2024-09-19T05:45:21.709231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:03.909805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"102'162","reported_seq":240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.543556+0000","last_change":"2024-09-19T05:44:15.787415+0000","last_active":"2024-09-19T05:45:26.543556+0000","last_peered":"2024-09-19T05:45:26.543556+0000","last_clean":"2024-09-19T05:45:26.543556+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:26.543556+0000","last_undegraded":"2024-09-19T05:45:26.543556+0000","last_fullsized":"2024-09-19T05:45:26.543556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:10.095369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":125,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688186+0000","last_change":"2024-09-19T05:44:53.058979+0000","last_active":"2024-09-19T05:45:21.688186+0000","last_peered":"2024-09-19T05:45:21.688186+0000","last_clean":"2024-09-19T05:45:21.688186+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:21.688186+0000","last_undegraded":"2024-09-19T05:45:21.688186+0000","last_fullsized":"2024-09-19T05:45:21.688186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:30:04.530725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"102'165","reported_seq":338,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.566926+0000","last_change":"2024-09-19T05:42:32.934210+0000","last_active":"2024-09-19T05:45:26.566926+0000","last_peered":"2024-09-19T05:45:26.566926+0000","last_clean":"2024-09-19T05:45:26.566926+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:26.566926+0000","last_undegraded":"2024-09-19T05:45:26.566926+0000","last_fullsized":"2024-09-19T05:45:26.566926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773397+0000","last_change":"2024-09-19T05:42:31.994744+0000","last_active":"2024-09-19T05:45:21.773397+0000","last_peered":"2024-09-19T05:45:21.773397+0000","last_clean":"2024-09-19T05:45:21.773397+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:21.773397+0000","last_undegraded":"2024-09-19T05:45:21.773397+0000","last_fullsized":"2024-09-19T05:45:21.773397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:15.089797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"102'175","reported_seq":356,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.555622+0000","last_change":"2024-09-19T05:43:19.548826+0000","last_active":"2024-09-19T05:45:26.555622+0000","last_peered":"2024-09-19T05:45:26.555622+0000","last_clean":"2024-09-19T05:45:26.555622+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:26.555622+0000","last_undegraded":"2024-09-19T05:45:26.555622+0000","last_fullsized":"2024-09-19T05:45:26.555622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:40.066235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":261,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:22.189880+0000","last_change":"2024-09-19T05:45:10.627237+0000","last_active":"2024-09-19T05:45:22.189880+0000","last_peered":"2024-09-19T05:45:22.189880+0000","last_clean":"2024-09-19T05:45:22.189880+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:22.189880+0000","last_undegraded":"2024-09-19T05:45:22.189880+0000","last_fullsized":"2024-09-19T05:45:22.189880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:21.864335+0000","last_clean_scrub_stamp":"2024-09-19T05:42:21.864335+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:04.886546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716440+0000","last_change":"2024-09-19T05:42:31.994252+0000","last_active":"2024-09-19T05:45:21.716440+0000","last_peered":"2024-09-19T05:45:21.716440+0000","last_clean":"2024-09-19T05:45:21.716440+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:21.716440+0000","last_undegraded":"2024-09-19T05:45:21.716440+0000","last_fullsized":"2024-09-19T05:45:21.716440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:00.367827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"102'183","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.545372+0000","last_change":"2024-09-19T05:42:32.930821+0000","last_active":"2024-09-19T05:45:26.545372+0000","last_peered":"2024-09-19T05:45:26.545372+0000","last_clean":"2024-09-19T05:45:26.545372+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:26.545372+0000","last_undegraded":"2024-09-19T05:45:26.545372+0000","last_fullsized":"2024-09-19T05:45:26.545372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:05.482402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716388+0000","last_change":"2024-09-19T05:44:53.069283+0000","last_active":"2024-09-19T05:45:21.716388+0000","last_peered":"2024-09-19T05:45:21.716388+0000","last_clean":"2024-09-19T05:45:21.716388+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:21.716388+0000","last_undegraded":"2024-09-19T05:45:21.716388+0000","last_fullsized":"2024-09-19T05:45:21.716388+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:51.209720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"102'165","reported_seq":337,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.545952+0000","last_change":"2024-09-19T05:42:32.935107+0000","last_active":"2024-09-19T05:45:26.545952+0000","last_peered":"2024-09-19T05:45:26.545952+0000","last_clean":"2024-09-19T05:45:26.545952+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:26.545952+0000","last_undegraded":"2024-09-19T05:45:26.545952+0000","last_fullsized":"2024-09-19T05:45:26.545952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:46.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687539+0000","last_change":"2024-09-19T05:44:53.068415+0000","last_active":"2024-09-19T05:45:21.687539+0000","last_peered":"2024-09-19T05:45:21.687539+0000","last_clean":"2024-09-19T05:45:21.687539+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:21.687539+0000","last_undegraded":"2024-09-19T05:45:21.687539+0000","last_fullsized":"2024-09-19T05:45:21.687539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:16.609871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"102'165","reported_seq":318,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.548144+0000","last_change":"2024-09-19T05:44:58.820818+0000","last_active":"2024-09-19T05:45:26.548144+0000","last_peered":"2024-09-19T05:45:26.548144+0000","last_clean":"2024-09-19T05:45:26.548144+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:26.548144+0000","last_undegraded":"2024-09-19T05:45:26.548144+0000","last_fullsized":"2024-09-19T05:45:26.548144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:32.427214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716567+0000","last_change":"2024-09-19T05:42:31.997462+0000","last_active":"2024-09-19T05:45:21.716567+0000","last_peered":"2024-09-19T05:45:21.716567+0000","last_clean":"2024-09-19T05:45:21.716567+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:21.716567+0000","last_undegraded":"2024-09-19T05:45:21.716567+0000","last_fullsized":"2024-09-19T05:45:21.716567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:22.798228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"102'177","reported_seq":350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.566653+0000","last_change":"2024-09-19T05:42:32.935391+0000","last_active":"2024-09-19T05:45:26.566653+0000","last_peered":"2024-09-19T05:45:26.566653+0000","last_clean":"2024-09-19T05:45:26.566653+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:26.566653+0000","last_undegraded":"2024-09-19T05:45:26.566653+0000","last_fullsized":"2024-09-19T05:45:26.566653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.687970+0000","last_change":"2024-09-19T05:44:53.063483+0000","last_active":"2024-09-19T05:45:21.687970+0000","last_peered":"2024-09-19T05:45:21.687970+0000","last_clean":"2024-09-19T05:45:21.687970+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:21.687970+0000","last_undegraded":"2024-09-19T05:45:21.687970+0000","last_fullsized":"2024-09-19T05:45:21.687970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:40.306784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"102'160","reported_seq":332,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.540311+0000","last_change":"2024-09-19T05:42:32.935495+0000","last_active":"2024-09-19T05:45:26.540311+0000","last_peered":"2024-09-19T05:45:26.540311+0000","last_clean":"2024-09-19T05:45:26.540311+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:26.540311+0000","last_undegraded":"2024-09-19T05:45:26.540311+0000","last_fullsized":"2024-09-19T05:45:26.540311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:48.584577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":173,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716905+0000","last_change":"2024-09-19T05:42:31.995631+0000","last_active":"2024-09-19T05:45:21.716905+0000","last_peered":"2024-09-19T05:45:21.716905+0000","last_clean":"2024-09-19T05:45:21.716905+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:21.716905+0000","last_undegraded":"2024-09-19T05:45:21.716905+0000","last_fullsized":"2024-09-19T05:45:21.716905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:45.985924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"102'162","reported_seq":433,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.557181+0000","last_change":"2024-09-19T05:44:55.675882+0000","last_active":"2024-09-19T05:45:26.557181+0000","last_peered":"2024-09-19T05:45:26.557181+0000","last_clean":"2024-09-19T05:45:26.557181+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:26.557181+0000","last_undegraded":"2024-09-19T05:45:26.557181+0000","last_fullsized":"2024-09-19T05:45:26.557181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:04.593886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"102'18","reported_seq":192,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709363+0000","last_change":"2024-09-19T05:42:31.992843+0000","last_active":"2024-09-19T05:45:21.709363+0000","last_peered":"2024-09-19T05:45:21.709363+0000","last_clean":"2024-09-19T05:45:21.709363+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:21.709363+0000","last_undegraded":"2024-09-19T05:45:21.709363+0000","last_fullsized":"2024-09-19T05:45:21.709363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"102'166","reported_seq":338,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.544768+0000","last_change":"2024-09-19T05:42:32.935640+0000","last_active":"2024-09-19T05:45:26.544768+0000","last_peered":"2024-09-19T05:45:26.544768+0000","last_clean":"2024-09-19T05:45:26.544768+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:26.544768+0000","last_undegraded":"2024-09-19T05:45:26.544768+0000","last_fullsized":"2024-09-19T05:45:26.544768+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:04.083659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"85'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716921+0000","last_change":"2024-09-19T05:42:31.994426+0000","last_active":"2024-09-19T05:45:21.716921+0000","last_peered":"2024-09-19T05:45:21.716921+0000","last_clean":"2024-09-19T05:45:21.716921+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:21.716921+0000","last_undegraded":"2024-09-19T05:45:21.716921+0000","last_fullsized":"2024-09-19T05:45:21.716921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:08.156140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"102'183","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.545080+0000","last_change":"2024-09-19T05:42:32.931330+0000","last_active":"2024-09-19T05:45:26.545080+0000","last_peered":"2024-09-19T05:45:26.545080+0000","last_clean":"2024-09-19T05:45:26.545080+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:26.545080+0000","last_undegraded":"2024-09-19T05:45:26.545080+0000","last_fullsized":"2024-09-19T05:45:26.545080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:55.383415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:29.781412+0000","last_change":"2024-09-19T05:45:29.781446+0000","last_active":"2024-09-19T05:45:29.781412+0000","last_peered":"2024-09-19T05:45:29.781412+0000","last_clean":"2024-09-19T05:45:29.781412+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:29.781412+0000","last_undegraded":"2024-09-19T05:45:29.781412+0000","last_fullsized":"2024-09-19T05:45:29.781412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:29.781333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:29.781333+0000","last_clean_scrub_stamp":"2024-09-19T05:45:29.781333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:06.401746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"102'162","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.521290+0000","last_change":"2024-09-19T05:42:32.941923+0000","last_active":"2024-09-19T05:45:26.521290+0000","last_peered":"2024-09-19T05:45:26.521290+0000","last_clean":"2024-09-19T05:45:26.521290+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:26.521290+0000","last_undegraded":"2024-09-19T05:45:26.521290+0000","last_fullsized":"2024-09-19T05:45:26.521290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:07.808500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":174,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709021+0000","last_change":"2024-09-19T05:42:31.992172+0000","last_active":"2024-09-19T05:45:21.709021+0000","last_peered":"2024-09-19T05:45:21.709021+0000","last_clean":"2024-09-19T05:45:21.709021+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:21.709021+0000","last_undegraded":"2024-09-19T05:45:21.709021+0000","last_fullsized":"2024-09-19T05:45:21.709021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:26.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"102'173","reported_seq":345,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.544918+0000","last_change":"2024-09-19T05:42:32.924698+0000","last_active":"2024-09-19T05:45:26.544918+0000","last_peered":"2024-09-19T05:45:26.544918+0000","last_clean":"2024-09-19T05:45:26.544918+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:26.544918+0000","last_undegraded":"2024-09-19T05:45:26.544918+0000","last_fullsized":"2024-09-19T05:45:26.544918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:57:35.437187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.773238+0000","last_change":"2024-09-19T05:44:53.065594+0000","last_active":"2024-09-19T05:45:21.773238+0000","last_peered":"2024-09-19T05:45:21.773238+0000","last_clean":"2024-09-19T05:45:21.773238+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:21.773238+0000","last_undegraded":"2024-09-19T05:45:21.773238+0000","last_fullsized":"2024-09-19T05:45:21.773238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:54:47.254673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"102'164","reported_seq":337,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.536035+0000","last_change":"2024-09-19T05:42:32.933121+0000","last_active":"2024-09-19T05:45:26.536035+0000","last_peered":"2024-09-19T05:45:26.536035+0000","last_clean":"2024-09-19T05:45:26.536035+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:26.536035+0000","last_undegraded":"2024-09-19T05:45:26.536035+0000","last_fullsized":"2024-09-19T05:45:26.536035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:09.739107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"59'3","reported_seq":176,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.716236+0000","last_change":"2024-09-19T05:42:31.994828+0000","last_active":"2024-09-19T05:45:21.716236+0000","last_peered":"2024-09-19T05:45:21.716236+0000","last_clean":"2024-09-19T05:45:21.716236+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:21.716236+0000","last_undegraded":"2024-09-19T05:45:21.716236+0000","last_fullsized":"2024-09-19T05:45:21.716236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:35.015685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"102'154","reported_seq":335,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.550081+0000","last_change":"2024-09-19T05:43:19.549366+0000","last_active":"2024-09-19T05:45:26.550081+0000","last_peered":"2024-09-19T05:45:26.550081+0000","last_clean":"2024-09-19T05:45:26.550081+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:26.550081+0000","last_undegraded":"2024-09-19T05:45:26.550081+0000","last_fullsized":"2024-09-19T05:45:26.550081+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:37:13.490980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"102'205","reported_seq":376,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.565938+0000","last_change":"2024-09-19T05:42:32.948084+0000","last_active":"2024-09-19T05:45:26.565938+0000","last_peered":"2024-09-19T05:45:26.565938+0000","last_clean":"2024-09-19T05:45:26.565938+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:26.565938+0000","last_undegraded":"2024-09-19T05:45:26.565938+0000","last_fullsized":"2024-09-19T05:45:26.565938+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:30.675290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":195,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688956+0000","last_change":"2024-09-19T05:43:19.560474+0000","last_active":"2024-09-19T05:45:21.688956+0000","last_peered":"2024-09-19T05:45:21.688956+0000","last_clean":"2024-09-19T05:45:21.688956+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:21.688956+0000","last_undegraded":"2024-09-19T05:45:21.688956+0000","last_fullsized":"2024-09-19T05:45:21.688956+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"102'162","reported_seq":275,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.561964+0000","last_change":"2024-09-19T05:45:10.627058+0000","last_active":"2024-09-19T05:45:26.561964+0000","last_peered":"2024-09-19T05:45:26.561964+0000","last_clean":"2024-09-19T05:45:26.561964+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:26.561964+0000","last_undegraded":"2024-09-19T05:45:26.561964+0000","last_fullsized":"2024-09-19T05:45:26.561964+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:52.373727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"70'4","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688343+0000","last_change":"2024-09-19T05:43:19.558521+0000","last_active":"2024-09-19T05:45:21.688343+0000","last_peered":"2024-09-19T05:45:21.688343+0000","last_clean":"2024-09-19T05:45:21.688343+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:21.688343+0000","last_undegraded":"2024-09-19T05:45:21.688343+0000","last_fullsized":"2024-09-19T05:45:21.688343+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"102'160","reported_seq":320,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.554648+0000","last_change":"2024-09-19T05:44:57.957776+0000","last_active":"2024-09-19T05:45:26.554648+0000","last_peered":"2024-09-19T05:45:26.554648+0000","last_clean":"2024-09-19T05:45:26.554648+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:26.554648+0000","last_undegraded":"2024-09-19T05:45:26.554648+0000","last_fullsized":"2024-09-19T05:45:26.554648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:37.587819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":156,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.688211+0000","last_change":"2024-09-19T05:44:53.061701+0000","last_active":"2024-09-19T05:45:21.688211+0000","last_peered":"2024-09-19T05:45:21.688211+0000","last_clean":"2024-09-19T05:45:21.688211+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:21.688211+0000","last_undegraded":"2024-09-19T05:45:21.688211+0000","last_fullsized":"2024-09-19T05:45:21.688211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:26.709808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"102'181","reported_seq":326,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.556230+0000","last_change":"2024-09-19T05:44:57.503018+0000","last_active":"2024-09-19T05:45:26.556230+0000","last_peered":"2024-09-19T05:45:26.556230+0000","last_clean":"2024-09-19T05:45:26.556230+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:26.556230+0000","last_undegraded":"2024-09-19T05:45:26.556230+0000","last_fullsized":"2024-09-19T05:45:26.556230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:44.482643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:21.709387+0000","last_change":"2024-09-19T05:42:31.995450+0000","last_active":"2024-09-19T05:45:21.709387+0000","last_peered":"2024-09-19T05:45:21.709387+0000","last_clean":"2024-09-19T05:45:21.709387+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:21.709387+0000","last_undegraded":"2024-09-19T05:45:21.709387+0000","last_fullsized":"2024-09-19T05:45:21.709387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:30.962942+0000","last_clean_scrub_stamp":"2024-09-19T05:42:30.962942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:38:08.736793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":11255,"ondisk_log_size":11255,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970622,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3910,"num_write_kb":11502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15941244,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3896,"ondisk_log_size":3896,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54516,"kb_used_data":5284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317324,"statfs":{"total":96636764160,"available":96580939776,"internally_reserved":0,"allocated":5410816,"data_stored":4755262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542185,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":4984832,"data_stored":4334629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449098,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":1736,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":1777664,"data_stored":4925594,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50364,"kb_used_data":5292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321476,"statfs":{"total":96636764160,"available":96585191424,"internally_reserved":0,"allocated":5419008,"data_stored":4755934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542185,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1212416,"data_stored":560868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40736,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331104,"statfs":{"total":96636764160,"available":96595050496,"internally_reserved":0,"allocated":1261568,"data_stored":1150724,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542186,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40692,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331148,"statfs":{"total":96636764160,"available":96595095552,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774940,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:45:33.064 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.064 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.064 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.064 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.064 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.064 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.064 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.065 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.066 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.067 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.068 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.069 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.070 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-19T05:42:21.864335+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=21, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.071 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.072 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.073 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.074 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.074 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.074 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.074 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.074 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-19T05:42:30.962942+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=30, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:33.074 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T05:45:33.973 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:45:33.976 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:45:53.075 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:45:53.191 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.189+0000 7ff4ee591640 1 Processor -- start 2024-09-19T05:45:53.192 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.189+0000 7ff4ee591640 1 -- start start 2024-09-19T05:45:53.193 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.190+0000 7ff4ee591640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff4e80d06e0 0x7ff4e80cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.190+0000 7ff4ee591640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e80cb400 con 0x7ff4e815eb50 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.190+0000 7ff4ee591640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e80cb6e0 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.190+0000 7ff4ee591640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e80cc970 con 0x7ff4e80d06e0 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.190+0000 7ff4e7fff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff4e80d06e0 0x7ff4e80cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.190+0000 7ff4ecb2e640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff4e815eb50 0x7ff4e80cd450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:36536/0 (socket says 172.21.15.2:36536) 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.190+0000 7ff4ecb2e640 1 -- 172.21.15.2:0/154906487 learned_addr learned my addr 172.21.15.2:0/154906487 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.191+0000 7ff4e7fff640 1 -- 172.21.15.2:0/154906487 >> v1:172.21.15.123:6789/0 conn(0x7ff4e80d0ac0 legacy=0x7ff4e8154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.191+0000 7ff4e6ffd640 1 -- 172.21.15.2:0/154906487 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420708330 0 0) 0x7ff4e80cb400 con 0x7ff4e815eb50 2024-09-19T05:45:53.194 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.191+0000 7ff4e7fff640 1 -- 172.21.15.2:0/154906487 >> v1:172.21.15.2:6789/0 conn(0x7ff4e815eb50 legacy=0x7ff4e80cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:53.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.191+0000 7ff4e7fff640 1 -- 172.21.15.2:0/154906487 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e80ccc50 con 0x7ff4e80d06e0 2024-09-19T05:45:53.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.191+0000 7ff4e7fff640 1 --2- 172.21.15.2:0/154906487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d06e0 0x7ff4e80cbb40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80097a0 tx=0x7ff4d802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38651bc50a709317 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:53.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4e6ffd640 1 -- 172.21.15.2:0/154906487 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037070 con 0x7ff4e80d06e0 2024-09-19T05:45:53.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4e6ffd640 1 -- 172.21.15.2:0/154906487 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8032440 con 0x7ff4e80d06e0 2024-09-19T05:45:53.195 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4e6ffd640 1 -- 172.21.15.2:0/154906487 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8032810 con 0x7ff4e80d06e0 2024-09-19T05:45:53.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4ee591640 1 -- 172.21.15.2:0/154906487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d06e0 msgr2=0x7ff4e80cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:53.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4ee591640 1 --2- 172.21.15.2:0/154906487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d06e0 0x7ff4e80cbb40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80097a0 tx=0x7ff4d802a230 comp rx=0 tx=0).stop 2024-09-19T05:45:53.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4ee591640 1 -- 172.21.15.2:0/154906487 shutdown_connections 2024-09-19T05:45:53.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4ee591640 1 --2- 172.21.15.2:0/154906487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d06e0 0x7ff4e80cbb40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4ee591640 1 -- 172.21.15.2:0/154906487 >> 172.21.15.2:0/154906487 conn(0x7ff4e806d740 msgr2=0x7ff4e8073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:53.198 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4ee591640 1 -- 172.21.15.2:0/154906487 shutdown_connections 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.192+0000 7ff4ee591640 1 -- 172.21.15.2:0/154906487 wait complete. 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.193+0000 7ff4ee591640 1 Processor -- start 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4ee591640 1 -- start start 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4ee591640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e80d06e0 0x7ff4e81f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4ee591640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d0ac0 0x7ff4e81565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4ee591640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4e815eb50 0x7ff4e8159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4ee591640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e80c5830 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4ee591640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e815efc0 con 0x7ff4e80d06e0 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4ee591640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e8155200 con 0x7ff4e815eb50 2024-09-19T05:45:53.199 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d0ac0 0x7ff4e81565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4e77fe640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d0ac0 0x7ff4e81565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:50506/0 (socket says 172.21.15.2:50506) 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4e77fe640 1 -- 172.21.15.2:0/2598970663 learned_addr learned my addr 172.21.15.2:0/2598970663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.194+0000 7ff4e7fff640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e80d06e0 0x7ff4e81f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4ecb2e640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4e815eb50 0x7ff4e8159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e77fe640 1 -- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4e815eb50 msgr2=0x7ff4e8159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e77fe640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4e815eb50 0x7ff4e8159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e77fe640 1 -- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e80d06e0 msgr2=0x7ff4e81f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e77fe640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e80d06e0 0x7ff4e81f0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e77fe640 1 -- 172.21.15.2:0/2598970663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e8158a00 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4ecb2e640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4e815eb50 0x7ff4e8159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e77fe640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d0ac0 0x7ff4e81565e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c430 tx=0x7ff4dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:53.200 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e4ff9640 1 -- 172.21.15.2:0/2598970663 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc017070 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e4ff9640 1 -- 172.21.15.2:0/2598970663 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4dc0052d0 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e80c8400 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4e4ff9640 1 -- 172.21.15.2:0/2598970663 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc003830 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.201 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.195+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e81f0730 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.196+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e81f0930 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.197+0000 7ff4e4ff9640 1 -- 172.21.15.2:0/2598970663 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff4dc01a070 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.197+0000 7ff4e4ff9640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff4d406c2b0 0x7ff4d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:45:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.197+0000 7ff4e4ff9640 1 -- 172.21.15.2:0/2598970663 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7ff4dc08a730 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.197+0000 7ff4e7fff640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff4d406c2b0 0x7ff4d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:45:53.202 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.198+0000 7ff4e7fff640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff4d406c2b0 0x7ff4d406e770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8034000 tx=0x7ff4d802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:45:53.204 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.202+0000 7ff4e4ff9640 1 -- 172.21.15.2:0/2598970663 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7ff4e81f0930 con 0x7ff4e80d0ac0 2024-09-19T05:45:53.376 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.374+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4e80c6eb0 con 0x7ff4d406c2b0 2024-09-19T05:45:53.382 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.381+0000 7ff4e4ff9640 1 -- 172.21.15.2:0/2598970663 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254279 (secure 0 0 0) 0x7ff4e80c6eb0 con 0x7ff4d406c2b0 2024-09-19T05:45:53.386 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:45:53.388 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:45:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff4d406c2b0 msgr2=0x7ff4d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff4d406c2b0 0x7ff4d406e770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8034000 tx=0x7ff4d802e000 comp rx=0 tx=0).stop 2024-09-19T05:45:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d0ac0 msgr2=0x7ff4e81565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:45:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d0ac0 0x7ff4e81565e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00c430 tx=0x7ff4dc00c8f0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 shutdown_connections 2024-09-19T05:45:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7ff4d406c2b0 0x7ff4d406e770 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.390 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4e815eb50 0x7ff4e8159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4e80d0ac0 0x7ff4e81565e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 --2- 172.21.15.2:0/2598970663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4e80d06e0 0x7ff4e81f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:45:53.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.383+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 >> 172.21.15.2:0/2598970663 conn(0x7ff4e806d740 msgr2=0x7ff4e8153780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:45:53.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.384+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 shutdown_connections 2024-09-19T05:45:53.391 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:45:53.384+0000 7ff4ee591640 1 -- 172.21.15.2:0/2598970663 wait complete. 2024-09-19T05:45:53.401 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-19T05:45:51.862065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560990,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13285,"num_write_kb":25179742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15290,"ondisk_log_size":15290,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342232,"kb_used_data":21976,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632488,"statfs":{"total":773094113280,"available":772743667712,"internally_reserved":0,"allocated":22503424,"data_stored":21604268,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002965"},"pg_stats":[{"pgid":"3.3f","version":"102'167","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.917070+0000","last_change":"2024-09-19T05:45:37.917099+0000","last_active":"2024-09-19T05:45:37.917070+0000","last_peered":"2024-09-19T05:45:37.917070+0000","last_clean":"2024-09-19T05:45:37.917070+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:37.917070+0000","last_undegraded":"2024-09-19T05:45:37.917070+0000","last_fullsized":"2024-09-19T05:45:37.917070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'167","last_scrub_stamp":"2024-09-19T05:45:37.916981+0000","last_deep_scrub":"102'167","last_deep_scrub_stamp":"2024-09-19T05:45:37.916981+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.916981+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:44.030022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"102'152","reported_seq":314,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.515982+0000","last_change":"2024-09-19T05:45:38.516031+0000","last_active":"2024-09-19T05:45:38.515982+0000","last_peered":"2024-09-19T05:45:38.515982+0000","last_clean":"2024-09-19T05:45:38.515982+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:38.515982+0000","last_undegraded":"2024-09-19T05:45:38.515982+0000","last_fullsized":"2024-09-19T05:45:38.515982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'152","last_scrub_stamp":"2024-09-19T05:45:38.515863+0000","last_deep_scrub":"102'152","last_deep_scrub_stamp":"2024-09-19T05:45:38.515863+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.515863+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"102'178","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.549899+0000","last_change":"2024-09-19T05:45:36.549934+0000","last_active":"2024-09-19T05:45:36.549899+0000","last_peered":"2024-09-19T05:45:36.549899+0000","last_clean":"2024-09-19T05:45:36.549899+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:36.549899+0000","last_undegraded":"2024-09-19T05:45:36.549899+0000","last_fullsized":"2024-09-19T05:45:36.549899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'178","last_scrub_stamp":"2024-09-19T05:45:36.549818+0000","last_deep_scrub":"102'178","last_deep_scrub_stamp":"2024-09-19T05:45:36.549818+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.549818+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:59.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"102'193","reported_seq":374,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.100279+0000","last_change":"2024-09-19T05:45:36.100312+0000","last_active":"2024-09-19T05:45:36.100279+0000","last_peered":"2024-09-19T05:45:36.100279+0000","last_clean":"2024-09-19T05:45:36.100279+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:36.100279+0000","last_undegraded":"2024-09-19T05:45:36.100279+0000","last_fullsized":"2024-09-19T05:45:36.100279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'193","last_scrub_stamp":"2024-09-19T05:45:36.100183+0000","last_deep_scrub":"102'193","last_deep_scrub_stamp":"2024-09-19T05:45:36.100183+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.100183+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:17:22.471667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"102'174","reported_seq":451,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:44.889221+0000","last_change":"2024-09-19T05:45:44.889249+0000","last_active":"2024-09-19T05:45:44.889221+0000","last_peered":"2024-09-19T05:45:44.889221+0000","last_clean":"2024-09-19T05:45:44.889221+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:44.889221+0000","last_undegraded":"2024-09-19T05:45:44.889221+0000","last_fullsized":"2024-09-19T05:45:44.889221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'174","last_scrub_stamp":"2024-09-19T05:45:44.889134+0000","last_deep_scrub":"102'174","last_deep_scrub_stamp":"2024-09-19T05:45:44.889134+0000","last_clean_scrub_stamp":"2024-09-19T05:45:44.889134+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:40.958698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"102'162","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.057061+0000","last_change":"2024-09-19T05:45:35.057110+0000","last_active":"2024-09-19T05:45:35.057061+0000","last_peered":"2024-09-19T05:45:35.057061+0000","last_clean":"2024-09-19T05:45:35.057061+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:35.057061+0000","last_undegraded":"2024-09-19T05:45:35.057061+0000","last_fullsized":"2024-09-19T05:45:35.057061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:35.056955+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:35.056955+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.056955+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:28.992568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"102'188","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.909755+0000","last_change":"2024-09-19T05:45:35.909784+0000","last_active":"2024-09-19T05:45:35.909755+0000","last_peered":"2024-09-19T05:45:35.909755+0000","last_clean":"2024-09-19T05:45:35.909755+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:35.909755+0000","last_undegraded":"2024-09-19T05:45:35.909755+0000","last_fullsized":"2024-09-19T05:45:35.909755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'188","last_scrub_stamp":"2024-09-19T05:45:35.909668+0000","last_deep_scrub":"102'188","last_deep_scrub_stamp":"2024-09-19T05:45:35.909668+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.909668+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:50:39.017237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"102'204","reported_seq":375,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.574474+0000","last_change":"2024-09-19T05:45:42.574500+0000","last_active":"2024-09-19T05:45:42.574474+0000","last_peered":"2024-09-19T05:45:42.574474+0000","last_clean":"2024-09-19T05:45:42.574474+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:42.574474+0000","last_undegraded":"2024-09-19T05:45:42.574474+0000","last_fullsized":"2024-09-19T05:45:42.574474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'204","last_scrub_stamp":"2024-09-19T05:45:42.574395+0000","last_deep_scrub":"102'204","last_deep_scrub_stamp":"2024-09-19T05:45:42.574395+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.574395+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:00.568585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"102'178","reported_seq":301,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.006939+0000","last_change":"2024-09-19T05:45:36.006939+0000","last_active":"2024-09-19T05:45:36.006939+0000","last_peered":"2024-09-19T05:45:36.006939+0000","last_clean":"2024-09-19T05:45:36.006939+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:36.006939+0000","last_undegraded":"2024-09-19T05:45:36.006939+0000","last_fullsized":"2024-09-19T05:45:36.006939+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'178","last_scrub_stamp":"2024-09-19T05:45:36.006832+0000","last_deep_scrub":"102'178","last_deep_scrub_stamp":"2024-09-19T05:45:36.006832+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.006832+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:04.915216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"102'190","reported_seq":371,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.853419+0000","last_change":"2024-09-19T05:45:33.853445+0000","last_active":"2024-09-19T05:45:33.853419+0000","last_peered":"2024-09-19T05:45:33.853419+0000","last_clean":"2024-09-19T05:45:33.853419+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:33.853419+0000","last_undegraded":"2024-09-19T05:45:33.853419+0000","last_fullsized":"2024-09-19T05:45:33.853419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'190","last_scrub_stamp":"2024-09-19T05:45:33.853325+0000","last_deep_scrub":"102'190","last_deep_scrub_stamp":"2024-09-19T05:45:33.853325+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.853325+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:16.203364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"102'214","reported_seq":395,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.912842+0000","last_change":"2024-09-19T05:45:36.912868+0000","last_active":"2024-09-19T05:45:36.912842+0000","last_peered":"2024-09-19T05:45:36.912842+0000","last_clean":"2024-09-19T05:45:36.912842+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:36.912842+0000","last_undegraded":"2024-09-19T05:45:36.912842+0000","last_fullsized":"2024-09-19T05:45:36.912842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'214","last_scrub_stamp":"2024-09-19T05:45:36.912751+0000","last_deep_scrub":"102'214","last_deep_scrub_stamp":"2024-09-19T05:45:36.912751+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.912751+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:58.232479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"102'166","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.928695+0000","last_change":"2024-09-19T05:45:40.928748+0000","last_active":"2024-09-19T05:45:40.928695+0000","last_peered":"2024-09-19T05:45:40.928695+0000","last_clean":"2024-09-19T05:45:40.928695+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:40.928695+0000","last_undegraded":"2024-09-19T05:45:40.928695+0000","last_fullsized":"2024-09-19T05:45:40.928695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'166","last_scrub_stamp":"2024-09-19T05:45:40.928588+0000","last_deep_scrub":"102'166","last_deep_scrub_stamp":"2024-09-19T05:45:40.928588+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.928588+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:53.325578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"102'177","reported_seq":358,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.216910+0000","last_change":"2024-09-19T05:45:39.216947+0000","last_active":"2024-09-19T05:45:39.216910+0000","last_peered":"2024-09-19T05:45:39.216910+0000","last_clean":"2024-09-19T05:45:39.216910+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:39.216910+0000","last_undegraded":"2024-09-19T05:45:39.216910+0000","last_fullsized":"2024-09-19T05:45:39.216910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:39.216812+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:39.216812+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.216812+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:51.833446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"102'172","reported_seq":353,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.224580+0000","last_change":"2024-09-19T05:45:36.224630+0000","last_active":"2024-09-19T05:45:36.224580+0000","last_peered":"2024-09-19T05:45:36.224580+0000","last_clean":"2024-09-19T05:45:36.224580+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:36.224580+0000","last_undegraded":"2024-09-19T05:45:36.224580+0000","last_fullsized":"2024-09-19T05:45:36.224580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'172","last_scrub_stamp":"2024-09-19T05:45:36.224485+0000","last_deep_scrub":"102'172","last_deep_scrub_stamp":"2024-09-19T05:45:36.224485+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.224485+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:36.245176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"102'177","reported_seq":335,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.510306+0000","last_change":"2024-09-19T05:45:37.510332+0000","last_active":"2024-09-19T05:45:37.510306+0000","last_peered":"2024-09-19T05:45:37.510306+0000","last_clean":"2024-09-19T05:45:37.510306+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:37.510306+0000","last_undegraded":"2024-09-19T05:45:37.510306+0000","last_fullsized":"2024-09-19T05:45:37.510306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:37.510190+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:37.510190+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.510190+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:10.310827+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"102'162","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.799878+0000","last_change":"2024-09-19T05:45:33.799902+0000","last_active":"2024-09-19T05:45:33.799878+0000","last_peered":"2024-09-19T05:45:33.799878+0000","last_clean":"2024-09-19T05:45:33.799878+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:33.799878+0000","last_undegraded":"2024-09-19T05:45:33.799878+0000","last_fullsized":"2024-09-19T05:45:33.799878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:33.799773+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:33.799773+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.799773+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:33.090240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"102'176","reported_seq":356,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.816760+0000","last_change":"2024-09-19T05:45:32.816783+0000","last_active":"2024-09-19T05:45:32.816760+0000","last_peered":"2024-09-19T05:45:32.816760+0000","last_clean":"2024-09-19T05:45:32.816760+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:32.816760+0000","last_undegraded":"2024-09-19T05:45:32.816760+0000","last_fullsized":"2024-09-19T05:45:32.816760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'176","last_scrub_stamp":"2024-09-19T05:45:32.816697+0000","last_deep_scrub":"102'176","last_deep_scrub_stamp":"2024-09-19T05:45:32.816697+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.816697+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:21.013902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"102'179","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.267287+0000","last_change":"2024-09-19T05:45:36.267312+0000","last_active":"2024-09-19T05:45:36.267287+0000","last_peered":"2024-09-19T05:45:36.267287+0000","last_clean":"2024-09-19T05:45:36.267287+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:36.267287+0000","last_undegraded":"2024-09-19T05:45:36.267287+0000","last_fullsized":"2024-09-19T05:45:36.267287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'179","last_scrub_stamp":"2024-09-19T05:45:36.267187+0000","last_deep_scrub":"102'179","last_deep_scrub_stamp":"2024-09-19T05:45:36.267187+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.267187+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:04.121186+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"102'205","reported_seq":385,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.802247+0000","last_change":"2024-09-19T05:45:34.802286+0000","last_active":"2024-09-19T05:45:34.802247+0000","last_peered":"2024-09-19T05:45:34.802247+0000","last_clean":"2024-09-19T05:45:34.802247+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:34.802247+0000","last_undegraded":"2024-09-19T05:45:34.802247+0000","last_fullsized":"2024-09-19T05:45:34.802247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'205","last_scrub_stamp":"2024-09-19T05:45:34.802159+0000","last_deep_scrub":"102'205","last_deep_scrub_stamp":"2024-09-19T05:45:34.802159+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.802159+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:01.143189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"102'179","reported_seq":370,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.888232+0000","last_change":"2024-09-19T05:45:42.888261+0000","last_active":"2024-09-19T05:45:42.888232+0000","last_peered":"2024-09-19T05:45:42.888232+0000","last_clean":"2024-09-19T05:45:42.888232+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:42.888232+0000","last_undegraded":"2024-09-19T05:45:42.888232+0000","last_fullsized":"2024-09-19T05:45:42.888232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'179","last_scrub_stamp":"2024-09-19T05:45:42.888140+0000","last_deep_scrub":"102'179","last_deep_scrub_stamp":"2024-09-19T05:45:42.888140+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.888140+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:31.311294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"102'154","reported_seq":335,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.795730+0000","last_change":"2024-09-19T05:45:38.795758+0000","last_active":"2024-09-19T05:45:38.795730+0000","last_peered":"2024-09-19T05:45:38.795730+0000","last_clean":"2024-09-19T05:45:38.795730+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:38.795730+0000","last_undegraded":"2024-09-19T05:45:38.795730+0000","last_fullsized":"2024-09-19T05:45:38.795730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'154","last_scrub_stamp":"2024-09-19T05:45:38.795632+0000","last_deep_scrub":"102'154","last_deep_scrub_stamp":"2024-09-19T05:45:38.795632+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.795632+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:18.826189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"102'199","reported_seq":265,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.502634+0000","last_change":"2024-09-19T05:45:40.502660+0000","last_active":"2024-09-19T05:45:40.502634+0000","last_peered":"2024-09-19T05:45:40.502634+0000","last_clean":"2024-09-19T05:45:40.502634+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:40.502634+0000","last_undegraded":"2024-09-19T05:45:40.502634+0000","last_fullsized":"2024-09-19T05:45:40.502634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'199","last_scrub_stamp":"2024-09-19T05:45:40.502514+0000","last_deep_scrub":"102'199","last_deep_scrub_stamp":"2024-09-19T05:45:40.502514+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.502514+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:19.991273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"102'150","reported_seq":312,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.542756+0000","last_change":"2024-09-19T05:45:41.542781+0000","last_active":"2024-09-19T05:45:41.542756+0000","last_peered":"2024-09-19T05:45:41.542756+0000","last_clean":"2024-09-19T05:45:41.542756+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:41.542756+0000","last_undegraded":"2024-09-19T05:45:41.542756+0000","last_fullsized":"2024-09-19T05:45:41.542756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'150","last_scrub_stamp":"2024-09-19T05:45:41.542638+0000","last_deep_scrub":"102'150","last_deep_scrub_stamp":"2024-09-19T05:45:41.542638+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.542638+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:58.446555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"102'207","reported_seq":499,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.554234+0000","last_change":"2024-09-19T05:44:56.565770+0000","last_active":"2024-09-19T05:45:26.554234+0000","last_peered":"2024-09-19T05:45:26.554234+0000","last_clean":"2024-09-19T05:45:26.554234+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:26.554234+0000","last_undegraded":"2024-09-19T05:45:26.554234+0000","last_fullsized":"2024-09-19T05:45:26.554234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"102'160","reported_seq":330,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:43.611748+0000","last_change":"2024-09-19T05:45:43.611774+0000","last_active":"2024-09-19T05:45:43.611748+0000","last_peered":"2024-09-19T05:45:43.611748+0000","last_clean":"2024-09-19T05:45:43.611748+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:43.611748+0000","last_undegraded":"2024-09-19T05:45:43.611748+0000","last_fullsized":"2024-09-19T05:45:43.611748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:43.611661+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:43.611661+0000","last_clean_scrub_stamp":"2024-09-19T05:45:43.611661+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.431838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"102'217","reported_seq":398,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.096956+0000","last_change":"2024-09-19T05:45:38.096985+0000","last_active":"2024-09-19T05:45:38.096956+0000","last_peered":"2024-09-19T05:45:38.096956+0000","last_clean":"2024-09-19T05:45:38.096956+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:38.096956+0000","last_undegraded":"2024-09-19T05:45:38.096956+0000","last_fullsized":"2024-09-19T05:45:38.096956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'217","last_scrub_stamp":"2024-09-19T05:45:38.096867+0000","last_deep_scrub":"102'217","last_deep_scrub_stamp":"2024-09-19T05:45:38.096867+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.096867+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:08.940001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"102'166","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.214314+0000","last_change":"2024-09-19T05:45:35.214339+0000","last_active":"2024-09-19T05:45:35.214314+0000","last_peered":"2024-09-19T05:45:35.214314+0000","last_clean":"2024-09-19T05:45:35.214314+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:35.214314+0000","last_undegraded":"2024-09-19T05:45:35.214314+0000","last_fullsized":"2024-09-19T05:45:35.214314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'166","last_scrub_stamp":"2024-09-19T05:45:35.214225+0000","last_deep_scrub":"102'166","last_deep_scrub_stamp":"2024-09-19T05:45:35.214225+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.214225+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:53.696897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"102'152","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.180427+0000","last_change":"2024-09-19T05:45:40.180487+0000","last_active":"2024-09-19T05:45:40.180427+0000","last_peered":"2024-09-19T05:45:40.180427+0000","last_clean":"2024-09-19T05:45:40.180427+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:40.180427+0000","last_undegraded":"2024-09-19T05:45:40.180427+0000","last_fullsized":"2024-09-19T05:45:40.180427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'152","last_scrub_stamp":"2024-09-19T05:45:40.180307+0000","last_deep_scrub":"102'152","last_deep_scrub_stamp":"2024-09-19T05:45:40.180307+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.180307+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:47.582255+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"102'134","reported_seq":297,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.534857+0000","last_change":"2024-09-19T05:45:39.534898+0000","last_active":"2024-09-19T05:45:39.534857+0000","last_peered":"2024-09-19T05:45:39.534857+0000","last_clean":"2024-09-19T05:45:39.534857+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:39.534857+0000","last_undegraded":"2024-09-19T05:45:39.534857+0000","last_fullsized":"2024-09-19T05:45:39.534857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'134","last_scrub_stamp":"2024-09-19T05:45:39.534769+0000","last_deep_scrub":"102'134","last_deep_scrub_stamp":"2024-09-19T05:45:39.534769+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.534769+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:19.274773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"102'153","reported_seq":417,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:26.554231+0000","last_change":"2024-09-19T05:44:53.510921+0000","last_active":"2024-09-19T05:45:26.554231+0000","last_peered":"2024-09-19T05:45:26.554231+0000","last_clean":"2024-09-19T05:45:26.554231+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:26.554231+0000","last_undegraded":"2024-09-19T05:45:26.554231+0000","last_fullsized":"2024-09-19T05:45:26.554231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:42:31.965257+0000","last_clean_scrub_stamp":"2024-09-19T05:42:31.965257+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"102'158","reported_seq":350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.267518+0000","last_change":"2024-09-19T05:45:37.267546+0000","last_active":"2024-09-19T05:45:37.267518+0000","last_peered":"2024-09-19T05:45:37.267518+0000","last_clean":"2024-09-19T05:45:37.267518+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:37.267518+0000","last_undegraded":"2024-09-19T05:45:37.267518+0000","last_fullsized":"2024-09-19T05:45:37.267518+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'158","last_scrub_stamp":"2024-09-19T05:45:37.267434+0000","last_deep_scrub":"102'158","last_deep_scrub_stamp":"2024-09-19T05:45:37.267434+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.267434+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:55.750105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"102'180","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.057912+0000","last_change":"2024-09-19T05:45:33.057952+0000","last_active":"2024-09-19T05:45:33.057912+0000","last_peered":"2024-09-19T05:45:33.057912+0000","last_clean":"2024-09-19T05:45:33.057912+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:33.057912+0000","last_undegraded":"2024-09-19T05:45:33.057912+0000","last_fullsized":"2024-09-19T05:45:33.057912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'180","last_scrub_stamp":"2024-09-19T05:45:33.057837+0000","last_deep_scrub":"102'180","last_deep_scrub_stamp":"2024-09-19T05:45:33.057837+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.057837+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:27.994419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"102'149","reported_seq":257,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.974444+0000","last_change":"2024-09-19T05:45:31.974444+0000","last_active":"2024-09-19T05:45:31.974444+0000","last_peered":"2024-09-19T05:45:31.974444+0000","last_clean":"2024-09-19T05:45:31.974444+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:31.974444+0000","last_undegraded":"2024-09-19T05:45:31.974444+0000","last_fullsized":"2024-09-19T05:45:31.974444+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'149","last_scrub_stamp":"2024-09-19T05:45:31.974354+0000","last_deep_scrub":"102'149","last_deep_scrub_stamp":"2024-09-19T05:45:31.974354+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.974354+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:31.995046+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"102'1811","reported_seq":1896,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.534786+0000","last_change":"2024-09-19T05:45:32.534812+0000","last_active":"2024-09-19T05:45:32.534786+0000","last_peered":"2024-09-19T05:45:32.534786+0000","last_clean":"2024-09-19T05:45:32.534786+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:32.534786+0000","last_undegraded":"2024-09-19T05:45:32.534786+0000","last_fullsized":"2024-09-19T05:45:32.534786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'1811","last_scrub_stamp":"2024-09-19T05:45:32.534693+0000","last_deep_scrub":"102'1811","last_deep_scrub_stamp":"2024-09-19T05:45:32.534693+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.534693+0000","objects_scrubbed":1,"log_size":1811,"log_dups_size":0,"ondisk_log_size":1811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:52.367253+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774343,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":5432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"102'162","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.830449+0000","last_change":"2024-09-19T05:45:40.830476+0000","last_active":"2024-09-19T05:45:40.830449+0000","last_peered":"2024-09-19T05:45:40.830449+0000","last_clean":"2024-09-19T05:45:40.830449+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:40.830449+0000","last_undegraded":"2024-09-19T05:45:40.830449+0000","last_fullsized":"2024-09-19T05:45:40.830449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:40.830359+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:40.830359+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.830359+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:49.876265+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.777206+0000","last_change":"2024-09-19T05:45:41.777234+0000","last_active":"2024-09-19T05:45:41.777206+0000","last_peered":"2024-09-19T05:45:41.777206+0000","last_clean":"2024-09-19T05:45:41.777206+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:41.777206+0000","last_undegraded":"2024-09-19T05:45:41.777206+0000","last_fullsized":"2024-09-19T05:45:41.777206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-19T05:45:41.777118+0000","last_deep_scrub":"22'1","last_deep_scrub_stamp":"2024-09-19T05:45:41.777118+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.777118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:15.713480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.226659+0000","last_change":"2024-09-19T05:45:33.226723+0000","last_active":"2024-09-19T05:45:33.226659+0000","last_peered":"2024-09-19T05:45:33.226659+0000","last_clean":"2024-09-19T05:45:33.226659+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:33.226659+0000","last_undegraded":"2024-09-19T05:45:33.226659+0000","last_fullsized":"2024-09-19T05:45:33.226659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:33.226532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:33.226532+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.226532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:24.349722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"102'190","reported_seq":354,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:48.660050+0000","last_change":"2024-09-19T05:45:48.660106+0000","last_active":"2024-09-19T05:45:48.660050+0000","last_peered":"2024-09-19T05:45:48.660050+0000","last_clean":"2024-09-19T05:45:48.660050+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:48.660050+0000","last_undegraded":"2024-09-19T05:45:48.660050+0000","last_fullsized":"2024-09-19T05:45:48.660050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'190","last_scrub_stamp":"2024-09-19T05:45:48.659940+0000","last_deep_scrub":"102'190","last_deep_scrub_stamp":"2024-09-19T05:45:48.659940+0000","last_clean_scrub_stamp":"2024-09-19T05:45:48.659940+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:20.259036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":192,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:48.667187+0000","last_change":"2024-09-19T05:45:48.667213+0000","last_active":"2024-09-19T05:45:48.667187+0000","last_peered":"2024-09-19T05:45:48.667187+0000","last_clean":"2024-09-19T05:45:48.667187+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:48.667187+0000","last_undegraded":"2024-09-19T05:45:48.667187+0000","last_fullsized":"2024-09-19T05:45:48.667187+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:48.667092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:48.667092+0000","last_clean_scrub_stamp":"2024-09-19T05:45:48.667092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:03.431257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"102'187","reported_seq":378,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.803876+0000","last_change":"2024-09-19T05:45:41.803962+0000","last_active":"2024-09-19T05:45:41.803876+0000","last_peered":"2024-09-19T05:45:41.803876+0000","last_clean":"2024-09-19T05:45:41.803876+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:41.803876+0000","last_undegraded":"2024-09-19T05:45:41.803876+0000","last_fullsized":"2024-09-19T05:45:41.803876+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'187","last_scrub_stamp":"2024-09-19T05:45:41.803751+0000","last_deep_scrub":"102'187","last_deep_scrub_stamp":"2024-09-19T05:45:41.803751+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.803751+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:30.000956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":87,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:30.937711+0000","last_change":"2024-09-19T05:45:30.937711+0000","last_active":"2024-09-19T05:45:30.937711+0000","last_peered":"2024-09-19T05:45:30.937711+0000","last_clean":"2024-09-19T05:45:30.937711+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:30.937711+0000","last_undegraded":"2024-09-19T05:45:30.937711+0000","last_fullsized":"2024-09-19T05:45:30.937711+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-19T05:45:30.937601+0000","last_deep_scrub":"59'1","last_deep_scrub_stamp":"2024-09-19T05:45:30.937601+0000","last_clean_scrub_stamp":"2024-09-19T05:45:30.937601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:49.420272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"102'251","reported_seq":432,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.107629+0000","last_change":"2024-09-19T05:45:37.107679+0000","last_active":"2024-09-19T05:45:37.107629+0000","last_peered":"2024-09-19T05:45:37.107629+0000","last_clean":"2024-09-19T05:45:37.107629+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:37.107629+0000","last_undegraded":"2024-09-19T05:45:37.107629+0000","last_fullsized":"2024-09-19T05:45:37.107629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'251","last_scrub_stamp":"2024-09-19T05:45:37.107526+0000","last_deep_scrub":"102'251","last_deep_scrub_stamp":"2024-09-19T05:45:37.107526+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.107526+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:14.809070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":193,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.947047+0000","last_change":"2024-09-19T05:45:38.947090+0000","last_active":"2024-09-19T05:45:38.947047+0000","last_peered":"2024-09-19T05:45:38.947047+0000","last_clean":"2024-09-19T05:45:38.947047+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:38.947047+0000","last_undegraded":"2024-09-19T05:45:38.947047+0000","last_fullsized":"2024-09-19T05:45:38.947047+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:38.946945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:38.946945+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.946945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:52.461934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"102'160","reported_seq":318,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:47.628673+0000","last_change":"2024-09-19T05:45:47.628700+0000","last_active":"2024-09-19T05:45:47.628673+0000","last_peered":"2024-09-19T05:45:47.628673+0000","last_clean":"2024-09-19T05:45:47.628673+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:47.628673+0000","last_undegraded":"2024-09-19T05:45:47.628673+0000","last_fullsized":"2024-09-19T05:45:47.628673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:47.628584+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:47.628584+0000","last_clean_scrub_stamp":"2024-09-19T05:45:47.628584+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:49.123820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"102'11","reported_seq":195,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.307726+0000","last_change":"2024-09-19T05:45:33.307757+0000","last_active":"2024-09-19T05:45:33.307726+0000","last_peered":"2024-09-19T05:45:33.307726+0000","last_clean":"2024-09-19T05:45:33.307726+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:33.307726+0000","last_undegraded":"2024-09-19T05:45:33.307726+0000","last_fullsized":"2024-09-19T05:45:33.307726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'11","last_scrub_stamp":"2024-09-19T05:45:33.307632+0000","last_deep_scrub":"102'11","last_deep_scrub_stamp":"2024-09-19T05:45:33.307632+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.307632+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:53.716103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'168","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.767899+0000","last_change":"2024-09-19T05:45:35.767930+0000","last_active":"2024-09-19T05:45:35.767899+0000","last_peered":"2024-09-19T05:45:35.767899+0000","last_clean":"2024-09-19T05:45:35.767899+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:35.767899+0000","last_undegraded":"2024-09-19T05:45:35.767899+0000","last_fullsized":"2024-09-19T05:45:35.767899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'168","last_scrub_stamp":"2024-09-19T05:45:35.767811+0000","last_deep_scrub":"102'168","last_deep_scrub_stamp":"2024-09-19T05:45:35.767811+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.767811+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:33.355183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"102'2020","reported_seq":2203,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.022036+0000","last_change":"2024-09-19T05:45:31.022089+0000","last_active":"2024-09-19T05:45:31.022036+0000","last_peered":"2024-09-19T05:45:31.022036+0000","last_clean":"2024-09-19T05:45:31.022036+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:31.022036+0000","last_undegraded":"2024-09-19T05:45:31.022036+0000","last_fullsized":"2024-09-19T05:45:31.022036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'2020","last_scrub_stamp":"2024-09-19T05:45:31.021952+0000","last_deep_scrub":"102'2020","last_deep_scrub_stamp":"2024-09-19T05:45:31.021952+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.021952+0000","objects_scrubbed":4,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:59.591416+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"102'176","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.845180+0000","last_change":"2024-09-19T05:45:32.845203+0000","last_active":"2024-09-19T05:45:32.845180+0000","last_peered":"2024-09-19T05:45:32.845180+0000","last_clean":"2024-09-19T05:45:32.845180+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:32.845180+0000","last_undegraded":"2024-09-19T05:45:32.845180+0000","last_fullsized":"2024-09-19T05:45:32.845180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'176","last_scrub_stamp":"2024-09-19T05:45:32.845113+0000","last_deep_scrub":"102'176","last_deep_scrub_stamp":"2024-09-19T05:45:32.845113+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.845113+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:30.432485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"102'4","reported_seq":188,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.273378+0000","last_change":"2024-09-19T05:45:34.273411+0000","last_active":"2024-09-19T05:45:34.273378+0000","last_peered":"2024-09-19T05:45:34.273378+0000","last_clean":"2024-09-19T05:45:34.273378+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:34.273378+0000","last_undegraded":"2024-09-19T05:45:34.273378+0000","last_fullsized":"2024-09-19T05:45:34.273378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'4","last_scrub_stamp":"2024-09-19T05:45:34.273259+0000","last_deep_scrub":"102'4","last_deep_scrub_stamp":"2024-09-19T05:45:34.273259+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.273259+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:23.993898+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"102'177","reported_seq":358,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.169310+0000","last_change":"2024-09-19T05:45:38.169360+0000","last_active":"2024-09-19T05:45:38.169310+0000","last_peered":"2024-09-19T05:45:38.169310+0000","last_clean":"2024-09-19T05:45:38.169310+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:38.169310+0000","last_undegraded":"2024-09-19T05:45:38.169310+0000","last_fullsized":"2024-09-19T05:45:38.169310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:38.169212+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:38.169212+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.169212+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:49.384368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.751034+0000","last_change":"2024-09-19T05:45:36.751059+0000","last_active":"2024-09-19T05:45:36.751034+0000","last_peered":"2024-09-19T05:45:36.751034+0000","last_clean":"2024-09-19T05:45:36.751034+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:36.751034+0000","last_undegraded":"2024-09-19T05:45:36.751034+0000","last_fullsized":"2024-09-19T05:45:36.751034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:36.750946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:36.750946+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.750946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:19.100985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"102'201","reported_seq":381,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:43.760210+0000","last_change":"2024-09-19T05:45:43.760241+0000","last_active":"2024-09-19T05:45:43.760210+0000","last_peered":"2024-09-19T05:45:43.760210+0000","last_clean":"2024-09-19T05:45:43.760210+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:43.760210+0000","last_undegraded":"2024-09-19T05:45:43.760210+0000","last_fullsized":"2024-09-19T05:45:43.760210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'201","last_scrub_stamp":"2024-09-19T05:45:43.760115+0000","last_deep_scrub":"102'201","last_deep_scrub_stamp":"2024-09-19T05:45:43.760115+0000","last_clean_scrub_stamp":"2024-09-19T05:45:43.760115+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:47.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:30.749483+0000","last_change":"2024-09-19T05:45:30.749518+0000","last_active":"2024-09-19T05:45:30.749483+0000","last_peered":"2024-09-19T05:45:30.749483+0000","last_clean":"2024-09-19T05:45:30.749483+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:30.749483+0000","last_undegraded":"2024-09-19T05:45:30.749483+0000","last_fullsized":"2024-09-19T05:45:30.749483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:30.749367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:30.749367+0000","last_clean_scrub_stamp":"2024-09-19T05:45:30.749367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:01.968783+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"102'197","reported_seq":299,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.042227+0000","last_change":"2024-09-19T05:45:35.042227+0000","last_active":"2024-09-19T05:45:35.042227+0000","last_peered":"2024-09-19T05:45:35.042227+0000","last_clean":"2024-09-19T05:45:35.042227+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:35.042227+0000","last_undegraded":"2024-09-19T05:45:35.042227+0000","last_fullsized":"2024-09-19T05:45:35.042227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'197","last_scrub_stamp":"2024-09-19T05:45:35.042112+0000","last_deep_scrub":"102'197","last_deep_scrub_stamp":"2024-09-19T05:45:35.042112+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.042112+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:47.658746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"102'13","reported_seq":196,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.667490+0000","last_change":"2024-09-19T05:45:31.667541+0000","last_active":"2024-09-19T05:45:31.667490+0000","last_peered":"2024-09-19T05:45:31.667490+0000","last_clean":"2024-09-19T05:45:31.667490+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:31.667490+0000","last_undegraded":"2024-09-19T05:45:31.667490+0000","last_fullsized":"2024-09-19T05:45:31.667490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'13","last_scrub_stamp":"2024-09-19T05:45:31.667407+0000","last_deep_scrub":"102'13","last_deep_scrub_stamp":"2024-09-19T05:45:31.667407+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.667407+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:51.674841+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"102'215","reported_seq":395,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:44.770414+0000","last_change":"2024-09-19T05:45:44.770442+0000","last_active":"2024-09-19T05:45:44.770414+0000","last_peered":"2024-09-19T05:45:44.770414+0000","last_clean":"2024-09-19T05:45:44.770414+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:44.770414+0000","last_undegraded":"2024-09-19T05:45:44.770414+0000","last_fullsized":"2024-09-19T05:45:44.770414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'215","last_scrub_stamp":"2024-09-19T05:45:44.770328+0000","last_deep_scrub":"102'215","last_deep_scrub_stamp":"2024-09-19T05:45:44.770328+0000","last_clean_scrub_stamp":"2024-09-19T05:45:44.770328+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:47.527164+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"102'2","reported_seq":192,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:43.847654+0000","last_change":"2024-09-19T05:45:43.847686+0000","last_active":"2024-09-19T05:45:43.847654+0000","last_peered":"2024-09-19T05:45:43.847654+0000","last_clean":"2024-09-19T05:45:43.847654+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:43.847654+0000","last_undegraded":"2024-09-19T05:45:43.847654+0000","last_fullsized":"2024-09-19T05:45:43.847654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'2","last_scrub_stamp":"2024-09-19T05:45:43.847574+0000","last_deep_scrub":"102'2","last_deep_scrub_stamp":"2024-09-19T05:45:43.847574+0000","last_clean_scrub_stamp":"2024-09-19T05:45:43.847574+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:09.787140+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"102'175","reported_seq":356,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.054766+0000","last_change":"2024-09-19T05:45:32.054813+0000","last_active":"2024-09-19T05:45:32.054766+0000","last_peered":"2024-09-19T05:45:32.054766+0000","last_clean":"2024-09-19T05:45:32.054766+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:32.054766+0000","last_undegraded":"2024-09-19T05:45:32.054766+0000","last_fullsized":"2024-09-19T05:45:32.054766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'175","last_scrub_stamp":"2024-09-19T05:45:32.054677+0000","last_deep_scrub":"102'175","last_deep_scrub_stamp":"2024-09-19T05:45:32.054677+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.054677+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:42.949312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"102'3","reported_seq":193,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.799399+0000","last_change":"2024-09-19T05:45:42.799425+0000","last_active":"2024-09-19T05:45:42.799399+0000","last_peered":"2024-09-19T05:45:42.799399+0000","last_clean":"2024-09-19T05:45:42.799399+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:42.799399+0000","last_undegraded":"2024-09-19T05:45:42.799399+0000","last_fullsized":"2024-09-19T05:45:42.799399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'3","last_scrub_stamp":"2024-09-19T05:45:42.799316+0000","last_deep_scrub":"102'3","last_deep_scrub_stamp":"2024-09-19T05:45:42.799316+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.799316+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:14.018732+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"102'162","reported_seq":249,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.268652+0000","last_change":"2024-09-19T05:45:38.268679+0000","last_active":"2024-09-19T05:45:38.268652+0000","last_peered":"2024-09-19T05:45:38.268652+0000","last_clean":"2024-09-19T05:45:38.268652+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:38.268652+0000","last_undegraded":"2024-09-19T05:45:38.268652+0000","last_fullsized":"2024-09-19T05:45:38.268652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:38.268565+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:38.268565+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.268565+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:38.289257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"102'2","reported_seq":135,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.529324+0000","last_change":"2024-09-19T05:45:33.529353+0000","last_active":"2024-09-19T05:45:33.529324+0000","last_peered":"2024-09-19T05:45:33.529324+0000","last_clean":"2024-09-19T05:45:33.529324+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:33.529324+0000","last_undegraded":"2024-09-19T05:45:33.529324+0000","last_fullsized":"2024-09-19T05:45:33.529324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'2","last_scrub_stamp":"2024-09-19T05:45:33.529245+0000","last_deep_scrub":"102'2","last_deep_scrub_stamp":"2024-09-19T05:45:33.529245+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.529245+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:46.002935+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"102'165","reported_seq":346,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.798117+0000","last_change":"2024-09-19T05:45:31.798163+0000","last_active":"2024-09-19T05:45:31.798117+0000","last_peered":"2024-09-19T05:45:31.798117+0000","last_clean":"2024-09-19T05:45:31.798117+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:31.798117+0000","last_undegraded":"2024-09-19T05:45:31.798117+0000","last_fullsized":"2024-09-19T05:45:31.798117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'165","last_scrub_stamp":"2024-09-19T05:45:31.798029+0000","last_deep_scrub":"102'165","last_deep_scrub_stamp":"2024-09-19T05:45:31.798029+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.798029+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:25:37.298118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.200108+0000","last_change":"2024-09-19T05:45:34.200131+0000","last_active":"2024-09-19T05:45:34.200108+0000","last_peered":"2024-09-19T05:45:34.200108+0000","last_clean":"2024-09-19T05:45:34.200108+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:34.200108+0000","last_undegraded":"2024-09-19T05:45:34.200108+0000","last_fullsized":"2024-09-19T05:45:34.200108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:34.200018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:34.200018+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.200018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:54.207452+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"102'175","reported_seq":365,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:47.682596+0000","last_change":"2024-09-19T05:45:47.682624+0000","last_active":"2024-09-19T05:45:47.682596+0000","last_peered":"2024-09-19T05:45:47.682596+0000","last_clean":"2024-09-19T05:45:47.682596+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:47.682596+0000","last_undegraded":"2024-09-19T05:45:47.682596+0000","last_fullsized":"2024-09-19T05:45:47.682596+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'175","last_scrub_stamp":"2024-09-19T05:45:47.682504+0000","last_deep_scrub":"102'175","last_deep_scrub_stamp":"2024-09-19T05:45:47.682504+0000","last_clean_scrub_stamp":"2024-09-19T05:45:47.682504+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:38.251978+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":272,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.995860+0000","last_change":"2024-09-19T05:45:33.995895+0000","last_active":"2024-09-19T05:45:33.995860+0000","last_peered":"2024-09-19T05:45:33.995860+0000","last_clean":"2024-09-19T05:45:33.995860+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:33.995860+0000","last_undegraded":"2024-09-19T05:45:33.995860+0000","last_fullsized":"2024-09-19T05:45:33.995860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'119","last_scrub_stamp":"2024-09-19T05:45:33.995772+0000","last_deep_scrub":"102'119","last_deep_scrub_stamp":"2024-09-19T05:45:33.995772+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.995772+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:45.210928+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.768061+0000","last_change":"2024-09-19T05:45:37.768087+0000","last_active":"2024-09-19T05:45:37.768061+0000","last_peered":"2024-09-19T05:45:37.768061+0000","last_clean":"2024-09-19T05:45:37.768061+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:37.768061+0000","last_undegraded":"2024-09-19T05:45:37.768061+0000","last_fullsized":"2024-09-19T05:45:37.768061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:37.767970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:37.767970+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.767970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:29.604731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"102'183","reported_seq":364,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.766613+0000","last_change":"2024-09-19T05:45:36.766670+0000","last_active":"2024-09-19T05:45:36.766613+0000","last_peered":"2024-09-19T05:45:36.766613+0000","last_clean":"2024-09-19T05:45:36.766613+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:36.766613+0000","last_undegraded":"2024-09-19T05:45:36.766613+0000","last_fullsized":"2024-09-19T05:45:36.766613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'183","last_scrub_stamp":"2024-09-19T05:45:36.766515+0000","last_deep_scrub":"102'183","last_deep_scrub_stamp":"2024-09-19T05:45:36.766515+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.766515+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:19.693027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:49.713452+0000","last_change":"2024-09-19T05:45:49.713476+0000","last_active":"2024-09-19T05:45:49.713452+0000","last_peered":"2024-09-19T05:45:49.713452+0000","last_clean":"2024-09-19T05:45:49.713452+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:49.713452+0000","last_undegraded":"2024-09-19T05:45:49.713452+0000","last_fullsized":"2024-09-19T05:45:49.713452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:49.713346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:49.713346+0000","last_clean_scrub_stamp":"2024-09-19T05:45:49.713346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:02.064903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"102'165","reported_seq":346,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.199994+0000","last_change":"2024-09-19T05:45:37.200020+0000","last_active":"2024-09-19T05:45:37.199994+0000","last_peered":"2024-09-19T05:45:37.199994+0000","last_clean":"2024-09-19T05:45:37.199994+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:37.199994+0000","last_undegraded":"2024-09-19T05:45:37.199994+0000","last_fullsized":"2024-09-19T05:45:37.199994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'165","last_scrub_stamp":"2024-09-19T05:45:37.199880+0000","last_deep_scrub":"102'165","last_deep_scrub_stamp":"2024-09-19T05:45:37.199880+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.199880+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:36.556460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"102'7","reported_seq":207,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.225608+0000","last_change":"2024-09-19T05:45:39.225653+0000","last_active":"2024-09-19T05:45:39.225608+0000","last_peered":"2024-09-19T05:45:39.225608+0000","last_clean":"2024-09-19T05:45:39.225608+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:39.225608+0000","last_undegraded":"2024-09-19T05:45:39.225608+0000","last_fullsized":"2024-09-19T05:45:39.225608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'7","last_scrub_stamp":"2024-09-19T05:45:39.225500+0000","last_deep_scrub":"102'7","last_deep_scrub_stamp":"2024-09-19T05:45:39.225500+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.225500+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:38.582081+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"102'165","reported_seq":327,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:46.636789+0000","last_change":"2024-09-19T05:45:46.636817+0000","last_active":"2024-09-19T05:45:46.636789+0000","last_peered":"2024-09-19T05:45:46.636789+0000","last_clean":"2024-09-19T05:45:46.636789+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:46.636789+0000","last_undegraded":"2024-09-19T05:45:46.636789+0000","last_fullsized":"2024-09-19T05:45:46.636789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'165","last_scrub_stamp":"2024-09-19T05:45:46.636697+0000","last_deep_scrub":"102'165","last_deep_scrub_stamp":"2024-09-19T05:45:46.636697+0000","last_clean_scrub_stamp":"2024-09-19T05:45:46.636697+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:25.920793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.772284+0000","last_change":"2024-09-19T05:45:42.772308+0000","last_active":"2024-09-19T05:45:42.772284+0000","last_peered":"2024-09-19T05:45:42.772284+0000","last_clean":"2024-09-19T05:45:42.772284+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:42.772284+0000","last_undegraded":"2024-09-19T05:45:42.772284+0000","last_fullsized":"2024-09-19T05:45:42.772284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:42.772179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:42.772179+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.772179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:56.287167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"102'177","reported_seq":358,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.865683+0000","last_change":"2024-09-19T05:45:34.865709+0000","last_active":"2024-09-19T05:45:34.865683+0000","last_peered":"2024-09-19T05:45:34.865683+0000","last_clean":"2024-09-19T05:45:34.865683+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:34.865683+0000","last_undegraded":"2024-09-19T05:45:34.865683+0000","last_fullsized":"2024-09-19T05:45:34.865683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:34.865592+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:34.865592+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.865592+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:38.616601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":160,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.526145+0000","last_change":"2024-09-19T05:45:35.526168+0000","last_active":"2024-09-19T05:45:35.526145+0000","last_peered":"2024-09-19T05:45:35.526145+0000","last_clean":"2024-09-19T05:45:35.526145+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:35.526145+0000","last_undegraded":"2024-09-19T05:45:35.526145+0000","last_fullsized":"2024-09-19T05:45:35.526145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:35.526057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:35.526057+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.526057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:06.080131+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"102'160","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.073051+0000","last_change":"2024-09-19T05:45:34.073085+0000","last_active":"2024-09-19T05:45:34.073051+0000","last_peered":"2024-09-19T05:45:34.073051+0000","last_clean":"2024-09-19T05:45:34.073051+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:34.073051+0000","last_undegraded":"2024-09-19T05:45:34.073051+0000","last_fullsized":"2024-09-19T05:45:34.073051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:34.072956+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:34.072956+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.072956+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:29.693246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.761958+0000","last_change":"2024-09-19T05:45:40.761980+0000","last_active":"2024-09-19T05:45:40.761958+0000","last_peered":"2024-09-19T05:45:40.761958+0000","last_clean":"2024-09-19T05:45:40.761958+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:40.761958+0000","last_undegraded":"2024-09-19T05:45:40.761958+0000","last_fullsized":"2024-09-19T05:45:40.761958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:40.761873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:40.761873+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.761873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:02.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"102'162","reported_seq":442,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.190859+0000","last_change":"2024-09-19T05:45:42.190883+0000","last_active":"2024-09-19T05:45:42.190859+0000","last_peered":"2024-09-19T05:45:42.190859+0000","last_clean":"2024-09-19T05:45:42.190859+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:42.190859+0000","last_undegraded":"2024-09-19T05:45:42.190859+0000","last_fullsized":"2024-09-19T05:45:42.190859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:42.190770+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:42.190770+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.190770+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:56:50.729470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"102'18","reported_seq":201,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.668423+0000","last_change":"2024-09-19T05:45:33.668450+0000","last_active":"2024-09-19T05:45:33.668423+0000","last_peered":"2024-09-19T05:45:33.668423+0000","last_clean":"2024-09-19T05:45:33.668423+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:33.668423+0000","last_undegraded":"2024-09-19T05:45:33.668423+0000","last_fullsized":"2024-09-19T05:45:33.668423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'18","last_scrub_stamp":"2024-09-19T05:45:33.668343+0000","last_deep_scrub":"102'18","last_deep_scrub_stamp":"2024-09-19T05:45:33.668343+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.668343+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:55.586362+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"102'166","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.790245+0000","last_change":"2024-09-19T05:45:39.790269+0000","last_active":"2024-09-19T05:45:39.790245+0000","last_peered":"2024-09-19T05:45:39.790245+0000","last_clean":"2024-09-19T05:45:39.790245+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:39.790245+0000","last_undegraded":"2024-09-19T05:45:39.790245+0000","last_fullsized":"2024-09-19T05:45:39.790245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'166","last_scrub_stamp":"2024-09-19T05:45:39.790160+0000","last_deep_scrub":"102'166","last_deep_scrub_stamp":"2024-09-19T05:45:39.790160+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.790160+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:56:48.328860+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"102'4","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.762532+0000","last_change":"2024-09-19T05:45:38.762562+0000","last_active":"2024-09-19T05:45:38.762532+0000","last_peered":"2024-09-19T05:45:38.762532+0000","last_clean":"2024-09-19T05:45:38.762532+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:38.762532+0000","last_undegraded":"2024-09-19T05:45:38.762532+0000","last_fullsized":"2024-09-19T05:45:38.762532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'4","last_scrub_stamp":"2024-09-19T05:45:38.762445+0000","last_deep_scrub":"102'4","last_deep_scrub_stamp":"2024-09-19T05:45:38.762445+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.762445+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:52.648999+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"102'183","reported_seq":364,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.780265+0000","last_change":"2024-09-19T05:45:37.780304+0000","last_active":"2024-09-19T05:45:37.780265+0000","last_peered":"2024-09-19T05:45:37.780265+0000","last_clean":"2024-09-19T05:45:37.780265+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:37.780265+0000","last_undegraded":"2024-09-19T05:45:37.780265+0000","last_fullsized":"2024-09-19T05:45:37.780265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'183","last_scrub_stamp":"2024-09-19T05:45:37.780138+0000","last_deep_scrub":"102'183","last_deep_scrub_stamp":"2024-09-19T05:45:37.780138+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.780138+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:06.688522+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:29.781412+0000","last_change":"2024-09-19T05:45:29.781446+0000","last_active":"2024-09-19T05:45:29.781412+0000","last_peered":"2024-09-19T05:45:29.781412+0000","last_clean":"2024-09-19T05:45:29.781412+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:29.781412+0000","last_undegraded":"2024-09-19T05:45:29.781412+0000","last_fullsized":"2024-09-19T05:45:29.781412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:29.781333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:29.781333+0000","last_clean_scrub_stamp":"2024-09-19T05:45:29.781333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:06.401746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"102'162","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.844185+0000","last_change":"2024-09-19T05:45:31.844225+0000","last_active":"2024-09-19T05:45:31.844185+0000","last_peered":"2024-09-19T05:45:31.844185+0000","last_clean":"2024-09-19T05:45:31.844185+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:31.844185+0000","last_undegraded":"2024-09-19T05:45:31.844185+0000","last_fullsized":"2024-09-19T05:45:31.844185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:31.844083+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:31.844083+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.844083+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:40.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.631333+0000","last_change":"2024-09-19T05:45:32.631357+0000","last_active":"2024-09-19T05:45:32.631333+0000","last_peered":"2024-09-19T05:45:32.631333+0000","last_clean":"2024-09-19T05:45:32.631333+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:32.631333+0000","last_undegraded":"2024-09-19T05:45:32.631333+0000","last_fullsized":"2024-09-19T05:45:32.631333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:32.631251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:32.631251+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.631251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:54.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"102'173","reported_seq":354,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.761103+0000","last_change":"2024-09-19T05:45:35.761129+0000","last_active":"2024-09-19T05:45:35.761103+0000","last_peered":"2024-09-19T05:45:35.761103+0000","last_clean":"2024-09-19T05:45:35.761103+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:35.761103+0000","last_undegraded":"2024-09-19T05:45:35.761103+0000","last_fullsized":"2024-09-19T05:45:35.761103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'173","last_scrub_stamp":"2024-09-19T05:45:35.761012+0000","last_deep_scrub":"102'173","last_deep_scrub_stamp":"2024-09-19T05:45:35.761012+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.761012+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:46.976167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":190,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.168489+0000","last_change":"2024-09-19T05:45:41.168521+0000","last_active":"2024-09-19T05:45:41.168489+0000","last_peered":"2024-09-19T05:45:41.168489+0000","last_clean":"2024-09-19T05:45:41.168489+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:41.168489+0000","last_undegraded":"2024-09-19T05:45:41.168489+0000","last_fullsized":"2024-09-19T05:45:41.168489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:41.168401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:41.168401+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.168401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:47.777454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"102'164","reported_seq":346,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.238777+0000","last_change":"2024-09-19T05:45:35.238803+0000","last_active":"2024-09-19T05:45:35.238777+0000","last_peered":"2024-09-19T05:45:35.238777+0000","last_clean":"2024-09-19T05:45:35.238777+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:35.238777+0000","last_undegraded":"2024-09-19T05:45:35.238777+0000","last_fullsized":"2024-09-19T05:45:35.238777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'164","last_scrub_stamp":"2024-09-19T05:45:35.238693+0000","last_deep_scrub":"102'164","last_deep_scrub_stamp":"2024-09-19T05:45:35.238693+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.238693+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:26.361883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"102'4","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.809734+0000","last_change":"2024-09-19T05:45:39.809775+0000","last_active":"2024-09-19T05:45:39.809734+0000","last_peered":"2024-09-19T05:45:39.809734+0000","last_clean":"2024-09-19T05:45:39.809734+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:39.809734+0000","last_undegraded":"2024-09-19T05:45:39.809734+0000","last_fullsized":"2024-09-19T05:45:39.809734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'4","last_scrub_stamp":"2024-09-19T05:45:39.809653+0000","last_deep_scrub":"102'4","last_deep_scrub_stamp":"2024-09-19T05:45:39.809653+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.809653+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:50:42.917222+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"102'154","reported_seq":344,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:46.709760+0000","last_change":"2024-09-19T05:45:46.709788+0000","last_active":"2024-09-19T05:45:46.709760+0000","last_peered":"2024-09-19T05:45:46.709760+0000","last_clean":"2024-09-19T05:45:46.709760+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:46.709760+0000","last_undegraded":"2024-09-19T05:45:46.709760+0000","last_fullsized":"2024-09-19T05:45:46.709760+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'154","last_scrub_stamp":"2024-09-19T05:45:46.709671+0000","last_deep_scrub":"102'154","last_deep_scrub_stamp":"2024-09-19T05:45:46.709671+0000","last_clean_scrub_stamp":"2024-09-19T05:45:46.709671+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:41.429840+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"102'205","reported_seq":385,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:45.721727+0000","last_change":"2024-09-19T05:45:45.721775+0000","last_active":"2024-09-19T05:45:45.721727+0000","last_peered":"2024-09-19T05:45:45.721727+0000","last_clean":"2024-09-19T05:45:45.721727+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:45.721727+0000","last_undegraded":"2024-09-19T05:45:45.721727+0000","last_fullsized":"2024-09-19T05:45:45.721727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'205","last_scrub_stamp":"2024-09-19T05:45:45.721639+0000","last_deep_scrub":"102'205","last_deep_scrub_stamp":"2024-09-19T05:45:45.721639+0000","last_clean_scrub_stamp":"2024-09-19T05:45:45.721639+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:52.244624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"102'3","reported_seq":204,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.902337+0000","last_change":"2024-09-19T05:45:41.902369+0000","last_active":"2024-09-19T05:45:41.902337+0000","last_peered":"2024-09-19T05:45:41.902337+0000","last_clean":"2024-09-19T05:45:41.902337+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:41.902337+0000","last_undegraded":"2024-09-19T05:45:41.902337+0000","last_fullsized":"2024-09-19T05:45:41.902337+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'3","last_scrub_stamp":"2024-09-19T05:45:41.902233+0000","last_deep_scrub":"102'3","last_deep_scrub_stamp":"2024-09-19T05:45:41.902233+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.902233+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:53.704954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"102'162","reported_seq":285,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.007918+0000","last_change":"2024-09-19T05:45:33.007918+0000","last_active":"2024-09-19T05:45:33.007918+0000","last_peered":"2024-09-19T05:45:33.007918+0000","last_clean":"2024-09-19T05:45:33.007918+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:33.007918+0000","last_undegraded":"2024-09-19T05:45:33.007918+0000","last_fullsized":"2024-09-19T05:45:33.007918+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:33.007831+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:33.007831+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.007831+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:53.254609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"102'5","reported_seq":206,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.957671+0000","last_change":"2024-09-19T05:45:39.957699+0000","last_active":"2024-09-19T05:45:39.957671+0000","last_peered":"2024-09-19T05:45:39.957671+0000","last_clean":"2024-09-19T05:45:39.957671+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:39.957671+0000","last_undegraded":"2024-09-19T05:45:39.957671+0000","last_fullsized":"2024-09-19T05:45:39.957671+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'5","last_scrub_stamp":"2024-09-19T05:45:39.957583+0000","last_deep_scrub":"102'5","last_deep_scrub_stamp":"2024-09-19T05:45:39.957583+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.957583+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:43.815874+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"102'160","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:45.609520+0000","last_change":"2024-09-19T05:45:45.609550+0000","last_active":"2024-09-19T05:45:45.609520+0000","last_peered":"2024-09-19T05:45:45.609520+0000","last_clean":"2024-09-19T05:45:45.609520+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:45.609520+0000","last_undegraded":"2024-09-19T05:45:45.609520+0000","last_fullsized":"2024-09-19T05:45:45.609520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:45.609429+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:45.609429+0000","last_clean_scrub_stamp":"2024-09-19T05:45:45.609429+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:12.464926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":165,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.540836+0000","last_change":"2024-09-19T05:45:34.540863+0000","last_active":"2024-09-19T05:45:34.540836+0000","last_peered":"2024-09-19T05:45:34.540836+0000","last_clean":"2024-09-19T05:45:34.540836+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:34.540836+0000","last_undegraded":"2024-09-19T05:45:34.540836+0000","last_fullsized":"2024-09-19T05:45:34.540836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-19T05:45:34.540747+0000","last_deep_scrub":"22'1","last_deep_scrub_stamp":"2024-09-19T05:45:34.540747+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.540747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:33.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"102'181","reported_seq":335,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:44.635675+0000","last_change":"2024-09-19T05:45:44.635703+0000","last_active":"2024-09-19T05:45:44.635675+0000","last_peered":"2024-09-19T05:45:44.635675+0000","last_clean":"2024-09-19T05:45:44.635675+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:44.635675+0000","last_undegraded":"2024-09-19T05:45:44.635675+0000","last_fullsized":"2024-09-19T05:45:44.635675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'181","last_scrub_stamp":"2024-09-19T05:45:44.635567+0000","last_deep_scrub":"102'181","last_deep_scrub_stamp":"2024-09-19T05:45:44.635567+0000","last_clean_scrub_stamp":"2024-09-19T05:45:44.635567+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:24.061136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"102'6","reported_seq":189,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.718061+0000","last_change":"2024-09-19T05:45:34.718089+0000","last_active":"2024-09-19T05:45:34.718061+0000","last_peered":"2024-09-19T05:45:34.718061+0000","last_clean":"2024-09-19T05:45:34.718061+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:34.718061+0000","last_undegraded":"2024-09-19T05:45:34.718061+0000","last_fullsized":"2024-09-19T05:45:34.718061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'6","last_scrub_stamp":"2024-09-19T05:45:34.717984+0000","last_deep_scrub":"102'6","last_deep_scrub_stamp":"2024-09-19T05:45:34.717984+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.717984+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:54.964762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":11255,"ondisk_log_size":11255,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970622,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3910,"num_write_kb":11502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15941244,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3916,"ondisk_log_size":3916,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54516,"kb_used_data":5284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317324,"statfs":{"total":96636764160,"available":96580939776,"internally_reserved":0,"allocated":5410816,"data_stored":4755262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":4984832,"data_stored":4334629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449102,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":1736,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":1777664,"data_stored":4925594,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50364,"kb_used_data":5292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321476,"statfs":{"total":96636764160,"available":96585191424,"internally_reserved":0,"allocated":5419008,"data_stored":4755934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1212416,"data_stored":560868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40736,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331104,"statfs":{"total":96636764160,"available":96595050496,"internally_reserved":0,"allocated":1261568,"data_stored":1150724,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40692,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331148,"statfs":{"total":96636764160,"available":96595095552,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774940,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:45:53.409 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:53.409 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-19T05:42:31.965257+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=42, tm_sec=31, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=5, tm_min=45, tm_sec=28, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T05:45:53.410 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T05:46:04.087 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:46:04.091 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:46:13.411 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T05:46:13.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.535+0000 7fdfbf3a2640 1 Processor -- start 2024-09-19T05:46:13.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.535+0000 7fdfbf3a2640 1 -- start start 2024-09-19T05:46:13.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.536+0000 7fdfbf3a2640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdfb80c53f0 0x7fdfb80c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.544 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.536+0000 7fdfbf3a2640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb80c5dc0 con 0x7fdfb8154a00 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.536+0000 7fdfbf3a2640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb806c1d0 con 0x7fdfb806d730 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.536+0000 7fdfbf3a2640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb806c470 con 0x7fdfb80c53f0 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.536+0000 7fdfbeba1640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdfb80c53f0 0x7fdfb80c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.536+0000 7fdfbe3a0640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fdfb8154a00 0x7fdfb815b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:60916/0 (socket says 172.21.15.2:60916) 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.536+0000 7fdfbe3a0640 1 -- 172.21.15.2:0/307662995 learned_addr learned my addr 172.21.15.2:0/307662995 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.537+0000 7fdfbeba1640 1 -- 172.21.15.2:0/307662995 >> v1:172.21.15.123:6789/0 conn(0x7fdfb806d730 legacy=0x7fdfb80ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.537+0000 7fdfbeba1640 1 -- 172.21.15.2:0/307662995 >> v1:172.21.15.2:6789/0 conn(0x7fdfb8154a00 legacy=0x7fdfb815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.537+0000 7fdfbeba1640 1 -- 172.21.15.2:0/307662995 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb8154de0 con 0x7fdfb80c53f0 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.537+0000 7fdfbd39e640 1 -- 172.21.15.2:0/307662995 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838667479 0 0) 0x7fdfb80c5dc0 con 0x7fdfb8154a00 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.537+0000 7fdfbeba1640 1 --2- 172.21.15.2:0/307662995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb80c53f0 0x7fdfb80c5880 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00097d0 tx=0x7fdfb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1da3eae9757428 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbd39e640 1 -- 172.21.15.2:0/307662995 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037070 con 0x7fdfb80c53f0 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbd39e640 1 -- 172.21.15.2:0/307662995 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0032440 con 0x7fdfb80c53f0 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbd39e640 1 -- 172.21.15.2:0/307662995 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0032810 con 0x7fdfb80c53f0 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/307662995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb80c53f0 msgr2=0x7fdfb80c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/307662995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb80c53f0 0x7fdfb80c5880 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00097d0 tx=0x7fdfb002a270 comp rx=0 tx=0).stop 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/307662995 shutdown_connections 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/307662995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb80c53f0 0x7fdfb80c5880 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/307662995 >> 172.21.15.2:0/307662995 conn(0x7fdfb8067950 msgr2=0x7fdfb8067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/307662995 shutdown_connections 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.538+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/307662995 wait complete. 2024-09-19T05:46:13.551 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.539+0000 7fdfbf3a2640 1 Processor -- start 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.539+0000 7fdfbf3a2640 1 -- start start 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbf3a2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfb806d730 0x7fdfb8153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbf3a2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb80c53f0 0x7fdfb80c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbf3a2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb8154a00 0x7fdfb80c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbf3a2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb806ccf0 con 0x7fdfb8154a00 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbf3a2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb806ca10 con 0x7fdfb80c53f0 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbf3a2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb806cfd0 con 0x7fdfb806d730 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbeba1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb8154a00 0x7fdfb80c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbeba1640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb8154a00 0x7fdfb80c9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60504/0 (socket says 172.21.15.2:60504) 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbeba1640 1 -- 172.21.15.2:0/3669745859 learned_addr learned my addr 172.21.15.2:0/3669745859 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbdb9f640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb80c53f0 0x7fdfb80c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.540+0000 7fdfbe3a0640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfb806d730 0x7fdfb8153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbeba1640 1 -- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfb806d730 msgr2=0x7fdfb8153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.552 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbeba1640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfb806d730 0x7fdfb8153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbeba1640 1 -- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb80c53f0 msgr2=0x7fdfb80c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbeba1640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb80c53f0 0x7fdfb80c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbeba1640 1 -- 172.21.15.2:0/3669745859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb814af20 con 0x7fdfb8154a00 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbeba1640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb8154a00 0x7fdfb80c9690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00098f0 tx=0x7fdfb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfb6ffd640 1 -- 172.21.15.2:0/3669745859 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037050 con 0x7fdfb8154a00 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfb6ffd640 1 -- 172.21.15.2:0/3669745859 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb0038040 con 0x7fdfb8154a00 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb814ac40 con 0x7fdfb8154a00 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfb6ffd640 1 -- 172.21.15.2:0/3669745859 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0004490 con 0x7fdfb8154a00 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.541+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb81f4670 con 0x7fdfb8154a00 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.542+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb80b3050 con 0x7fdfb8154a00 2024-09-19T05:46:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.543+0000 7fdfb6ffd640 1 -- 172.21.15.2:0/3669745859 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdfb0030980 con 0x7fdfb8154a00 2024-09-19T05:46:13.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.543+0000 7fdfb6ffd640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdfa406c2b0 0x7fdfa406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.544+0000 7fdfb6ffd640 1 -- 172.21.15.2:0/3669745859 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fdfb00abe60 con 0x7fdfb8154a00 2024-09-19T05:46:13.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.544+0000 7fdfbe3a0640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdfa406c2b0 0x7fdfa406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.554 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.544+0000 7fdfbe3a0640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdfa406c2b0 0x7fdfa406e770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80012e0 tx=0x7fdfa8004d40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.570 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.548+0000 7fdfb6ffd640 1 -- 172.21.15.2:0/3669745859 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fdfb80b3050 con 0x7fdfb8154a00 2024-09-19T05:46:13.730 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.721+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 --> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfb80cba10 con 0x7fdfa406c2b0 2024-09-19T05:46:13.733 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.728+0000 7fdfb6ffd640 1 -- 172.21.15.2:0/3669745859 <== mgr.4111 v2:172.21.15.123:6832/302582025 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254369 (secure 0 0 0) 0x7fdfa80075b0 con 0x7fdfa406c2b0 2024-09-19T05:46:13.734 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:46:13.743 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.730+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdfa406c2b0 msgr2=0x7fdfa406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.730+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdfa406c2b0 0x7fdfa406e770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80012e0 tx=0x7fdfa8004d40 comp rx=0 tx=0).stop 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.730+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb8154a00 msgr2=0x7fdfb80c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.730+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb8154a00 0x7fdfb80c9690 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00098f0 tx=0x7fdfb00046c0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 shutdown_connections 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fdfa406c2b0 0x7fdfa406e770 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfb80c53f0 0x7fdfb80c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb8154a00 0x7fdfb80c9690 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 --2- 172.21.15.2:0/3669745859 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfb806d730 0x7fdfb8153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.745 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 >> 172.21.15.2:0/3669745859 conn(0x7fdfb8067950 msgr2=0x7fdfb81e4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 shutdown_connections 2024-09-19T05:46:13.746 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.731+0000 7fdfbf3a2640 1 -- 172.21.15.2:0/3669745859 wait complete. 2024-09-19T05:46:13.746 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-19T05:46:11.867584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8560990,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":121,"num_read_kb":105,"num_write":13285,"num_write_kb":25179742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3145882543,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15290,"ondisk_log_size":15290,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":342232,"kb_used_data":21976,"kb_used_omap":18,"kb_used_meta":320109,"kb_avail":754632488,"statfs":{"total":773094113280,"available":772743667712,"internally_reserved":0,"allocated":22503424,"data_stored":21604268,"data_compressed":482085,"data_compressed_allocated":561152,"data_compressed_original":4919296,"omap_allocated":19176,"internal_metadata":327791896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003392"},"pg_stats":[{"pgid":"3.3f","version":"102'167","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.917070+0000","last_change":"2024-09-19T05:45:37.917099+0000","last_active":"2024-09-19T05:45:37.917070+0000","last_peered":"2024-09-19T05:45:37.917070+0000","last_clean":"2024-09-19T05:45:37.917070+0000","last_became_active":"2024-09-19T05:42:32.946886+0000","last_became_peered":"2024-09-19T05:42:32.946886+0000","last_unstale":"2024-09-19T05:45:37.917070+0000","last_undegraded":"2024-09-19T05:45:37.917070+0000","last_fullsized":"2024-09-19T05:45:37.917070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'167","last_scrub_stamp":"2024-09-19T05:45:37.916981+0000","last_deep_scrub":"102'167","last_deep_scrub_stamp":"2024-09-19T05:45:37.916981+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.916981+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:44.030022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"102'152","reported_seq":314,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.515982+0000","last_change":"2024-09-19T05:45:38.516031+0000","last_active":"2024-09-19T05:45:38.515982+0000","last_peered":"2024-09-19T05:45:38.515982+0000","last_clean":"2024-09-19T05:45:38.515982+0000","last_became_active":"2024-09-19T05:44:53.063354+0000","last_became_peered":"2024-09-19T05:44:53.063354+0000","last_unstale":"2024-09-19T05:45:38.515982+0000","last_undegraded":"2024-09-19T05:45:38.515982+0000","last_fullsized":"2024-09-19T05:45:38.515982+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'152","last_scrub_stamp":"2024-09-19T05:45:38.515863+0000","last_deep_scrub":"102'152","last_deep_scrub_stamp":"2024-09-19T05:45:38.515863+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.515863+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:16.939062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"102'178","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.549899+0000","last_change":"2024-09-19T05:45:36.549934+0000","last_active":"2024-09-19T05:45:36.549899+0000","last_peered":"2024-09-19T05:45:36.549899+0000","last_clean":"2024-09-19T05:45:36.549899+0000","last_became_active":"2024-09-19T05:44:53.058580+0000","last_became_peered":"2024-09-19T05:44:53.058580+0000","last_unstale":"2024-09-19T05:45:36.549899+0000","last_undegraded":"2024-09-19T05:45:36.549899+0000","last_fullsized":"2024-09-19T05:45:36.549899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'178","last_scrub_stamp":"2024-09-19T05:45:36.549818+0000","last_deep_scrub":"102'178","last_deep_scrub_stamp":"2024-09-19T05:45:36.549818+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.549818+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:53:59.941418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"102'193","reported_seq":374,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.100279+0000","last_change":"2024-09-19T05:45:36.100312+0000","last_active":"2024-09-19T05:45:36.100279+0000","last_peered":"2024-09-19T05:45:36.100279+0000","last_clean":"2024-09-19T05:45:36.100279+0000","last_became_active":"2024-09-19T05:42:32.936963+0000","last_became_peered":"2024-09-19T05:42:32.936963+0000","last_unstale":"2024-09-19T05:45:36.100279+0000","last_undegraded":"2024-09-19T05:45:36.100279+0000","last_fullsized":"2024-09-19T05:45:36.100279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'193","last_scrub_stamp":"2024-09-19T05:45:36.100183+0000","last_deep_scrub":"102'193","last_deep_scrub_stamp":"2024-09-19T05:45:36.100183+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.100183+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:17:22.471667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"102'174","reported_seq":451,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:44.889221+0000","last_change":"2024-09-19T05:45:44.889249+0000","last_active":"2024-09-19T05:45:44.889221+0000","last_peered":"2024-09-19T05:45:44.889221+0000","last_clean":"2024-09-19T05:45:44.889221+0000","last_became_active":"2024-09-19T05:44:53.067461+0000","last_became_peered":"2024-09-19T05:44:53.067461+0000","last_unstale":"2024-09-19T05:45:44.889221+0000","last_undegraded":"2024-09-19T05:45:44.889221+0000","last_fullsized":"2024-09-19T05:45:44.889221+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'174","last_scrub_stamp":"2024-09-19T05:45:44.889134+0000","last_deep_scrub":"102'174","last_deep_scrub_stamp":"2024-09-19T05:45:44.889134+0000","last_clean_scrub_stamp":"2024-09-19T05:45:44.889134+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:40.958698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"102'162","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.057061+0000","last_change":"2024-09-19T05:45:35.057110+0000","last_active":"2024-09-19T05:45:35.057061+0000","last_peered":"2024-09-19T05:45:35.057061+0000","last_clean":"2024-09-19T05:45:35.057061+0000","last_became_active":"2024-09-19T05:42:32.936109+0000","last_became_peered":"2024-09-19T05:42:32.936109+0000","last_unstale":"2024-09-19T05:45:35.057061+0000","last_undegraded":"2024-09-19T05:45:35.057061+0000","last_fullsized":"2024-09-19T05:45:35.057061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:35.056955+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:35.056955+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.056955+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:28.992568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"102'188","reported_seq":369,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.909755+0000","last_change":"2024-09-19T05:45:35.909784+0000","last_active":"2024-09-19T05:45:35.909755+0000","last_peered":"2024-09-19T05:45:35.909755+0000","last_clean":"2024-09-19T05:45:35.909755+0000","last_became_active":"2024-09-19T05:42:32.939255+0000","last_became_peered":"2024-09-19T05:42:32.939255+0000","last_unstale":"2024-09-19T05:45:35.909755+0000","last_undegraded":"2024-09-19T05:45:35.909755+0000","last_fullsized":"2024-09-19T05:45:35.909755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'188","last_scrub_stamp":"2024-09-19T05:45:35.909668+0000","last_deep_scrub":"102'188","last_deep_scrub_stamp":"2024-09-19T05:45:35.909668+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.909668+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:50:39.017237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"102'204","reported_seq":375,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.574474+0000","last_change":"2024-09-19T05:45:42.574500+0000","last_active":"2024-09-19T05:45:42.574474+0000","last_peered":"2024-09-19T05:45:42.574474+0000","last_clean":"2024-09-19T05:45:42.574474+0000","last_became_active":"2024-09-19T05:44:56.566802+0000","last_became_peered":"2024-09-19T05:44:56.566802+0000","last_unstale":"2024-09-19T05:45:42.574474+0000","last_undegraded":"2024-09-19T05:45:42.574474+0000","last_fullsized":"2024-09-19T05:45:42.574474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'204","last_scrub_stamp":"2024-09-19T05:45:42.574395+0000","last_deep_scrub":"102'204","last_deep_scrub_stamp":"2024-09-19T05:45:42.574395+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.574395+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:00.568585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":483328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":213909504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"102'178","reported_seq":301,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.006939+0000","last_change":"2024-09-19T05:45:36.006939+0000","last_active":"2024-09-19T05:45:36.006939+0000","last_peered":"2024-09-19T05:45:36.006939+0000","last_clean":"2024-09-19T05:45:36.006939+0000","last_became_active":"2024-09-19T05:45:10.626067+0000","last_became_peered":"2024-09-19T05:45:10.626067+0000","last_unstale":"2024-09-19T05:45:36.006939+0000","last_undegraded":"2024-09-19T05:45:36.006939+0000","last_fullsized":"2024-09-19T05:45:36.006939+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'178","last_scrub_stamp":"2024-09-19T05:45:36.006832+0000","last_deep_scrub":"102'178","last_deep_scrub_stamp":"2024-09-19T05:45:36.006832+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.006832+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:04.915216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"102'190","reported_seq":371,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.853419+0000","last_change":"2024-09-19T05:45:33.853445+0000","last_active":"2024-09-19T05:45:33.853419+0000","last_peered":"2024-09-19T05:45:33.853419+0000","last_clean":"2024-09-19T05:45:33.853419+0000","last_became_active":"2024-09-19T05:42:32.934287+0000","last_became_peered":"2024-09-19T05:42:32.934287+0000","last_unstale":"2024-09-19T05:45:33.853419+0000","last_undegraded":"2024-09-19T05:45:33.853419+0000","last_fullsized":"2024-09-19T05:45:33.853419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'190","last_scrub_stamp":"2024-09-19T05:45:33.853325+0000","last_deep_scrub":"102'190","last_deep_scrub_stamp":"2024-09-19T05:45:33.853325+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.853325+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:16.203364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"102'214","reported_seq":395,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.912842+0000","last_change":"2024-09-19T05:45:36.912868+0000","last_active":"2024-09-19T05:45:36.912842+0000","last_peered":"2024-09-19T05:45:36.912842+0000","last_clean":"2024-09-19T05:45:36.912842+0000","last_became_active":"2024-09-19T05:42:32.945885+0000","last_became_peered":"2024-09-19T05:42:32.945885+0000","last_unstale":"2024-09-19T05:45:36.912842+0000","last_undegraded":"2024-09-19T05:45:36.912842+0000","last_fullsized":"2024-09-19T05:45:36.912842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'214","last_scrub_stamp":"2024-09-19T05:45:36.912751+0000","last_deep_scrub":"102'214","last_deep_scrub_stamp":"2024-09-19T05:45:36.912751+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.912751+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:58.232479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"102'166","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.928695+0000","last_change":"2024-09-19T05:45:40.928748+0000","last_active":"2024-09-19T05:45:40.928695+0000","last_peered":"2024-09-19T05:45:40.928695+0000","last_clean":"2024-09-19T05:45:40.928695+0000","last_became_active":"2024-09-19T05:43:19.551069+0000","last_became_peered":"2024-09-19T05:43:19.551069+0000","last_unstale":"2024-09-19T05:45:40.928695+0000","last_undegraded":"2024-09-19T05:45:40.928695+0000","last_fullsized":"2024-09-19T05:45:40.928695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'166","last_scrub_stamp":"2024-09-19T05:45:40.928588+0000","last_deep_scrub":"102'166","last_deep_scrub_stamp":"2024-09-19T05:45:40.928588+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.928588+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:53.325578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"102'177","reported_seq":358,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.216910+0000","last_change":"2024-09-19T05:45:39.216947+0000","last_active":"2024-09-19T05:45:39.216910+0000","last_peered":"2024-09-19T05:45:39.216910+0000","last_clean":"2024-09-19T05:45:39.216910+0000","last_became_active":"2024-09-19T05:42:32.942950+0000","last_became_peered":"2024-09-19T05:42:32.942950+0000","last_unstale":"2024-09-19T05:45:39.216910+0000","last_undegraded":"2024-09-19T05:45:39.216910+0000","last_fullsized":"2024-09-19T05:45:39.216910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:39.216812+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:39.216812+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.216812+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:51.833446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"102'172","reported_seq":353,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.224580+0000","last_change":"2024-09-19T05:45:36.224630+0000","last_active":"2024-09-19T05:45:36.224580+0000","last_peered":"2024-09-19T05:45:36.224580+0000","last_clean":"2024-09-19T05:45:36.224580+0000","last_became_active":"2024-09-19T05:42:32.933105+0000","last_became_peered":"2024-09-19T05:42:32.933105+0000","last_unstale":"2024-09-19T05:45:36.224580+0000","last_undegraded":"2024-09-19T05:45:36.224580+0000","last_fullsized":"2024-09-19T05:45:36.224580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'172","last_scrub_stamp":"2024-09-19T05:45:36.224485+0000","last_deep_scrub":"102'172","last_deep_scrub_stamp":"2024-09-19T05:45:36.224485+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.224485+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:36.245176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"102'177","reported_seq":335,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.510306+0000","last_change":"2024-09-19T05:45:37.510332+0000","last_active":"2024-09-19T05:45:37.510306+0000","last_peered":"2024-09-19T05:45:37.510306+0000","last_clean":"2024-09-19T05:45:37.510306+0000","last_became_active":"2024-09-19T05:44:53.058466+0000","last_became_peered":"2024-09-19T05:44:53.058466+0000","last_unstale":"2024-09-19T05:45:37.510306+0000","last_undegraded":"2024-09-19T05:45:37.510306+0000","last_fullsized":"2024-09-19T05:45:37.510306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:37.510190+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:37.510190+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.510190+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:10.310827+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":171966464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"102'162","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.799878+0000","last_change":"2024-09-19T05:45:33.799902+0000","last_active":"2024-09-19T05:45:33.799878+0000","last_peered":"2024-09-19T05:45:33.799878+0000","last_clean":"2024-09-19T05:45:33.799878+0000","last_became_active":"2024-09-19T05:42:32.939304+0000","last_became_peered":"2024-09-19T05:42:32.939304+0000","last_unstale":"2024-09-19T05:45:33.799878+0000","last_undegraded":"2024-09-19T05:45:33.799878+0000","last_fullsized":"2024-09-19T05:45:33.799878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:33.799773+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:33.799773+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.799773+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:33.090240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"102'176","reported_seq":356,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.816760+0000","last_change":"2024-09-19T05:45:32.816783+0000","last_active":"2024-09-19T05:45:32.816760+0000","last_peered":"2024-09-19T05:45:32.816760+0000","last_clean":"2024-09-19T05:45:32.816760+0000","last_became_active":"2024-09-19T05:42:32.941781+0000","last_became_peered":"2024-09-19T05:42:32.941781+0000","last_unstale":"2024-09-19T05:45:32.816760+0000","last_undegraded":"2024-09-19T05:45:32.816760+0000","last_fullsized":"2024-09-19T05:45:32.816760+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'176","last_scrub_stamp":"2024-09-19T05:45:32.816697+0000","last_deep_scrub":"102'176","last_deep_scrub_stamp":"2024-09-19T05:45:32.816697+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.816697+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:21.013902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"102'179","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.267287+0000","last_change":"2024-09-19T05:45:36.267312+0000","last_active":"2024-09-19T05:45:36.267287+0000","last_peered":"2024-09-19T05:45:36.267287+0000","last_clean":"2024-09-19T05:45:36.267287+0000","last_became_active":"2024-09-19T05:42:32.932723+0000","last_became_peered":"2024-09-19T05:42:32.932723+0000","last_unstale":"2024-09-19T05:45:36.267287+0000","last_undegraded":"2024-09-19T05:45:36.267287+0000","last_fullsized":"2024-09-19T05:45:36.267287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'179","last_scrub_stamp":"2024-09-19T05:45:36.267187+0000","last_deep_scrub":"102'179","last_deep_scrub_stamp":"2024-09-19T05:45:36.267187+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.267187+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:04.121186+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"102'205","reported_seq":385,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.802247+0000","last_change":"2024-09-19T05:45:34.802286+0000","last_active":"2024-09-19T05:45:34.802247+0000","last_peered":"2024-09-19T05:45:34.802247+0000","last_clean":"2024-09-19T05:45:34.802247+0000","last_became_active":"2024-09-19T05:42:32.942278+0000","last_became_peered":"2024-09-19T05:42:32.942278+0000","last_unstale":"2024-09-19T05:45:34.802247+0000","last_undegraded":"2024-09-19T05:45:34.802247+0000","last_fullsized":"2024-09-19T05:45:34.802247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'205","last_scrub_stamp":"2024-09-19T05:45:34.802159+0000","last_deep_scrub":"102'205","last_deep_scrub_stamp":"2024-09-19T05:45:34.802159+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.802159+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:01.143189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":165,"num_write_kb":450560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"102'179","reported_seq":370,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.888232+0000","last_change":"2024-09-19T05:45:42.888261+0000","last_active":"2024-09-19T05:45:42.888232+0000","last_peered":"2024-09-19T05:45:42.888232+0000","last_clean":"2024-09-19T05:45:42.888232+0000","last_became_active":"2024-09-19T05:43:19.551487+0000","last_became_peered":"2024-09-19T05:43:19.551487+0000","last_unstale":"2024-09-19T05:45:42.888232+0000","last_undegraded":"2024-09-19T05:45:42.888232+0000","last_fullsized":"2024-09-19T05:45:42.888232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'179","last_scrub_stamp":"2024-09-19T05:45:42.888140+0000","last_deep_scrub":"102'179","last_deep_scrub_stamp":"2024-09-19T05:45:42.888140+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.888140+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:31.311294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":153,"num_write_kb":417792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"102'154","reported_seq":335,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.795730+0000","last_change":"2024-09-19T05:45:38.795758+0000","last_active":"2024-09-19T05:45:38.795730+0000","last_peered":"2024-09-19T05:45:38.795730+0000","last_clean":"2024-09-19T05:45:38.795730+0000","last_became_active":"2024-09-19T05:42:32.932343+0000","last_became_peered":"2024-09-19T05:42:32.932343+0000","last_unstale":"2024-09-19T05:45:38.795730+0000","last_undegraded":"2024-09-19T05:45:38.795730+0000","last_fullsized":"2024-09-19T05:45:38.795730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'154","last_scrub_stamp":"2024-09-19T05:45:38.795632+0000","last_deep_scrub":"102'154","last_deep_scrub_stamp":"2024-09-19T05:45:38.795632+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.795632+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:18.826189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"102'199","reported_seq":265,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.502634+0000","last_change":"2024-09-19T05:45:40.502660+0000","last_active":"2024-09-19T05:45:40.502634+0000","last_peered":"2024-09-19T05:45:40.502634+0000","last_clean":"2024-09-19T05:45:40.502634+0000","last_became_active":"2024-09-19T05:44:53.063155+0000","last_became_peered":"2024-09-19T05:44:53.063155+0000","last_unstale":"2024-09-19T05:45:40.502634+0000","last_undegraded":"2024-09-19T05:45:40.502634+0000","last_fullsized":"2024-09-19T05:45:40.502634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'199","last_scrub_stamp":"2024-09-19T05:45:40.502514+0000","last_deep_scrub":"102'199","last_deep_scrub_stamp":"2024-09-19T05:45:40.502514+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.502514+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:19.991273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":180355072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"102'150","reported_seq":312,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.542756+0000","last_change":"2024-09-19T05:45:41.542781+0000","last_active":"2024-09-19T05:45:41.542756+0000","last_peered":"2024-09-19T05:45:41.542756+0000","last_clean":"2024-09-19T05:45:41.542756+0000","last_became_active":"2024-09-19T05:44:53.062258+0000","last_became_peered":"2024-09-19T05:44:53.062258+0000","last_unstale":"2024-09-19T05:45:41.542756+0000","last_undegraded":"2024-09-19T05:45:41.542756+0000","last_fullsized":"2024-09-19T05:45:41.542756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'150","last_scrub_stamp":"2024-09-19T05:45:41.542638+0000","last_deep_scrub":"102'150","last_deep_scrub_stamp":"2024-09-19T05:45:41.542638+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.542638+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:58.446555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":150994944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"102'207","reported_seq":507,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:51.679171+0000","last_change":"2024-09-19T05:45:51.679198+0000","last_active":"2024-09-19T05:45:51.679171+0000","last_peered":"2024-09-19T05:45:51.679171+0000","last_clean":"2024-09-19T05:45:51.679171+0000","last_became_active":"2024-09-19T05:44:56.565050+0000","last_became_peered":"2024-09-19T05:44:56.565050+0000","last_unstale":"2024-09-19T05:45:51.679171+0000","last_undegraded":"2024-09-19T05:45:51.679171+0000","last_fullsized":"2024-09-19T05:45:51.679171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'207","last_scrub_stamp":"2024-09-19T05:45:51.679080+0000","last_deep_scrub":"102'207","last_deep_scrub_stamp":"2024-09-19T05:45:51.679080+0000","last_clean_scrub_stamp":"2024-09-19T05:45:51.679080+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:26.062279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":174,"num_write_kb":475136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":205520896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"102'160","reported_seq":330,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:43.611748+0000","last_change":"2024-09-19T05:45:43.611774+0000","last_active":"2024-09-19T05:45:43.611748+0000","last_peered":"2024-09-19T05:45:43.611748+0000","last_clean":"2024-09-19T05:45:43.611748+0000","last_became_active":"2024-09-19T05:44:53.062357+0000","last_became_peered":"2024-09-19T05:44:53.062357+0000","last_unstale":"2024-09-19T05:45:43.611748+0000","last_undegraded":"2024-09-19T05:45:43.611748+0000","last_fullsized":"2024-09-19T05:45:43.611748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:43.611661+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:43.611661+0000","last_clean_scrub_stamp":"2024-09-19T05:45:43.611661+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:24.431838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"102'217","reported_seq":398,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.096956+0000","last_change":"2024-09-19T05:45:38.096985+0000","last_active":"2024-09-19T05:45:38.096956+0000","last_peered":"2024-09-19T05:45:38.096956+0000","last_clean":"2024-09-19T05:45:38.096956+0000","last_became_active":"2024-09-19T05:42:32.943858+0000","last_became_peered":"2024-09-19T05:42:32.943858+0000","last_unstale":"2024-09-19T05:45:38.096956+0000","last_undegraded":"2024-09-19T05:45:38.096956+0000","last_fullsized":"2024-09-19T05:45:38.096956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'217","last_scrub_stamp":"2024-09-19T05:45:38.096867+0000","last_deep_scrub":"102'217","last_deep_scrub_stamp":"2024-09-19T05:45:38.096867+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.096867+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:08.940001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"102'166","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.214314+0000","last_change":"2024-09-19T05:45:35.214339+0000","last_active":"2024-09-19T05:45:35.214314+0000","last_peered":"2024-09-19T05:45:35.214314+0000","last_clean":"2024-09-19T05:45:35.214314+0000","last_became_active":"2024-09-19T05:42:32.932950+0000","last_became_peered":"2024-09-19T05:42:32.932950+0000","last_unstale":"2024-09-19T05:45:35.214314+0000","last_undegraded":"2024-09-19T05:45:35.214314+0000","last_fullsized":"2024-09-19T05:45:35.214314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'166","last_scrub_stamp":"2024-09-19T05:45:35.214225+0000","last_deep_scrub":"102'166","last_deep_scrub_stamp":"2024-09-19T05:45:35.214225+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.214225+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:53.696897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"102'152","reported_seq":333,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.180427+0000","last_change":"2024-09-19T05:45:40.180487+0000","last_active":"2024-09-19T05:45:40.180427+0000","last_peered":"2024-09-19T05:45:40.180427+0000","last_clean":"2024-09-19T05:45:40.180427+0000","last_became_active":"2024-09-19T05:42:32.942999+0000","last_became_peered":"2024-09-19T05:42:32.942999+0000","last_unstale":"2024-09-19T05:45:40.180427+0000","last_undegraded":"2024-09-19T05:45:40.180427+0000","last_fullsized":"2024-09-19T05:45:40.180427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'152","last_scrub_stamp":"2024-09-19T05:45:40.180307+0000","last_deep_scrub":"102'152","last_deep_scrub_stamp":"2024-09-19T05:45:40.180307+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.180307+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:47.582255+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"102'134","reported_seq":297,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.534857+0000","last_change":"2024-09-19T05:45:39.534898+0000","last_active":"2024-09-19T05:45:39.534857+0000","last_peered":"2024-09-19T05:45:39.534857+0000","last_clean":"2024-09-19T05:45:39.534857+0000","last_became_active":"2024-09-19T05:44:53.062199+0000","last_became_peered":"2024-09-19T05:44:53.062199+0000","last_unstale":"2024-09-19T05:45:39.534857+0000","last_undegraded":"2024-09-19T05:45:39.534857+0000","last_fullsized":"2024-09-19T05:45:39.534857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'134","last_scrub_stamp":"2024-09-19T05:45:39.534769+0000","last_deep_scrub":"102'134","last_deep_scrub_stamp":"2024-09-19T05:45:39.534769+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.534769+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:19.274773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":303104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":117440512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"102'153","reported_seq":425,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:50.667406+0000","last_change":"2024-09-19T05:45:50.667436+0000","last_active":"2024-09-19T05:45:50.667406+0000","last_peered":"2024-09-19T05:45:50.667406+0000","last_clean":"2024-09-19T05:45:50.667406+0000","last_became_active":"2024-09-19T05:44:53.065817+0000","last_became_peered":"2024-09-19T05:44:53.065817+0000","last_unstale":"2024-09-19T05:45:50.667406+0000","last_undegraded":"2024-09-19T05:45:50.667406+0000","last_fullsized":"2024-09-19T05:45:50.667406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'153","last_scrub_stamp":"2024-09-19T05:45:50.667317+0000","last_deep_scrub":"102'153","last_deep_scrub_stamp":"2024-09-19T05:45:50.667317+0000","last_clean_scrub_stamp":"2024-09-19T05:45:50.667317+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:21.299507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"102'158","reported_seq":350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.267518+0000","last_change":"2024-09-19T05:45:37.267546+0000","last_active":"2024-09-19T05:45:37.267518+0000","last_peered":"2024-09-19T05:45:37.267518+0000","last_clean":"2024-09-19T05:45:37.267518+0000","last_became_active":"2024-09-19T05:43:19.548390+0000","last_became_peered":"2024-09-19T05:43:19.548390+0000","last_unstale":"2024-09-19T05:45:37.267518+0000","last_undegraded":"2024-09-19T05:45:37.267518+0000","last_fullsized":"2024-09-19T05:45:37.267518+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'158","last_scrub_stamp":"2024-09-19T05:45:37.267434+0000","last_deep_scrub":"102'158","last_deep_scrub_stamp":"2024-09-19T05:45:37.267434+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.267434+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:55.750105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"102'180","reported_seq":361,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.057912+0000","last_change":"2024-09-19T05:45:33.057952+0000","last_active":"2024-09-19T05:45:33.057912+0000","last_peered":"2024-09-19T05:45:33.057912+0000","last_clean":"2024-09-19T05:45:33.057912+0000","last_became_active":"2024-09-19T05:42:32.934431+0000","last_became_peered":"2024-09-19T05:42:32.934431+0000","last_unstale":"2024-09-19T05:45:33.057912+0000","last_undegraded":"2024-09-19T05:45:33.057912+0000","last_fullsized":"2024-09-19T05:45:33.057912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'180","last_scrub_stamp":"2024-09-19T05:45:33.057837+0000","last_deep_scrub":"102'180","last_deep_scrub_stamp":"2024-09-19T05:45:33.057837+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.057837+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:27.994419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"102'149","reported_seq":257,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.974444+0000","last_change":"2024-09-19T05:45:31.974444+0000","last_active":"2024-09-19T05:45:31.974444+0000","last_peered":"2024-09-19T05:45:31.974444+0000","last_clean":"2024-09-19T05:45:31.974444+0000","last_became_active":"2024-09-19T05:45:10.624294+0000","last_became_peered":"2024-09-19T05:45:10.624294+0000","last_unstale":"2024-09-19T05:45:31.974444+0000","last_undegraded":"2024-09-19T05:45:31.974444+0000","last_fullsized":"2024-09-19T05:45:31.974444+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'149","last_scrub_stamp":"2024-09-19T05:45:31.974354+0000","last_deep_scrub":"102'149","last_deep_scrub_stamp":"2024-09-19T05:45:31.974354+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.974354+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:31.995046+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"102'1811","reported_seq":1896,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.534786+0000","last_change":"2024-09-19T05:45:32.534812+0000","last_active":"2024-09-19T05:45:32.534786+0000","last_peered":"2024-09-19T05:45:32.534786+0000","last_clean":"2024-09-19T05:45:32.534786+0000","last_became_active":"2024-09-19T05:44:55.559554+0000","last_became_peered":"2024-09-19T05:44:55.559554+0000","last_unstale":"2024-09-19T05:45:32.534786+0000","last_undegraded":"2024-09-19T05:45:32.534786+0000","last_fullsized":"2024-09-19T05:45:32.534786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'1811","last_scrub_stamp":"2024-09-19T05:45:32.534693+0000","last_deep_scrub":"102'1811","last_deep_scrub_stamp":"2024-09-19T05:45:32.534693+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.534693+0000","objects_scrubbed":1,"log_size":1811,"log_dups_size":0,"ondisk_log_size":1811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:52.367253+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3774343,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":5432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":3758308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"102'162","reported_seq":343,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.830449+0000","last_change":"2024-09-19T05:45:40.830476+0000","last_active":"2024-09-19T05:45:40.830449+0000","last_peered":"2024-09-19T05:45:40.830449+0000","last_clean":"2024-09-19T05:45:40.830449+0000","last_became_active":"2024-09-19T05:42:32.935394+0000","last_became_peered":"2024-09-19T05:42:32.935394+0000","last_unstale":"2024-09-19T05:45:40.830449+0000","last_undegraded":"2024-09-19T05:45:40.830449+0000","last_fullsized":"2024-09-19T05:45:40.830449+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:40.830359+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:40.830359+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.830359+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:49.876265+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.777206+0000","last_change":"2024-09-19T05:45:41.777234+0000","last_active":"2024-09-19T05:45:41.777206+0000","last_peered":"2024-09-19T05:45:41.777206+0000","last_clean":"2024-09-19T05:45:41.777206+0000","last_became_active":"2024-09-19T05:42:31.994919+0000","last_became_peered":"2024-09-19T05:42:31.994919+0000","last_unstale":"2024-09-19T05:45:41.777206+0000","last_undegraded":"2024-09-19T05:45:41.777206+0000","last_fullsized":"2024-09-19T05:45:41.777206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-19T05:45:41.777118+0000","last_deep_scrub":"22'1","last_deep_scrub_stamp":"2024-09-19T05:45:41.777118+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.777118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:15.713480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.226659+0000","last_change":"2024-09-19T05:45:33.226723+0000","last_active":"2024-09-19T05:45:33.226659+0000","last_peered":"2024-09-19T05:45:33.226659+0000","last_clean":"2024-09-19T05:45:33.226659+0000","last_became_active":"2024-09-19T05:42:31.991361+0000","last_became_peered":"2024-09-19T05:42:31.991361+0000","last_unstale":"2024-09-19T05:45:33.226659+0000","last_undegraded":"2024-09-19T05:45:33.226659+0000","last_fullsized":"2024-09-19T05:45:33.226659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:33.226532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:33.226532+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.226532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:24.349722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"102'190","reported_seq":354,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:48.660050+0000","last_change":"2024-09-19T05:45:48.660106+0000","last_active":"2024-09-19T05:45:48.660050+0000","last_peered":"2024-09-19T05:45:48.660050+0000","last_clean":"2024-09-19T05:45:48.660050+0000","last_became_active":"2024-09-19T05:44:53.069354+0000","last_became_peered":"2024-09-19T05:44:53.069354+0000","last_unstale":"2024-09-19T05:45:48.660050+0000","last_undegraded":"2024-09-19T05:45:48.660050+0000","last_fullsized":"2024-09-19T05:45:48.660050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'190","last_scrub_stamp":"2024-09-19T05:45:48.659940+0000","last_deep_scrub":"102'190","last_deep_scrub_stamp":"2024-09-19T05:45:48.659940+0000","last_clean_scrub_stamp":"2024-09-19T05:45:48.659940+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:20.259036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":162,"num_write_kb":442368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":197132288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":192,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:48.667187+0000","last_change":"2024-09-19T05:45:48.667213+0000","last_active":"2024-09-19T05:45:48.667187+0000","last_peered":"2024-09-19T05:45:48.667187+0000","last_clean":"2024-09-19T05:45:48.667187+0000","last_became_active":"2024-09-19T05:43:19.543435+0000","last_became_peered":"2024-09-19T05:43:19.543435+0000","last_unstale":"2024-09-19T05:45:48.667187+0000","last_undegraded":"2024-09-19T05:45:48.667187+0000","last_fullsized":"2024-09-19T05:45:48.667187+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:48.667092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:48.667092+0000","last_clean_scrub_stamp":"2024-09-19T05:45:48.667092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:03.431257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"102'187","reported_seq":378,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.803876+0000","last_change":"2024-09-19T05:45:41.803962+0000","last_active":"2024-09-19T05:45:41.803876+0000","last_peered":"2024-09-19T05:45:41.803876+0000","last_clean":"2024-09-19T05:45:41.803876+0000","last_became_active":"2024-09-19T05:43:19.549365+0000","last_became_peered":"2024-09-19T05:43:19.549365+0000","last_unstale":"2024-09-19T05:45:41.803876+0000","last_undegraded":"2024-09-19T05:45:41.803876+0000","last_fullsized":"2024-09-19T05:45:41.803876+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'187","last_scrub_stamp":"2024-09-19T05:45:41.803751+0000","last_deep_scrub":"102'187","last_deep_scrub_stamp":"2024-09-19T05:45:41.803751+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.803751+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:30.000956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"59'1","reported_seq":87,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:30.937711+0000","last_change":"2024-09-19T05:45:30.937711+0000","last_active":"2024-09-19T05:45:30.937711+0000","last_peered":"2024-09-19T05:45:30.937711+0000","last_clean":"2024-09-19T05:45:30.937711+0000","last_became_active":"2024-09-19T05:45:10.622631+0000","last_became_peered":"2024-09-19T05:45:10.622631+0000","last_unstale":"2024-09-19T05:45:30.937711+0000","last_undegraded":"2024-09-19T05:45:30.937711+0000","last_fullsized":"2024-09-19T05:45:30.937711+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-19T05:45:30.937601+0000","last_deep_scrub":"59'1","last_deep_scrub_stamp":"2024-09-19T05:45:30.937601+0000","last_clean_scrub_stamp":"2024-09-19T05:45:30.937601+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:49.420272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"102'251","reported_seq":432,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.107629+0000","last_change":"2024-09-19T05:45:37.107679+0000","last_active":"2024-09-19T05:45:37.107629+0000","last_peered":"2024-09-19T05:45:37.107629+0000","last_clean":"2024-09-19T05:45:37.107629+0000","last_became_active":"2024-09-19T05:42:32.943465+0000","last_became_peered":"2024-09-19T05:42:32.943465+0000","last_unstale":"2024-09-19T05:45:37.107629+0000","last_undegraded":"2024-09-19T05:45:37.107629+0000","last_fullsized":"2024-09-19T05:45:37.107629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'251","last_scrub_stamp":"2024-09-19T05:45:37.107526+0000","last_deep_scrub":"102'251","last_deep_scrub_stamp":"2024-09-19T05:45:37.107526+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.107526+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:14.809070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":219,"num_write_kb":598016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":193,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.947047+0000","last_change":"2024-09-19T05:45:38.947090+0000","last_active":"2024-09-19T05:45:38.947047+0000","last_peered":"2024-09-19T05:45:38.947047+0000","last_clean":"2024-09-19T05:45:38.947047+0000","last_became_active":"2024-09-19T05:43:19.551842+0000","last_became_peered":"2024-09-19T05:43:19.551842+0000","last_unstale":"2024-09-19T05:45:38.947047+0000","last_undegraded":"2024-09-19T05:45:38.947047+0000","last_fullsized":"2024-09-19T05:45:38.947047+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:38.946945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:38.946945+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.946945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:52.461934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"102'160","reported_seq":318,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:47.628673+0000","last_change":"2024-09-19T05:45:47.628700+0000","last_active":"2024-09-19T05:45:47.628673+0000","last_peered":"2024-09-19T05:45:47.628673+0000","last_clean":"2024-09-19T05:45:47.628673+0000","last_became_active":"2024-09-19T05:44:53.067607+0000","last_became_peered":"2024-09-19T05:44:53.067607+0000","last_unstale":"2024-09-19T05:45:47.628673+0000","last_undegraded":"2024-09-19T05:45:47.628673+0000","last_fullsized":"2024-09-19T05:45:47.628673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:47.628584+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:47.628584+0000","last_clean_scrub_stamp":"2024-09-19T05:45:47.628584+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:49.123820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":146800640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"102'11","reported_seq":195,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.307726+0000","last_change":"2024-09-19T05:45:33.307757+0000","last_active":"2024-09-19T05:45:33.307726+0000","last_peered":"2024-09-19T05:45:33.307726+0000","last_clean":"2024-09-19T05:45:33.307726+0000","last_became_active":"2024-09-19T05:42:31.985475+0000","last_became_peered":"2024-09-19T05:42:31.985475+0000","last_unstale":"2024-09-19T05:45:33.307726+0000","last_undegraded":"2024-09-19T05:45:33.307726+0000","last_fullsized":"2024-09-19T05:45:33.307726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'11","last_scrub_stamp":"2024-09-19T05:45:33.307632+0000","last_deep_scrub":"102'11","last_deep_scrub_stamp":"2024-09-19T05:45:33.307632+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.307632+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:53.716103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'168","reported_seq":348,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.767899+0000","last_change":"2024-09-19T05:45:35.767930+0000","last_active":"2024-09-19T05:45:35.767899+0000","last_peered":"2024-09-19T05:45:35.767899+0000","last_clean":"2024-09-19T05:45:35.767899+0000","last_became_active":"2024-09-19T05:42:32.939242+0000","last_became_peered":"2024-09-19T05:42:32.939242+0000","last_unstale":"2024-09-19T05:45:35.767899+0000","last_undegraded":"2024-09-19T05:45:35.767899+0000","last_fullsized":"2024-09-19T05:45:35.767899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'168","last_scrub_stamp":"2024-09-19T05:45:35.767811+0000","last_deep_scrub":"102'168","last_deep_scrub_stamp":"2024-09-19T05:45:35.767811+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.767811+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:33.355183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"102'2020","reported_seq":2203,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.022036+0000","last_change":"2024-09-19T05:45:31.022089+0000","last_active":"2024-09-19T05:45:31.022036+0000","last_peered":"2024-09-19T05:45:31.022036+0000","last_clean":"2024-09-19T05:45:31.022036+0000","last_became_active":"2024-09-19T05:42:31.988121+0000","last_became_peered":"2024-09-19T05:42:31.988121+0000","last_unstale":"2024-09-19T05:45:31.022036+0000","last_undegraded":"2024-09-19T05:45:31.022036+0000","last_fullsized":"2024-09-19T05:45:31.022036+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'2020","last_scrub_stamp":"2024-09-19T05:45:31.021952+0000","last_deep_scrub":"102'2020","last_deep_scrub_stamp":"2024-09-19T05:45:31.021952+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.021952+0000","objects_scrubbed":4,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:59.591416+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2016,"num_write_kb":6036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"102'176","reported_seq":357,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.845180+0000","last_change":"2024-09-19T05:45:32.845203+0000","last_active":"2024-09-19T05:45:32.845180+0000","last_peered":"2024-09-19T05:45:32.845180+0000","last_clean":"2024-09-19T05:45:32.845180+0000","last_became_active":"2024-09-19T05:42:32.933165+0000","last_became_peered":"2024-09-19T05:42:32.933165+0000","last_unstale":"2024-09-19T05:45:32.845180+0000","last_undegraded":"2024-09-19T05:45:32.845180+0000","last_fullsized":"2024-09-19T05:45:32.845180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'176","last_scrub_stamp":"2024-09-19T05:45:32.845113+0000","last_deep_scrub":"102'176","last_deep_scrub_stamp":"2024-09-19T05:45:32.845113+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.845113+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:30.432485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"102'4","reported_seq":188,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.273378+0000","last_change":"2024-09-19T05:45:34.273411+0000","last_active":"2024-09-19T05:45:34.273378+0000","last_peered":"2024-09-19T05:45:34.273378+0000","last_clean":"2024-09-19T05:45:34.273378+0000","last_became_active":"2024-09-19T05:42:31.989223+0000","last_became_peered":"2024-09-19T05:42:31.989223+0000","last_unstale":"2024-09-19T05:45:34.273378+0000","last_undegraded":"2024-09-19T05:45:34.273378+0000","last_fullsized":"2024-09-19T05:45:34.273378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'4","last_scrub_stamp":"2024-09-19T05:45:34.273259+0000","last_deep_scrub":"102'4","last_deep_scrub_stamp":"2024-09-19T05:45:34.273259+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.273259+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:23.993898+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"102'177","reported_seq":358,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.169310+0000","last_change":"2024-09-19T05:45:38.169360+0000","last_active":"2024-09-19T05:45:38.169310+0000","last_peered":"2024-09-19T05:45:38.169310+0000","last_clean":"2024-09-19T05:45:38.169310+0000","last_became_active":"2024-09-19T05:42:32.941968+0000","last_became_peered":"2024-09-19T05:42:32.941968+0000","last_unstale":"2024-09-19T05:45:38.169310+0000","last_undegraded":"2024-09-19T05:45:38.169310+0000","last_fullsized":"2024-09-19T05:45:38.169310+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:38.169212+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:38.169212+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.169212+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:49.384368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.751034+0000","last_change":"2024-09-19T05:45:36.751059+0000","last_active":"2024-09-19T05:45:36.751034+0000","last_peered":"2024-09-19T05:45:36.751034+0000","last_clean":"2024-09-19T05:45:36.751034+0000","last_became_active":"2024-09-19T05:42:31.987224+0000","last_became_peered":"2024-09-19T05:42:31.987224+0000","last_unstale":"2024-09-19T05:45:36.751034+0000","last_undegraded":"2024-09-19T05:45:36.751034+0000","last_fullsized":"2024-09-19T05:45:36.751034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:36.750946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:36.750946+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.750946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:19.100985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"102'201","reported_seq":381,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:43.760210+0000","last_change":"2024-09-19T05:45:43.760241+0000","last_active":"2024-09-19T05:45:43.760210+0000","last_peered":"2024-09-19T05:45:43.760210+0000","last_clean":"2024-09-19T05:45:43.760210+0000","last_became_active":"2024-09-19T05:42:32.933126+0000","last_became_peered":"2024-09-19T05:42:32.933126+0000","last_unstale":"2024-09-19T05:45:43.760210+0000","last_undegraded":"2024-09-19T05:45:43.760210+0000","last_fullsized":"2024-09-19T05:45:43.760210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'201","last_scrub_stamp":"2024-09-19T05:45:43.760115+0000","last_deep_scrub":"102'201","last_deep_scrub_stamp":"2024-09-19T05:45:43.760115+0000","last_clean_scrub_stamp":"2024-09-19T05:45:43.760115+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:47.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:30.749483+0000","last_change":"2024-09-19T05:45:30.749518+0000","last_active":"2024-09-19T05:45:30.749483+0000","last_peered":"2024-09-19T05:45:30.749483+0000","last_clean":"2024-09-19T05:45:30.749483+0000","last_became_active":"2024-09-19T05:42:31.995889+0000","last_became_peered":"2024-09-19T05:42:31.995889+0000","last_unstale":"2024-09-19T05:45:30.749483+0000","last_undegraded":"2024-09-19T05:45:30.749483+0000","last_fullsized":"2024-09-19T05:45:30.749483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:30.749367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:30.749367+0000","last_clean_scrub_stamp":"2024-09-19T05:45:30.749367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:01.968783+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"102'197","reported_seq":299,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.042227+0000","last_change":"2024-09-19T05:45:35.042227+0000","last_active":"2024-09-19T05:45:35.042227+0000","last_peered":"2024-09-19T05:45:35.042227+0000","last_clean":"2024-09-19T05:45:35.042227+0000","last_became_active":"2024-09-19T05:45:10.626877+0000","last_became_peered":"2024-09-19T05:45:10.626877+0000","last_unstale":"2024-09-19T05:45:35.042227+0000","last_undegraded":"2024-09-19T05:45:35.042227+0000","last_fullsized":"2024-09-19T05:45:35.042227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'197","last_scrub_stamp":"2024-09-19T05:45:35.042112+0000","last_deep_scrub":"102'197","last_deep_scrub_stamp":"2024-09-19T05:45:35.042112+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.042112+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:22:47.658746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":159,"num_write_kb":434176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":167772160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"102'13","reported_seq":196,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.667490+0000","last_change":"2024-09-19T05:45:31.667541+0000","last_active":"2024-09-19T05:45:31.667490+0000","last_peered":"2024-09-19T05:45:31.667490+0000","last_clean":"2024-09-19T05:45:31.667490+0000","last_became_active":"2024-09-19T05:42:31.984120+0000","last_became_peered":"2024-09-19T05:42:31.984120+0000","last_unstale":"2024-09-19T05:45:31.667490+0000","last_undegraded":"2024-09-19T05:45:31.667490+0000","last_fullsized":"2024-09-19T05:45:31.667490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'13","last_scrub_stamp":"2024-09-19T05:45:31.667407+0000","last_deep_scrub":"102'13","last_deep_scrub_stamp":"2024-09-19T05:45:31.667407+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.667407+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:51.674841+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"102'215","reported_seq":395,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:44.770414+0000","last_change":"2024-09-19T05:45:44.770442+0000","last_active":"2024-09-19T05:45:44.770414+0000","last_peered":"2024-09-19T05:45:44.770414+0000","last_clean":"2024-09-19T05:45:44.770414+0000","last_became_active":"2024-09-19T05:42:32.943798+0000","last_became_peered":"2024-09-19T05:42:32.943798+0000","last_unstale":"2024-09-19T05:45:44.770414+0000","last_undegraded":"2024-09-19T05:45:44.770414+0000","last_fullsized":"2024-09-19T05:45:44.770414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'215","last_scrub_stamp":"2024-09-19T05:45:44.770328+0000","last_deep_scrub":"102'215","last_deep_scrub_stamp":"2024-09-19T05:45:44.770328+0000","last_clean_scrub_stamp":"2024-09-19T05:45:44.770328+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:47.527164+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":183,"num_write_kb":499712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"102'2","reported_seq":192,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:43.847654+0000","last_change":"2024-09-19T05:45:43.847686+0000","last_active":"2024-09-19T05:45:43.847654+0000","last_peered":"2024-09-19T05:45:43.847654+0000","last_clean":"2024-09-19T05:45:43.847654+0000","last_became_active":"2024-09-19T05:44:53.065224+0000","last_became_peered":"2024-09-19T05:44:53.065224+0000","last_unstale":"2024-09-19T05:45:43.847654+0000","last_undegraded":"2024-09-19T05:45:43.847654+0000","last_fullsized":"2024-09-19T05:45:43.847654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'2","last_scrub_stamp":"2024-09-19T05:45:43.847574+0000","last_deep_scrub":"102'2","last_deep_scrub_stamp":"2024-09-19T05:45:43.847574+0000","last_clean_scrub_stamp":"2024-09-19T05:45:43.847574+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:09.787140+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"102'175","reported_seq":356,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.054766+0000","last_change":"2024-09-19T05:45:32.054813+0000","last_active":"2024-09-19T05:45:32.054766+0000","last_peered":"2024-09-19T05:45:32.054766+0000","last_clean":"2024-09-19T05:45:32.054766+0000","last_became_active":"2024-09-19T05:42:32.930975+0000","last_became_peered":"2024-09-19T05:42:32.930975+0000","last_unstale":"2024-09-19T05:45:32.054766+0000","last_undegraded":"2024-09-19T05:45:32.054766+0000","last_fullsized":"2024-09-19T05:45:32.054766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'175","last_scrub_stamp":"2024-09-19T05:45:32.054677+0000","last_deep_scrub":"102'175","last_deep_scrub_stamp":"2024-09-19T05:45:32.054677+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.054677+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:42.949312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":385024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"102'3","reported_seq":193,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.799399+0000","last_change":"2024-09-19T05:45:42.799425+0000","last_active":"2024-09-19T05:45:42.799399+0000","last_peered":"2024-09-19T05:45:42.799399+0000","last_clean":"2024-09-19T05:45:42.799399+0000","last_became_active":"2024-09-19T05:44:53.059439+0000","last_became_peered":"2024-09-19T05:44:53.059439+0000","last_unstale":"2024-09-19T05:45:42.799399+0000","last_undegraded":"2024-09-19T05:45:42.799399+0000","last_fullsized":"2024-09-19T05:45:42.799399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'3","last_scrub_stamp":"2024-09-19T05:45:42.799316+0000","last_deep_scrub":"102'3","last_deep_scrub_stamp":"2024-09-19T05:45:42.799316+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.799316+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:14.018732+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"102'162","reported_seq":249,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.268652+0000","last_change":"2024-09-19T05:45:38.268679+0000","last_active":"2024-09-19T05:45:38.268652+0000","last_peered":"2024-09-19T05:45:38.268652+0000","last_clean":"2024-09-19T05:45:38.268652+0000","last_became_active":"2024-09-19T05:44:15.758949+0000","last_became_peered":"2024-09-19T05:44:15.758949+0000","last_unstale":"2024-09-19T05:45:38.268652+0000","last_undegraded":"2024-09-19T05:45:38.268652+0000","last_fullsized":"2024-09-19T05:45:38.268652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:38.268565+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:38.268565+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.268565+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:38.289257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":8388608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"102'2","reported_seq":135,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.529324+0000","last_change":"2024-09-19T05:45:33.529353+0000","last_active":"2024-09-19T05:45:33.529324+0000","last_peered":"2024-09-19T05:45:33.529324+0000","last_clean":"2024-09-19T05:45:33.529324+0000","last_became_active":"2024-09-19T05:44:53.058591+0000","last_became_peered":"2024-09-19T05:44:53.058591+0000","last_unstale":"2024-09-19T05:45:33.529324+0000","last_undegraded":"2024-09-19T05:45:33.529324+0000","last_fullsized":"2024-09-19T05:45:33.529324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'2","last_scrub_stamp":"2024-09-19T05:45:33.529245+0000","last_deep_scrub":"102'2","last_deep_scrub_stamp":"2024-09-19T05:45:33.529245+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.529245+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:46.002935+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":70,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"102'165","reported_seq":346,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.798117+0000","last_change":"2024-09-19T05:45:31.798163+0000","last_active":"2024-09-19T05:45:31.798117+0000","last_peered":"2024-09-19T05:45:31.798117+0000","last_clean":"2024-09-19T05:45:31.798117+0000","last_became_active":"2024-09-19T05:42:32.933889+0000","last_became_peered":"2024-09-19T05:42:32.933889+0000","last_unstale":"2024-09-19T05:45:31.798117+0000","last_undegraded":"2024-09-19T05:45:31.798117+0000","last_fullsized":"2024-09-19T05:45:31.798117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'165","last_scrub_stamp":"2024-09-19T05:45:31.798029+0000","last_deep_scrub":"102'165","last_deep_scrub_stamp":"2024-09-19T05:45:31.798029+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.798029+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:25:37.298118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.200108+0000","last_change":"2024-09-19T05:45:34.200131+0000","last_active":"2024-09-19T05:45:34.200108+0000","last_peered":"2024-09-19T05:45:34.200108+0000","last_clean":"2024-09-19T05:45:34.200108+0000","last_became_active":"2024-09-19T05:42:31.994552+0000","last_became_peered":"2024-09-19T05:42:31.994552+0000","last_unstale":"2024-09-19T05:45:34.200108+0000","last_undegraded":"2024-09-19T05:45:34.200108+0000","last_fullsized":"2024-09-19T05:45:34.200108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:34.200018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:34.200018+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.200018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:54.207452+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"102'175","reported_seq":365,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:47.682596+0000","last_change":"2024-09-19T05:45:47.682624+0000","last_active":"2024-09-19T05:45:47.682596+0000","last_peered":"2024-09-19T05:45:47.682596+0000","last_clean":"2024-09-19T05:45:47.682596+0000","last_became_active":"2024-09-19T05:43:19.548461+0000","last_became_peered":"2024-09-19T05:43:19.548461+0000","last_unstale":"2024-09-19T05:45:47.682596+0000","last_undegraded":"2024-09-19T05:45:47.682596+0000","last_fullsized":"2024-09-19T05:45:47.682596+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'175","last_scrub_stamp":"2024-09-19T05:45:47.682504+0000","last_deep_scrub":"102'175","last_deep_scrub_stamp":"2024-09-19T05:45:47.682504+0000","last_clean_scrub_stamp":"2024-09-19T05:45:47.682504+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:03:38.251978+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":272,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.995860+0000","last_change":"2024-09-19T05:45:33.995895+0000","last_active":"2024-09-19T05:45:33.995860+0000","last_peered":"2024-09-19T05:45:33.995860+0000","last_clean":"2024-09-19T05:45:33.995860+0000","last_became_active":"2024-09-19T05:45:10.626852+0000","last_became_peered":"2024-09-19T05:45:10.626852+0000","last_unstale":"2024-09-19T05:45:33.995860+0000","last_undegraded":"2024-09-19T05:45:33.995860+0000","last_fullsized":"2024-09-19T05:45:33.995860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'119","last_scrub_stamp":"2024-09-19T05:45:33.995772+0000","last_deep_scrub":"102'119","last_deep_scrub_stamp":"2024-09-19T05:45:33.995772+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.995772+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:45.210928+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.768061+0000","last_change":"2024-09-19T05:45:37.768087+0000","last_active":"2024-09-19T05:45:37.768061+0000","last_peered":"2024-09-19T05:45:37.768061+0000","last_clean":"2024-09-19T05:45:37.768061+0000","last_became_active":"2024-09-19T05:42:31.993952+0000","last_became_peered":"2024-09-19T05:42:31.993952+0000","last_unstale":"2024-09-19T05:45:37.768061+0000","last_undegraded":"2024-09-19T05:45:37.768061+0000","last_fullsized":"2024-09-19T05:45:37.768061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:37.767970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:37.767970+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.767970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:29.604731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"102'183","reported_seq":364,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:36.766613+0000","last_change":"2024-09-19T05:45:36.766670+0000","last_active":"2024-09-19T05:45:36.766613+0000","last_peered":"2024-09-19T05:45:36.766613+0000","last_clean":"2024-09-19T05:45:36.766613+0000","last_became_active":"2024-09-19T05:42:32.930561+0000","last_became_peered":"2024-09-19T05:42:32.930561+0000","last_unstale":"2024-09-19T05:45:36.766613+0000","last_undegraded":"2024-09-19T05:45:36.766613+0000","last_fullsized":"2024-09-19T05:45:36.766613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'183","last_scrub_stamp":"2024-09-19T05:45:36.766515+0000","last_deep_scrub":"102'183","last_deep_scrub_stamp":"2024-09-19T05:45:36.766515+0000","last_clean_scrub_stamp":"2024-09-19T05:45:36.766515+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:19.693027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":189,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:49.713452+0000","last_change":"2024-09-19T05:45:49.713476+0000","last_active":"2024-09-19T05:45:49.713452+0000","last_peered":"2024-09-19T05:45:49.713452+0000","last_clean":"2024-09-19T05:45:49.713452+0000","last_became_active":"2024-09-19T05:44:53.068984+0000","last_became_peered":"2024-09-19T05:44:53.068984+0000","last_unstale":"2024-09-19T05:45:49.713452+0000","last_undegraded":"2024-09-19T05:45:49.713452+0000","last_fullsized":"2024-09-19T05:45:49.713452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:49.713346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:49.713346+0000","last_clean_scrub_stamp":"2024-09-19T05:45:49.713346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:02.064903+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"102'165","reported_seq":346,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.199994+0000","last_change":"2024-09-19T05:45:37.200020+0000","last_active":"2024-09-19T05:45:37.199994+0000","last_peered":"2024-09-19T05:45:37.199994+0000","last_clean":"2024-09-19T05:45:37.199994+0000","last_became_active":"2024-09-19T05:42:32.934809+0000","last_became_peered":"2024-09-19T05:42:32.934809+0000","last_unstale":"2024-09-19T05:45:37.199994+0000","last_undegraded":"2024-09-19T05:45:37.199994+0000","last_fullsized":"2024-09-19T05:45:37.199994+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'165","last_scrub_stamp":"2024-09-19T05:45:37.199880+0000","last_deep_scrub":"102'165","last_deep_scrub_stamp":"2024-09-19T05:45:37.199880+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.199880+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:36.556460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":344064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"102'7","reported_seq":207,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.225608+0000","last_change":"2024-09-19T05:45:39.225653+0000","last_active":"2024-09-19T05:45:39.225608+0000","last_peered":"2024-09-19T05:45:39.225608+0000","last_clean":"2024-09-19T05:45:39.225608+0000","last_became_active":"2024-09-19T05:44:53.058822+0000","last_became_peered":"2024-09-19T05:44:53.058822+0000","last_unstale":"2024-09-19T05:45:39.225608+0000","last_undegraded":"2024-09-19T05:45:39.225608+0000","last_fullsized":"2024-09-19T05:45:39.225608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'7","last_scrub_stamp":"2024-09-19T05:45:39.225500+0000","last_deep_scrub":"102'7","last_deep_scrub_stamp":"2024-09-19T05:45:39.225500+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.225500+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:38.582081+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":101,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"102'165","reported_seq":327,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:46.636789+0000","last_change":"2024-09-19T05:45:46.636817+0000","last_active":"2024-09-19T05:45:46.636789+0000","last_peered":"2024-09-19T05:45:46.636789+0000","last_clean":"2024-09-19T05:45:46.636789+0000","last_became_active":"2024-09-19T05:44:53.068636+0000","last_became_peered":"2024-09-19T05:44:53.068636+0000","last_unstale":"2024-09-19T05:45:46.636789+0000","last_undegraded":"2024-09-19T05:45:46.636789+0000","last_fullsized":"2024-09-19T05:45:46.636789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'165","last_scrub_stamp":"2024-09-19T05:45:46.636697+0000","last_deep_scrub":"102'165","last_deep_scrub_stamp":"2024-09-19T05:45:46.636697+0000","last_clean_scrub_stamp":"2024-09-19T05:45:46.636697+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:25.920793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":142606336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.772284+0000","last_change":"2024-09-19T05:45:42.772308+0000","last_active":"2024-09-19T05:45:42.772284+0000","last_peered":"2024-09-19T05:45:42.772284+0000","last_clean":"2024-09-19T05:45:42.772284+0000","last_became_active":"2024-09-19T05:42:31.997127+0000","last_became_peered":"2024-09-19T05:42:31.997127+0000","last_unstale":"2024-09-19T05:45:42.772284+0000","last_undegraded":"2024-09-19T05:45:42.772284+0000","last_fullsized":"2024-09-19T05:45:42.772284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:42.772179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:42.772179+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.772179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:56.287167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"102'177","reported_seq":358,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.865683+0000","last_change":"2024-09-19T05:45:34.865709+0000","last_active":"2024-09-19T05:45:34.865683+0000","last_peered":"2024-09-19T05:45:34.865683+0000","last_clean":"2024-09-19T05:45:34.865683+0000","last_became_active":"2024-09-19T05:42:32.935006+0000","last_became_peered":"2024-09-19T05:42:32.935006+0000","last_unstale":"2024-09-19T05:45:34.865683+0000","last_undegraded":"2024-09-19T05:45:34.865683+0000","last_fullsized":"2024-09-19T05:45:34.865683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'177","last_scrub_stamp":"2024-09-19T05:45:34.865592+0000","last_deep_scrub":"102'177","last_deep_scrub_stamp":"2024-09-19T05:45:34.865592+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.865592+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:38.616601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":160,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.526145+0000","last_change":"2024-09-19T05:45:35.526168+0000","last_active":"2024-09-19T05:45:35.526145+0000","last_peered":"2024-09-19T05:45:35.526145+0000","last_clean":"2024-09-19T05:45:35.526145+0000","last_became_active":"2024-09-19T05:44:53.061291+0000","last_became_peered":"2024-09-19T05:44:53.061291+0000","last_unstale":"2024-09-19T05:45:35.526145+0000","last_undegraded":"2024-09-19T05:45:35.526145+0000","last_fullsized":"2024-09-19T05:45:35.526145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:35.526057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:35.526057+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.526057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:06.080131+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"102'160","reported_seq":341,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.073051+0000","last_change":"2024-09-19T05:45:34.073085+0000","last_active":"2024-09-19T05:45:34.073051+0000","last_peered":"2024-09-19T05:45:34.073051+0000","last_clean":"2024-09-19T05:45:34.073051+0000","last_became_active":"2024-09-19T05:42:32.935181+0000","last_became_peered":"2024-09-19T05:42:32.935181+0000","last_unstale":"2024-09-19T05:45:34.073051+0000","last_undegraded":"2024-09-19T05:45:34.073051+0000","last_fullsized":"2024-09-19T05:45:34.073051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:34.072956+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:34.072956+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.072956+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:29.693246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:40.761958+0000","last_change":"2024-09-19T05:45:40.761980+0000","last_active":"2024-09-19T05:45:40.761958+0000","last_peered":"2024-09-19T05:45:40.761958+0000","last_clean":"2024-09-19T05:45:40.761958+0000","last_became_active":"2024-09-19T05:42:31.995344+0000","last_became_peered":"2024-09-19T05:42:31.995344+0000","last_unstale":"2024-09-19T05:45:40.761958+0000","last_undegraded":"2024-09-19T05:45:40.761958+0000","last_fullsized":"2024-09-19T05:45:40.761958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:40.761873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:40.761873+0000","last_clean_scrub_stamp":"2024-09-19T05:45:40.761873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:02.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"102'162","reported_seq":442,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:42.190859+0000","last_change":"2024-09-19T05:45:42.190883+0000","last_active":"2024-09-19T05:45:42.190859+0000","last_peered":"2024-09-19T05:45:42.190859+0000","last_clean":"2024-09-19T05:45:42.190859+0000","last_became_active":"2024-09-19T05:44:53.068035+0000","last_became_peered":"2024-09-19T05:44:53.068035+0000","last_unstale":"2024-09-19T05:45:42.190859+0000","last_undegraded":"2024-09-19T05:45:42.190859+0000","last_fullsized":"2024-09-19T05:45:42.190859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:42.190770+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:42.190770+0000","last_clean_scrub_stamp":"2024-09-19T05:45:42.190770+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:56:50.729470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":163577856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"102'18","reported_seq":201,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.668423+0000","last_change":"2024-09-19T05:45:33.668450+0000","last_active":"2024-09-19T05:45:33.668423+0000","last_peered":"2024-09-19T05:45:33.668423+0000","last_clean":"2024-09-19T05:45:33.668423+0000","last_became_active":"2024-09-19T05:42:31.992288+0000","last_became_peered":"2024-09-19T05:42:31.992288+0000","last_unstale":"2024-09-19T05:45:33.668423+0000","last_undegraded":"2024-09-19T05:45:33.668423+0000","last_fullsized":"2024-09-19T05:45:33.668423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'18","last_scrub_stamp":"2024-09-19T05:45:33.668343+0000","last_deep_scrub":"102'18","last_deep_scrub_stamp":"2024-09-19T05:45:33.668343+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.668343+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:55.586362+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"102'166","reported_seq":347,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.790245+0000","last_change":"2024-09-19T05:45:39.790269+0000","last_active":"2024-09-19T05:45:39.790245+0000","last_peered":"2024-09-19T05:45:39.790245+0000","last_clean":"2024-09-19T05:45:39.790245+0000","last_became_active":"2024-09-19T05:42:32.935253+0000","last_became_peered":"2024-09-19T05:42:32.935253+0000","last_unstale":"2024-09-19T05:45:39.790245+0000","last_undegraded":"2024-09-19T05:45:39.790245+0000","last_fullsized":"2024-09-19T05:45:39.790245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'166","last_scrub_stamp":"2024-09-19T05:45:39.790160+0000","last_deep_scrub":"102'166","last_deep_scrub_stamp":"2024-09-19T05:45:39.790160+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.790160+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:56:48.328860+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":376832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"102'4","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:38.762532+0000","last_change":"2024-09-19T05:45:38.762562+0000","last_active":"2024-09-19T05:45:38.762532+0000","last_peered":"2024-09-19T05:45:38.762532+0000","last_clean":"2024-09-19T05:45:38.762532+0000","last_became_active":"2024-09-19T05:42:31.994093+0000","last_became_peered":"2024-09-19T05:42:31.994093+0000","last_unstale":"2024-09-19T05:45:38.762532+0000","last_undegraded":"2024-09-19T05:45:38.762532+0000","last_fullsized":"2024-09-19T05:45:38.762532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'4","last_scrub_stamp":"2024-09-19T05:45:38.762445+0000","last_deep_scrub":"102'4","last_deep_scrub_stamp":"2024-09-19T05:45:38.762445+0000","last_clean_scrub_stamp":"2024-09-19T05:45:38.762445+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:52.648999+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"102'183","reported_seq":364,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:37.780265+0000","last_change":"2024-09-19T05:45:37.780304+0000","last_active":"2024-09-19T05:45:37.780265+0000","last_peered":"2024-09-19T05:45:37.780265+0000","last_clean":"2024-09-19T05:45:37.780265+0000","last_became_active":"2024-09-19T05:42:32.927992+0000","last_became_peered":"2024-09-19T05:42:32.927992+0000","last_unstale":"2024-09-19T05:45:37.780265+0000","last_undegraded":"2024-09-19T05:45:37.780265+0000","last_fullsized":"2024-09-19T05:45:37.780265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'183","last_scrub_stamp":"2024-09-19T05:45:37.780138+0000","last_deep_scrub":"102'183","last_deep_scrub_stamp":"2024-09-19T05:45:37.780138+0000","last_clean_scrub_stamp":"2024-09-19T05:45:37.780138+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:35:06.688522+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":425984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:29.781412+0000","last_change":"2024-09-19T05:45:29.781446+0000","last_active":"2024-09-19T05:45:29.781412+0000","last_peered":"2024-09-19T05:45:29.781412+0000","last_clean":"2024-09-19T05:45:29.781412+0000","last_became_active":"2024-09-19T05:42:31.985345+0000","last_became_peered":"2024-09-19T05:42:31.985345+0000","last_unstale":"2024-09-19T05:45:29.781412+0000","last_undegraded":"2024-09-19T05:45:29.781412+0000","last_fullsized":"2024-09-19T05:45:29.781412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:29.781333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:29.781333+0000","last_clean_scrub_stamp":"2024-09-19T05:45:29.781333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:06.401746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"102'162","reported_seq":342,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:31.844185+0000","last_change":"2024-09-19T05:45:31.844225+0000","last_active":"2024-09-19T05:45:31.844185+0000","last_peered":"2024-09-19T05:45:31.844185+0000","last_clean":"2024-09-19T05:45:31.844185+0000","last_became_active":"2024-09-19T05:42:32.940736+0000","last_became_peered":"2024-09-19T05:42:32.940736+0000","last_unstale":"2024-09-19T05:45:31.844185+0000","last_undegraded":"2024-09-19T05:45:31.844185+0000","last_fullsized":"2024-09-19T05:45:31.844185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:31.844083+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:31.844083+0000","last_clean_scrub_stamp":"2024-09-19T05:45:31.844083+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:40.889989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":183,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:32.631333+0000","last_change":"2024-09-19T05:45:32.631357+0000","last_active":"2024-09-19T05:45:32.631333+0000","last_peered":"2024-09-19T05:45:32.631333+0000","last_clean":"2024-09-19T05:45:32.631333+0000","last_became_active":"2024-09-19T05:42:31.991836+0000","last_became_peered":"2024-09-19T05:42:31.991836+0000","last_unstale":"2024-09-19T05:45:32.631333+0000","last_undegraded":"2024-09-19T05:45:32.631333+0000","last_fullsized":"2024-09-19T05:45:32.631333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:32.631251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:32.631251+0000","last_clean_scrub_stamp":"2024-09-19T05:45:32.631251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:54.032241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"102'173","reported_seq":354,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.761103+0000","last_change":"2024-09-19T05:45:35.761129+0000","last_active":"2024-09-19T05:45:35.761103+0000","last_peered":"2024-09-19T05:45:35.761103+0000","last_clean":"2024-09-19T05:45:35.761103+0000","last_became_active":"2024-09-19T05:42:32.924413+0000","last_became_peered":"2024-09-19T05:42:32.924413+0000","last_unstale":"2024-09-19T05:45:35.761103+0000","last_undegraded":"2024-09-19T05:45:35.761103+0000","last_fullsized":"2024-09-19T05:45:35.761103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'173","last_scrub_stamp":"2024-09-19T05:45:35.761012+0000","last_deep_scrub":"102'173","last_deep_scrub_stamp":"2024-09-19T05:45:35.761012+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.761012+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:45:46.976167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":393216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":190,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.168489+0000","last_change":"2024-09-19T05:45:41.168521+0000","last_active":"2024-09-19T05:45:41.168489+0000","last_peered":"2024-09-19T05:45:41.168489+0000","last_clean":"2024-09-19T05:45:41.168489+0000","last_became_active":"2024-09-19T05:44:53.065298+0000","last_became_peered":"2024-09-19T05:44:53.065298+0000","last_unstale":"2024-09-19T05:45:41.168489+0000","last_undegraded":"2024-09-19T05:45:41.168489+0000","last_fullsized":"2024-09-19T05:45:41.168489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T05:45:41.168401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T05:45:41.168401+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.168401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:47.777454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"102'164","reported_seq":346,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:35.238777+0000","last_change":"2024-09-19T05:45:35.238803+0000","last_active":"2024-09-19T05:45:35.238777+0000","last_peered":"2024-09-19T05:45:35.238777+0000","last_clean":"2024-09-19T05:45:35.238777+0000","last_became_active":"2024-09-19T05:42:32.932540+0000","last_became_peered":"2024-09-19T05:42:32.932540+0000","last_unstale":"2024-09-19T05:45:35.238777+0000","last_undegraded":"2024-09-19T05:45:35.238777+0000","last_fullsized":"2024-09-19T05:45:35.238777+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'164","last_scrub_stamp":"2024-09-19T05:45:35.238693+0000","last_deep_scrub":"102'164","last_deep_scrub_stamp":"2024-09-19T05:45:35.238693+0000","last_clean_scrub_stamp":"2024-09-19T05:45:35.238693+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:42:26.361883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":368640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"102'4","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.809734+0000","last_change":"2024-09-19T05:45:39.809775+0000","last_active":"2024-09-19T05:45:39.809734+0000","last_peered":"2024-09-19T05:45:39.809734+0000","last_clean":"2024-09-19T05:45:39.809734+0000","last_became_active":"2024-09-19T05:42:31.994532+0000","last_became_peered":"2024-09-19T05:42:31.994532+0000","last_unstale":"2024-09-19T05:45:39.809734+0000","last_undegraded":"2024-09-19T05:45:39.809734+0000","last_fullsized":"2024-09-19T05:45:39.809734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'4","last_scrub_stamp":"2024-09-19T05:45:39.809653+0000","last_deep_scrub":"102'4","last_deep_scrub_stamp":"2024-09-19T05:45:39.809653+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.809653+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:50:42.917222+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"102'154","reported_seq":344,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:46.709760+0000","last_change":"2024-09-19T05:45:46.709788+0000","last_active":"2024-09-19T05:45:46.709760+0000","last_peered":"2024-09-19T05:45:46.709760+0000","last_clean":"2024-09-19T05:45:46.709760+0000","last_became_active":"2024-09-19T05:43:19.549075+0000","last_became_peered":"2024-09-19T05:43:19.549075+0000","last_unstale":"2024-09-19T05:45:46.709760+0000","last_undegraded":"2024-09-19T05:45:46.709760+0000","last_fullsized":"2024-09-19T05:45:46.709760+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'154","last_scrub_stamp":"2024-09-19T05:45:46.709671+0000","last_deep_scrub":"102'154","last_deep_scrub_stamp":"2024-09-19T05:45:46.709671+0000","last_clean_scrub_stamp":"2024-09-19T05:45:46.709671+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:41.429840+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":335872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"102'205","reported_seq":385,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:45.721727+0000","last_change":"2024-09-19T05:45:45.721775+0000","last_active":"2024-09-19T05:45:45.721727+0000","last_peered":"2024-09-19T05:45:45.721727+0000","last_clean":"2024-09-19T05:45:45.721727+0000","last_became_active":"2024-09-19T05:42:32.947626+0000","last_became_peered":"2024-09-19T05:42:32.947626+0000","last_unstale":"2024-09-19T05:45:45.721727+0000","last_undegraded":"2024-09-19T05:45:45.721727+0000","last_fullsized":"2024-09-19T05:45:45.721727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'205","last_scrub_stamp":"2024-09-19T05:45:45.721639+0000","last_deep_scrub":"102'205","last_deep_scrub_stamp":"2024-09-19T05:45:45.721639+0000","last_clean_scrub_stamp":"2024-09-19T05:45:45.721639+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:52.244624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":458752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"102'3","reported_seq":204,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:41.902337+0000","last_change":"2024-09-19T05:45:41.902369+0000","last_active":"2024-09-19T05:45:41.902337+0000","last_peered":"2024-09-19T05:45:41.902337+0000","last_clean":"2024-09-19T05:45:41.902337+0000","last_became_active":"2024-09-19T05:43:19.554473+0000","last_became_peered":"2024-09-19T05:43:19.554473+0000","last_unstale":"2024-09-19T05:45:41.902337+0000","last_undegraded":"2024-09-19T05:45:41.902337+0000","last_fullsized":"2024-09-19T05:45:41.902337+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'3","last_scrub_stamp":"2024-09-19T05:45:41.902233+0000","last_deep_scrub":"102'3","last_deep_scrub_stamp":"2024-09-19T05:45:41.902233+0000","last_clean_scrub_stamp":"2024-09-19T05:45:41.902233+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:53.704954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"102'162","reported_seq":285,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:33.007918+0000","last_change":"2024-09-19T05:45:33.007918+0000","last_active":"2024-09-19T05:45:33.007918+0000","last_peered":"2024-09-19T05:45:33.007918+0000","last_clean":"2024-09-19T05:45:33.007918+0000","last_became_active":"2024-09-19T05:45:10.626177+0000","last_became_peered":"2024-09-19T05:45:10.626177+0000","last_unstale":"2024-09-19T05:45:33.007918+0000","last_undegraded":"2024-09-19T05:45:33.007918+0000","last_fullsized":"2024-09-19T05:45:33.007918+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'162","last_scrub_stamp":"2024-09-19T05:45:33.007831+0000","last_deep_scrub":"102'162","last_deep_scrub_stamp":"2024-09-19T05:45:33.007831+0000","last_clean_scrub_stamp":"2024-09-19T05:45:33.007831+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:53.254609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"102'5","reported_seq":206,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:39.957671+0000","last_change":"2024-09-19T05:45:39.957699+0000","last_active":"2024-09-19T05:45:39.957671+0000","last_peered":"2024-09-19T05:45:39.957671+0000","last_clean":"2024-09-19T05:45:39.957671+0000","last_became_active":"2024-09-19T05:43:19.554541+0000","last_became_peered":"2024-09-19T05:43:19.554541+0000","last_unstale":"2024-09-19T05:45:39.957671+0000","last_undegraded":"2024-09-19T05:45:39.957671+0000","last_fullsized":"2024-09-19T05:45:39.957671+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'5","last_scrub_stamp":"2024-09-19T05:45:39.957583+0000","last_deep_scrub":"102'5","last_deep_scrub_stamp":"2024-09-19T05:45:39.957583+0000","last_clean_scrub_stamp":"2024-09-19T05:45:39.957583+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:43.815874+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"102'160","reported_seq":329,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:45.609520+0000","last_change":"2024-09-19T05:45:45.609550+0000","last_active":"2024-09-19T05:45:45.609520+0000","last_peered":"2024-09-19T05:45:45.609520+0000","last_clean":"2024-09-19T05:45:45.609520+0000","last_became_active":"2024-09-19T05:44:53.065435+0000","last_became_peered":"2024-09-19T05:44:53.065435+0000","last_unstale":"2024-09-19T05:45:45.609520+0000","last_undegraded":"2024-09-19T05:45:45.609520+0000","last_fullsized":"2024-09-19T05:45:45.609520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'160","last_scrub_stamp":"2024-09-19T05:45:45.609429+0000","last_deep_scrub":"102'160","last_deep_scrub_stamp":"2024-09-19T05:45:45.609429+0000","last_clean_scrub_stamp":"2024-09-19T05:45:45.609429+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:12:12.464926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":352256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":159383552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":165,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.540836+0000","last_change":"2024-09-19T05:45:34.540863+0000","last_active":"2024-09-19T05:45:34.540836+0000","last_peered":"2024-09-19T05:45:34.540836+0000","last_clean":"2024-09-19T05:45:34.540836+0000","last_became_active":"2024-09-19T05:44:53.061142+0000","last_became_peered":"2024-09-19T05:44:53.061142+0000","last_unstale":"2024-09-19T05:45:34.540836+0000","last_undegraded":"2024-09-19T05:45:34.540836+0000","last_fullsized":"2024-09-19T05:45:34.540836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-19T05:45:34.540747+0000","last_deep_scrub":"22'1","last_deep_scrub_stamp":"2024-09-19T05:45:34.540747+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.540747+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:33.138437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"102'181","reported_seq":335,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:44.635675+0000","last_change":"2024-09-19T05:45:44.635703+0000","last_active":"2024-09-19T05:45:44.635675+0000","last_peered":"2024-09-19T05:45:44.635675+0000","last_clean":"2024-09-19T05:45:44.635675+0000","last_became_active":"2024-09-19T05:44:53.064575+0000","last_became_peered":"2024-09-19T05:44:53.064575+0000","last_unstale":"2024-09-19T05:45:44.635675+0000","last_undegraded":"2024-09-19T05:45:44.635675+0000","last_fullsized":"2024-09-19T05:45:44.635675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'181","last_scrub_stamp":"2024-09-19T05:45:44.635567+0000","last_deep_scrub":"102'181","last_deep_scrub_stamp":"2024-09-19T05:45:44.635567+0000","last_clean_scrub_stamp":"2024-09-19T05:45:44.635567+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:48:24.061136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":401408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":176160768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"102'6","reported_seq":189,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T05:45:34.718061+0000","last_change":"2024-09-19T05:45:34.718089+0000","last_active":"2024-09-19T05:45:34.718061+0000","last_peered":"2024-09-19T05:45:34.718061+0000","last_clean":"2024-09-19T05:45:34.718061+0000","last_became_active":"2024-09-19T05:42:31.995178+0000","last_became_peered":"2024-09-19T05:42:31.995178+0000","last_unstale":"2024-09-19T05:45:34.718061+0000","last_undegraded":"2024-09-19T05:45:34.718061+0000","last_fullsized":"2024-09-19T05:45:34.718061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'6","last_scrub_stamp":"2024-09-19T05:45:34.717984+0000","last_deep_scrub":"102'6","last_deep_scrub_stamp":"2024-09-19T05:45:34.717984+0000","last_clean_scrub_stamp":"2024-09-19T05:45:34.717984+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:54.964762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":25165824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":834,"num_bytes_recovered":3141533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},"log_size":11255,"ondisk_log_size":11255,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7970622,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3910,"num_write_kb":11502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3758479,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12722176,"data_stored":15941244,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},"log_size":3916,"ondisk_log_size":3916,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100995,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54516,"kb_used_data":5284,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317324,"statfs":{"total":96636764160,"available":96580939776,"internally_reserved":0,"allocated":5410816,"data_stored":4755262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36536,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335304,"statfs":{"total":96636764160,"available":96599351296,"internally_reserved":0,"allocated":1220608,"data_stored":560915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":4868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":4984832,"data_stored":4334629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":78,"seq":335007449106,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":1736,"kb_used_omap":7,"kb_used_meta":45048,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":1777664,"data_stored":4925594,"data_compressed":447875,"data_compressed_allocated":516096,"data_compressed_original":4329472,"omap_allocated":8019,"internal_metadata":46129325},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50364,"kb_used_data":5292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94321476,"statfs":{"total":96636764160,"available":96585191424,"internally_reserved":0,"allocated":5419008,"data_stored":4755934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40688,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331152,"statfs":{"total":96636764160,"available":96595099648,"internally_reserved":0,"allocated":1212416,"data_stored":560868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40736,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331104,"statfs":{"total":96636764160,"available":96595050496,"internally_reserved":0,"allocated":1261568,"data_stored":1150724,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40692,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331148,"statfs":{"total":96636764160,"available":96595095552,"internally_reserved":0,"allocated":1216512,"data_stored":560342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34210,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32575,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4202496,"data_stored":4194976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":503808,"data_stored":3774940,"data_compressed":415300,"data_compressed_allocated":471040,"data_compressed_original":3756032,"omap_allocated":1223,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3776512,"data_stored":3774343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2401,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T05:46:13.752 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-19T05:46:13.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.864+0000 7f525c1e8640 1 Processor -- start 2024-09-19T05:46:13.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.864+0000 7f525c1e8640 1 -- start start 2024-09-19T05:46:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f525c1e8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 0x7f5254073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f525c1e8640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f525415b660 0x7f52540738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f525c1e8640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5254155130 con 0x7f52540cc3f0 2024-09-19T05:46:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f525c1e8640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f52540cbc60 con 0x7f525415b660 2024-09-19T05:46:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f525c1e8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f52540cb050 con 0x7f5254154d50 2024-09-19T05:46:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5259783640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f52540cc3f0 0x7f525415af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:60922/0 (socket says 172.21.15.2:60922) 2024-09-19T05:46:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5259783640 1 -- 172.21.15.2:0/2589459878 learned_addr learned my addr 172.21.15.2:0/2589459878 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:46:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5259f84640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 0x7f5254073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f525a785640 1 --2- 172.21.15.2:0/2589459878 >> v2:172.21.15.123:3300/0 conn(0x7f525415b660 0x7f52540738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933087412 0 0) 0x7f5254155130 con 0x7f52540cc3f0 2024-09-19T05:46:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5240003520 con 0x7f52540cc3f0 2024-09-19T05:46:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4204392964 0 0) 0x7f5240003520 con 0x7f52540cc3f0 2024-09-19T05:46:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 msgr2=0x7f5254073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 0x7f5254073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 >> v2:172.21.15.123:3300/0 conn(0x7f525415b660 msgr2=0x7f52540738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 --2- 172.21.15.2:0/2589459878 >> v2:172.21.15.123:3300/0 conn(0x7f525415b660 0x7f52540738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52540cb330 con 0x7f52540cc3f0 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3058107890 0 0) 0x7f5244001670 con 0x7f52540cc3f0 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 >> v1:172.21.15.2:6789/0 conn(0x7f52540cc3f0 legacy=0x7f525415af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.865+0000 7f5258f82640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 0x7f52400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5258f82640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5240003f30 0x7f5240004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5258f82640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240004d40 0x7f52400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f52540cbc60 con 0x7f5240003f30 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52540cb050 con 0x7f5240004d40 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f524000b9f0 con 0x7f5254154d50 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5259783640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5240003f30 0x7f5240004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5259f84640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 0x7f52400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f525a785640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240004d40 0x7f52400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5259783640 1 -- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 msgr2=0x7f52400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5259783640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5254154d50 0x7f52400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5259783640 1 -- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240004d40 msgr2=0x7f52400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5259783640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5240004d40 0x7f52400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.866+0000 7f5259783640 1 -- 172.21.15.2:0/2589459878 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5240003520 con 0x7f5240003f30 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.867+0000 7f5259783640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5240003f30 0x7f5240004800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f52540755e0 tx=0x7f5240005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd15d4edd4492c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.867+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244007d50 con 0x7f5240003f30 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.867+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52440058d0 con 0x7f5240003f30 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.867+0000 7f5258f82640 1 -- 172.21.15.2:0/2589459878 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244005cc0 con 0x7f5240003f30 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.868+0000 7f525c1e8640 1 -- 172.21.15.2:0/2589459878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5240003f30 msgr2=0x7f5240004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.868+0000 7f525c1e8640 1 --2- 172.21.15.2:0/2589459878 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5240003f30 0x7f5240004800 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f52540755e0 tx=0x7f5240005830 comp rx=0 tx=0).stop 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.868+0000 7f525a785640 1 -- 172.21.15.2:0/2589459878 reap_dead start 2024-09-19T05:46:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.868+0000 7f525c1e8640 1 -- 172.21.15.2:0/2589459878 shutdown_connections 2024-09-19T05:46:13.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.868+0000 7f525c1e8640 1 -- 172.21.15.2:0/2589459878 >> 172.21.15.2:0/2589459878 conn(0x7f525406d250 msgr2=0x7f525406d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:13.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.868+0000 7f525c1e8640 1 -- 172.21.15.2:0/2589459878 shutdown_connections 2024-09-19T05:46:13.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.868+0000 7f525c1e8640 1 -- 172.21.15.2:0/2589459878 wait complete. 2024-09-19T05:46:13.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.869+0000 7f525c1e8640 1 Processor -- start 2024-09-19T05:46:13.877 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.869+0000 7f525c1e8640 1 -- start start 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525c1e8640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5240003f30 0x7f5254153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525c1e8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254147220 0x7f52540b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525c1e8640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f525414afb0 0x7f52541461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525c1e8640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f525415aa00 con 0x7f525414afb0 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525c1e8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52540c5c40 con 0x7f5254147220 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525c1e8640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f52540cbec0 con 0x7f5240003f30 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f525414afb0 0x7f52541461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.880 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f525414afb0 0x7f52541461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60516/0 (socket says 172.21.15.2:60516) 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 -- 172.21.15.2:0/200191871 learned_addr learned my addr 172.21.15.2:0/200191871 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f5259f84640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5240003f30 0x7f5254153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f5259783640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254147220 0x7f52540b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 -- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5240003f30 msgr2=0x7f5254153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5240003f30 0x7f5254153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 -- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254147220 msgr2=0x7f52540b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254147220 0x7f52540b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 -- 172.21.15.2:0/200191871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5254153d80 con 0x7f525414afb0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525a785640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f525414afb0 0x7f52541461d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f524c011bf0 tx=0x7f524c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c00fcb0 con 0x7f525414afb0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c00fe50 con 0x7f525414afb0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5254154490 con 0x7f525414afb0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.870+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c015670 con 0x7f525414afb0 2024-09-19T05:46:13.881 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.871+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52541541b0 con 0x7f525414afb0 2024-09-19T05:46:13.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.871+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52540c69f0 con 0x7f525414afb0 2024-09-19T05:46:13.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.872+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f524c015810 con 0x7f525414afb0 2024-09-19T05:46:13.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.872+0000 7f524a7fc640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f523806c2e0 0x7f523806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:13.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.872+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f524c08c0c0 con 0x7f525414afb0 2024-09-19T05:46:13.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.873+0000 7f5259f84640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f523806c2e0 0x7f523806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:13.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.873+0000 7f5259f84640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f523806c2e0 0x7f523806e7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5250009a60 tx=0x7f5250009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:13.882 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:13.877+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f524c019030 con 0x7f525414afb0 2024-09-19T05:46:14.055 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.053+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5254147a70 con 0x7f525414afb0 2024-09-19T05:46:14.058 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.056+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f524c010040 con 0x7f525414afb0 2024-09-19T05:46:14.060 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.057+0000 7f524a7fc640 1 -- 172.21.15.2:0/200191871 <== mon.0 v2:172.21.15.2:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f524c05a860 con 0x7f525414afb0 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f523806c2e0 msgr2=0x7f523806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f523806c2e0 0x7f523806e7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5250009a60 tx=0x7f5250009190 comp rx=0 tx=0).stop 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f525414afb0 msgr2=0x7f52541461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f525414afb0 0x7f52541461d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f524c011bf0 tx=0x7f524c00f1b0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 shutdown_connections 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f523806c2e0 0x7f523806e7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5254147220 0x7f52540b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f525414afb0 0x7f52541461d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 --2- 172.21.15.2:0/200191871 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5240003f30 0x7f5254153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.062 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 >> 172.21.15.2:0/200191871 conn(0x7f525406d250 msgr2=0x7f5254073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:14.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 shutdown_connections 2024-09-19T05:46:14.063 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.059+0000 7f525c1e8640 1 -- 172.21.15.2:0/200191871 wait complete. 2024-09-19T05:46:14.072 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T05:46:14.073 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:46:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.172+0000 7fc705f4b640 1 Processor -- start 2024-09-19T05:46:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.172+0000 7fc705f4b640 1 -- start start 2024-09-19T05:46:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc705f4b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7000ce7c0 0x7fc70015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc705f4b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc7000ceba0 0x7fc700157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc705f4b640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc700070f80 0x7fc700158410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc705f4b640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fc7000b13d0 con 0x7fc700070f80 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc705f4b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc7000c9910 con 0x7fc7000ceba0 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc705f4b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc7000c9bb0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc6fffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc700070f80 0x7fc700158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc6fffff640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc700070f80 0x7fc700158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60528/0 (socket says 172.21.15.2:60528) 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc6fffff640 1 -- 172.21.15.2:0/3995669897 learned_addr learned my addr 172.21.15.2:0/3995669897 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc6feffd640 1 --2- 172.21.15.2:0/3995669897 >> v2:172.21.15.123:3300/0 conn(0x7fc7000ceba0 0x7fc700157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.173+0000 7fc6ff7fe640 1 --2- 172.21.15.2:0/3995669897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7000ce7c0 0x7fc70015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.174+0000 7fc6fffff640 1 -- 172.21.15.2:0/3995669897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7000ce7c0 msgr2=0x7fc70015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.174+0000 7fc6fffff640 1 --2- 172.21.15.2:0/3995669897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7000ce7c0 0x7fc70015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.174+0000 7fc6fffff640 1 -- 172.21.15.2:0/3995669897 >> v2:172.21.15.123:3300/0 conn(0x7fc7000ceba0 msgr2=0x7fc700157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.174+0000 7fc6fffff640 1 --2- 172.21.15.2:0/3995669897 >> v2:172.21.15.123:3300/0 conn(0x7fc7000ceba0 0x7fc700157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.174+0000 7fc6fffff640 1 -- 172.21.15.2:0/3995669897 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7000b0e10 con 0x7fc700070f80 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.174+0000 7fc6fffff640 1 --2- 172.21.15.2:0/3995669897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc700070f80 0x7fc700158410 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc7000686c0 tx=0x7fc6f802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a462ddfe1ef8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.174+0000 7fc6fe7fc640 1 -- 172.21.15.2:0/3995669897 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8036070 con 0x7fc700070f80 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc6fe7fc640 1 -- 172.21.15.2:0/3995669897 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc6f8031600 con 0x7fc700070f80 2024-09-19T05:46:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc6fe7fc640 1 -- 172.21.15.2:0/3995669897 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80319f0 con 0x7fc700070f80 2024-09-19T05:46:14.181 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 -- 172.21.15.2:0/3995669897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc700070f80 msgr2=0x7fc700158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3995669897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc700070f80 0x7fc700158410 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc7000686c0 tx=0x7fc6f802a610 comp rx=0 tx=0).stop 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 -- 172.21.15.2:0/3995669897 shutdown_connections 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3995669897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc700070f80 0x7fc700158410 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3995669897 >> v2:172.21.15.123:3300/0 conn(0x7fc7000ceba0 0x7fc700157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3995669897 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc7000ce7c0 0x7fc70015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 -- 172.21.15.2:0/3995669897 >> 172.21.15.2:0/3995669897 conn(0x7fc70006c240 msgr2=0x7fc700067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 -- 172.21.15.2:0/3995669897 shutdown_connections 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.175+0000 7fc705f4b640 1 -- 172.21.15.2:0/3995669897 wait complete. 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.176+0000 7fc705f4b640 1 Processor -- start 2024-09-19T05:46:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.176+0000 7fc705f4b640 1 -- start start 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc705f4b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc700070f80 0x7fc7000c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc705f4b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7000ce7c0 0x7fc7000b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc705f4b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7000ceba0 0x7fc7001f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc705f4b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fc700070300 con 0x7fc7000ce7c0 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc705f4b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc70006ce30 con 0x7fc7000ceba0 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc705f4b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc70006e550 con 0x7fc700070f80 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc6feffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7000ce7c0 0x7fc7000b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc6ff7fe640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc700070f80 0x7fc7000c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc6feffd640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7000ce7c0 0x7fc7000b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60534/0 (socket says 172.21.15.2:60534) 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc6feffd640 1 -- 172.21.15.2:0/3427430472 learned_addr learned my addr 172.21.15.2:0/3427430472 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.177+0000 7fc6fffff640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7000ceba0 0x7fc7001f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6feffd640 1 -- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc700070f80 msgr2=0x7fc7000c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6feffd640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc700070f80 0x7fc7000c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6feffd640 1 -- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7000ceba0 msgr2=0x7fc7001f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6feffd640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7000ceba0 0x7fc7001f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6feffd640 1 -- 172.21.15.2:0/3427430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7001550e0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6feffd640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7000ce7c0 0x7fc7000b2f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400d5a0 tx=0x7fc6f400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6e7fff640 1 -- 172.21.15.2:0/3427430472 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4004240 con 0x7fc7000ce7c0 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6e7fff640 1 -- 172.21.15.2:0/3427430472 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc6f40043e0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7001efda0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc6e7fff640 1 -- 172.21.15.2:0/3427430472 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4003850 con 0x7fc7000ce7c0 2024-09-19T05:46:14.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.178+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7001e62c0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.179+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70006c8a0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.179+0000 7fc6e7fff640 1 -- 172.21.15.2:0/3427430472 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc6f40039f0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.185 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.180+0000 7fc6e7fff640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc6ec06c2b0 0x7fc6ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.180+0000 7fc6e7fff640 1 -- 172.21.15.2:0/3427430472 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fc6f408b690 con 0x7fc7000ce7c0 2024-09-19T05:46:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.180+0000 7fc6ff7fe640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc6ec06c2b0 0x7fc6ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.181+0000 7fc6ff7fe640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc6ec06c2b0 0x7fc6ec06e770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.184+0000 7fc6e7fff640 1 -- 172.21.15.2:0/3427430472 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fc6f4059630 con 0x7fc7000ce7c0 2024-09-19T05:46:14.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.397+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7000c95d0 con 0x7fc7000ce7c0 2024-09-19T05:46:14.400 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.398+0000 7fc6e7fff640 1 -- 172.21.15.2:0/3427430472 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6863 (secure 0 0 0) 0x7fc6f405d930 con 0x7fc7000ce7c0 2024-09-19T05:46:14.400 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:46:14.400 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"btime":"2024-09-19T05:42:38:988574+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/121690198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":121690198},{"type":"v1","addr":"172.21.15.2:6835","nonce":121690198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1118046326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1118046326},{"type":"v1","addr":"172.21.15.123:6837","nonce":1118046326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3264935286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3264935286},{"type":"v1","addr":"172.21.15.2:6837","nonce":3264935286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/4246839966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4246839966},{"type":"v1","addr":"172.21.15.123:6836","nonce":4246839966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:42:32.334667+0000","modified":"2024-09-19T05:42:38.988568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/4093916922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4093916922},{"type":"v1","addr":"172.21.15.123:6839","nonce":4093916922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4312,"qdb_cluster":[4312]},"id":1}]} 2024-09-19T05:46:14.400 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-09-19T05:46:14.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc6ec06c2b0 msgr2=0x7fc6ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc6ec06c2b0 0x7fc6ec06e770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).stop 2024-09-19T05:46:14.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7000ce7c0 msgr2=0x7fc7000b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7000ce7c0 0x7fc7000b2f30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400d5a0 tx=0x7fc6f400da60 comp rx=0 tx=0).stop 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 shutdown_connections 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fc6ec06c2b0 0x7fc6ec06e770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7000ceba0 0x7fc7001f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc7000ce7c0 0x7fc7000b2f30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 --2- 172.21.15.2:0/3427430472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc700070f80 0x7fc7000c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 >> 172.21.15.2:0/3427430472 conn(0x7fc70006c240 msgr2=0x7fc7000cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 shutdown_connections 2024-09-19T05:46:14.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.400+0000 7fc705f4b640 1 -- 172.21.15.2:0/3427430472 wait complete. 2024-09-19T05:46:14.413 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T05:46:14.515 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.513+0000 7fda8905e640 1 Processor -- start 2024-09-19T05:46:14.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.513+0000 7fda8905e640 1 -- start start 2024-09-19T05:46:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8905e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fda840d06e0 0x7fda840cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8905e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fda840d0ac0 0x7fda8415eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8905e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda84154af0 0x7fda84154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8905e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7fda8415f090 con 0x7fda840d0ac0 2024-09-19T05:46:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8905e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fda840cb400 con 0x7fda840d06e0 2024-09-19T05:46:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8905e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fda840cb6e0 con 0x7fda84154af0 2024-09-19T05:46:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fda840d0ac0 0x7fda8415eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8259c640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fda840d0ac0 0x7fda8415eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60544/0 (socket says 172.21.15.2:60544) 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8259c640 1 -- 172.21.15.2:0/1738014265 learned_addr learned my addr 172.21.15.2:0/1738014265 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda8359e640 1 --2- 172.21.15.2:0/1738014265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda84154af0 0x7fda84154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.514+0000 7fda82d9d640 1 --2- 172.21.15.2:0/1738014265 >> v2:172.21.15.123:3300/0 conn(0x7fda840d06e0 0x7fda840cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.515+0000 7fda8259c640 1 -- 172.21.15.2:0/1738014265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda84154af0 msgr2=0x7fda84154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.515+0000 7fda8259c640 1 --2- 172.21.15.2:0/1738014265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda84154af0 0x7fda84154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.515+0000 7fda8259c640 1 -- 172.21.15.2:0/1738014265 >> v2:172.21.15.123:3300/0 conn(0x7fda840d06e0 msgr2=0x7fda840cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.515+0000 7fda8259c640 1 --2- 172.21.15.2:0/1738014265 >> v2:172.21.15.123:3300/0 conn(0x7fda840d06e0 0x7fda840cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.515+0000 7fda8259c640 1 -- 172.21.15.2:0/1738014265 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda840cc970 con 0x7fda840d0ac0 2024-09-19T05:46:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.515+0000 7fda8259c640 1 --2- 172.21.15.2:0/1738014265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda840d0ac0 0x7fda8415eb50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fda840cdfd0 tx=0x7fda6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc71f5c5c9b93251 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.515+0000 7fda81d9b640 1 -- 172.21.15.2:0/1738014265 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c037070 con 0x7fda840d0ac0 2024-09-19T05:46:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda81d9b640 1 -- 172.21.15.2:0/1738014265 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fda6c032440 con 0x7fda840d0ac0 2024-09-19T05:46:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda81d9b640 1 -- 172.21.15.2:0/1738014265 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c032830 con 0x7fda840d0ac0 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 -- 172.21.15.2:0/1738014265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda840d0ac0 msgr2=0x7fda8415eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 --2- 172.21.15.2:0/1738014265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda840d0ac0 0x7fda8415eb50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fda840cdfd0 tx=0x7fda6c02a1e0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 -- 172.21.15.2:0/1738014265 shutdown_connections 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 --2- 172.21.15.2:0/1738014265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda84154af0 0x7fda84154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 --2- 172.21.15.2:0/1738014265 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda840d0ac0 0x7fda8415eb50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 --2- 172.21.15.2:0/1738014265 >> v2:172.21.15.123:3300/0 conn(0x7fda840d06e0 0x7fda840cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 -- 172.21.15.2:0/1738014265 >> 172.21.15.2:0/1738014265 conn(0x7fda8406d740 msgr2=0x7fda84073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 -- 172.21.15.2:0/1738014265 shutdown_connections 2024-09-19T05:46:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.516+0000 7fda8905e640 1 -- 172.21.15.2:0/1738014265 wait complete. 2024-09-19T05:46:14.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 Processor -- start 2024-09-19T05:46:14.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 -- start start 2024-09-19T05:46:14.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda840d06e0 0x7fda8414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda840d0ac0 0x7fda8414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda84154af0 0x7fda840c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fda840c5830 con 0x7fda84154af0 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda840ccf30 con 0x7fda840d0ac0 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8905e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fda840cd450 con 0x7fda840d06e0 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.518+0000 7fda8359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda84154af0 0x7fda840c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda84154af0 0x7fda840c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60550/0 (socket says 172.21.15.2:60550) 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 -- 172.21.15.2:0/2618468513 learned_addr learned my addr 172.21.15.2:0/2618468513 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda82d9d640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda840d06e0 0x7fda8414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8259c640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda840d0ac0 0x7fda8414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 -- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda840d06e0 msgr2=0x7fda8414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda840d06e0 0x7fda8414e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 -- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda840d0ac0 msgr2=0x7fda8414e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda840d0ac0 0x7fda8414e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 -- 172.21.15.2:0/2618468513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8414be50 con 0x7fda84154af0 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8359e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda84154af0 0x7fda840c94f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d30 tx=0x7fda7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda7b7fe640 1 -- 172.21.15.2:0/2618468513 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74018020 con 0x7fda84154af0 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda7b7fe640 1 -- 172.21.15.2:0/2618468513 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fda74014480 con 0x7fda84154af0 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda840c8f70 con 0x7fda84154af0 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda7b7fe640 1 -- 172.21.15.2:0/2618468513 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74014890 con 0x7fda84154af0 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.519+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda840c7910 con 0x7fda84154af0 2024-09-19T05:46:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.520+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda840cbb40 con 0x7fda84154af0 2024-09-19T05:46:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.521+0000 7fda7b7fe640 1 -- 172.21.15.2:0/2618468513 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fda74005d10 con 0x7fda84154af0 2024-09-19T05:46:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.521+0000 7fda7b7fe640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda6406c2b0 0x7fda6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.521+0000 7fda7b7fe640 1 -- 172.21.15.2:0/2618468513 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7fda7408c190 con 0x7fda84154af0 2024-09-19T05:46:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.521+0000 7fda82d9d640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda6406c2b0 0x7fda6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.522+0000 7fda82d9d640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda6406c2b0 0x7fda6406e770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fda7c006fd0 tx=0x7fda7c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.526+0000 7fda7b7fe640 1 -- 172.21.15.2:0/2618468513 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fda74022030 con 0x7fda84154af0 2024-09-19T05:46:14.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.737+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda840c5610 con 0x7fda84154af0 2024-09-19T05:46:14.739 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.737+0000 7fda7b7fe640 1 -- 172.21.15.2:0/2618468513 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6863 (secure 0 0 0) 0x7fda74010040 con 0x7fda84154af0 2024-09-19T05:46:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:46:14.740 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"btime":"2024-09-19T05:42:38:988574+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/121690198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":121690198},{"type":"v1","addr":"172.21.15.2:6835","nonce":121690198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1118046326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1118046326},{"type":"v1","addr":"172.21.15.123:6837","nonce":1118046326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6833/1783859706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1783859706},{"type":"v1","addr":"172.21.15.2:6833","nonce":1783859706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3264935286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3264935286},{"type":"v1","addr":"172.21.15.2:6837","nonce":3264935286}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4306,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/4246839966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4246839966},{"type":"v1","addr":"172.21.15.123:6836","nonce":4246839966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T05:42:32.334667+0000","modified":"2024-09-19T05:42:38.988568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4312":{"gid":4312,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.123:6839/4093916922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":4093916922},{"type":"v1","addr":"172.21.15.123:6839","nonce":4093916922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4312,"qdb_cluster":[4312]},"id":1}]} 2024-09-19T05:46:14.740 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda6406c2b0 msgr2=0x7fda6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda6406c2b0 0x7fda6406e770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fda7c006fd0 tx=0x7fda7c008040 comp rx=0 tx=0).stop 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda84154af0 msgr2=0x7fda840c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda84154af0 0x7fda840c94f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d30 tx=0x7fda7400c490 comp rx=0 tx=0).stop 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 shutdown_connections 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fda6406c2b0 0x7fda6406e770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda840d0ac0 0x7fda8414e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda84154af0 0x7fda840c94f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 --2- 172.21.15.2:0/2618468513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fda840d06e0 0x7fda8414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 >> 172.21.15.2:0/2618468513 conn(0x7fda8406d740 msgr2=0x7fda8415dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.739+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 shutdown_connections 2024-09-19T05:46:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.740+0000 7fda8905e640 1 -- 172.21.15.2:0/2618468513 wait complete. 2024-09-19T05:46:14.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T05:46:14.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.852+0000 7faf45eba640 1 Processor -- start 2024-09-19T05:46:14.855 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.852+0000 7faf45eba640 1 -- start start 2024-09-19T05:46:14.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.853+0000 7faf45eba640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400cdb60 0x7faf400cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.856 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.853+0000 7faf45eba640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf400cb400 con 0x7faf400d0ac0 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.853+0000 7faf45eba640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf400cb6e0 con 0x7faf400d06e0 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.853+0000 7faf45eba640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7faf400cc970 con 0x7faf400cdb60 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.853+0000 7faf3effd640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7faf400d0ac0 0x7faf400cd450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:60938/0 (socket says 172.21.15.2:60938) 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.853+0000 7faf3effd640 1 -- 172.21.15.2:0/2439583055 learned_addr learned my addr 172.21.15.2:0/2439583055 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3ffff640 1 --2- 172.21.15.2:0/2439583055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400cdb60 0x7faf400cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3e7fc640 1 -- 172.21.15.2:0/2439583055 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581980279 0 0) 0x7faf400cb400 con 0x7faf400d0ac0 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3e7fc640 1 -- 172.21.15.2:0/2439583055 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf28003520 con 0x7faf400d0ac0 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3e7fc640 1 -- 172.21.15.2:0/2439583055 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 13132704 0 0) 0x7faf400cb6e0 con 0x7faf400d06e0 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3e7fc640 1 -- 172.21.15.2:0/2439583055 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf400cb400 con 0x7faf400d06e0 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3ffff640 1 -- 172.21.15.2:0/2439583055 >> v1:172.21.15.123:6789/0 conn(0x7faf400d06e0 legacy=0x7faf40154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3ffff640 1 -- 172.21.15.2:0/2439583055 >> v1:172.21.15.2:6789/0 conn(0x7faf400d0ac0 legacy=0x7faf400cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.854+0000 7faf3ffff640 1 -- 172.21.15.2:0/2439583055 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf400ccc50 con 0x7faf400cdb60 2024-09-19T05:46:14.857 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf3ffff640 1 --2- 172.21.15.2:0/2439583055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400cdb60 0x7faf400cbb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf38009dc0 tx=0x7faf3802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65d10dce6880183a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf3e7fc640 1 -- 172.21.15.2:0/2439583055 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38007740 con 0x7faf400cdb60 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf3e7fc640 1 -- 172.21.15.2:0/2439583055 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7faf380078e0 con 0x7faf400cdb60 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf3e7fc640 1 -- 172.21.15.2:0/2439583055 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38007c90 con 0x7faf400cdb60 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf45eba640 1 -- 172.21.15.2:0/2439583055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400cdb60 msgr2=0x7faf400cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf45eba640 1 --2- 172.21.15.2:0/2439583055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400cdb60 0x7faf400cbb40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faf38009dc0 tx=0x7faf3802a6d0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf45eba640 1 -- 172.21.15.2:0/2439583055 shutdown_connections 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf45eba640 1 --2- 172.21.15.2:0/2439583055 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400cdb60 0x7faf400cbb40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf45eba640 1 -- 172.21.15.2:0/2439583055 >> 172.21.15.2:0/2439583055 conn(0x7faf4006d740 msgr2=0x7faf40073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf45eba640 1 -- 172.21.15.2:0/2439583055 shutdown_connections 2024-09-19T05:46:14.859 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.855+0000 7faf45eba640 1 -- 172.21.15.2:0/2439583055 wait complete. 2024-09-19T05:46:14.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.856+0000 7faf45eba640 1 Processor -- start 2024-09-19T05:46:14.860 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf45eba640 1 -- start start 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf45eba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf400cdb60 0x7faf400b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf45eba640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400d06e0 0x7faf401f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf45eba640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf400d0ac0 0x7faf401f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf45eba640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7faf40155200 con 0x7faf400d0ac0 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf45eba640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faf400c5830 con 0x7faf400cdb60 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf45eba640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7faf40155380 con 0x7faf400d06e0 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.857+0000 7faf3ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf400d0ac0 0x7faf401f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf400d0ac0 0x7faf401f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60564/0 (socket says 172.21.15.2:60564) 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 -- 172.21.15.2:0/1538336945 learned_addr learned my addr 172.21.15.2:0/1538336945 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:14.865 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3effd640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400d06e0 0x7faf401f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3f7fe640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf400cdb60 0x7faf400b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 -- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400d06e0 msgr2=0x7faf401f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400d06e0 0x7faf401f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 -- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf400cdb60 msgr2=0x7faf400b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf400cdb60 0x7faf400b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 -- 172.21.15.2:0/1538336945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf401e6650 con 0x7faf400d0ac0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf3ffff640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf400d0ac0 0x7faf401f4480 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faf3802acc0 tx=0x7faf38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf27fff640 1 -- 172.21.15.2:0/1538336945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3803f070 con 0x7faf400d0ac0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf27fff640 1 -- 172.21.15.2:0/1538336945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7faf38031c40 con 0x7faf400d0ac0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf401e6930 con 0x7faf400d0ac0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf27fff640 1 -- 172.21.15.2:0/1538336945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38009450 con 0x7faf400d0ac0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.858+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf40155ac0 con 0x7faf400d0ac0 2024-09-19T05:46:14.866 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.859+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf400cbb40 con 0x7faf400d0ac0 2024-09-19T05:46:14.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.860+0000 7faf27fff640 1 -- 172.21.15.2:0/1538336945 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faf38003990 con 0x7faf400d0ac0 2024-09-19T05:46:14.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.860+0000 7faf27fff640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf2c06c2e0 0x7faf2c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:14.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.860+0000 7faf27fff640 1 -- 172.21.15.2:0/1538336945 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7faf380acbb0 con 0x7faf400d0ac0 2024-09-19T05:46:14.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.861+0000 7faf3f7fe640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf2c06c2e0 0x7faf2c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:14.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.861+0000 7faf3f7fe640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf2c06c2e0 0x7faf2c06e7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf30007fd0 tx=0x7faf30009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:14.867 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:14.865+0000 7faf27fff640 1 -- 172.21.15.2:0/1538336945 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7faf38049060 con 0x7faf400d0ac0 2024-09-19T05:46:15.045 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.042+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf4015df90 con 0x7faf400d0ac0 2024-09-19T05:46:15.046 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.043+0000 7faf27fff640 1 -- 172.21.15.2:0/1538336945 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16762 (secure 0 0 0) 0x7faf3807abe0 con 0x7faf400d0ac0 2024-09-19T05:46:15.046 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-19T05:46:15.046 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":102,"fsid":"896e162c-4658-4743-b70e-7ec2b5bce950","created":"2024-09-19T05:42:09.734568+0000","modified":"2024-09-19T05:45:21.682862+0000","last_up_change":"2024-09-19T05:44:52.039950+0000","last_in_change":"2024-09-19T05:43:19.394170+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T05:42:21.863894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T05:42:30.912746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T05:42:31.312255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"665c2ac5-a964-4f38-8fa6-f4e95cac9272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6809","nonce":288510134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6811","nonce":288510134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6815","nonce":288510134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":288510134},{"type":"v1","addr":"172.21.15.2:6813","nonce":288510134}]},"public_addr":"172.21.15.2:6809/288510134","cluster_addr":"172.21.15.2:6811/288510134","heartbeat_back_addr":"172.21.15.2:6815/288510134","heartbeat_front_addr":"172.21.15.2:6813/288510134","state":["exists","up"]},{"osd":1,"uuid":"a4443f63-2c47-4c6f-a2df-6765b78afcc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6817","nonce":2074594312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6819","nonce":2074594312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6823","nonce":2074594312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2074594312},{"type":"v1","addr":"172.21.15.2:6821","nonce":2074594312}]},"public_addr":"172.21.15.2:6817/2074594312","cluster_addr":"172.21.15.2:6819/2074594312","heartbeat_back_addr":"172.21.15.2:6823/2074594312","heartbeat_front_addr":"172.21.15.2:6821/2074594312","state":["exists","up"]},{"osd":2,"uuid":"61465313-1ba1-4f26-b488-6ea73981419b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6825","nonce":2644191985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6827","nonce":2644191985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6831","nonce":2644191985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2644191985},{"type":"v1","addr":"172.21.15.2:6829","nonce":2644191985}]},"public_addr":"172.21.15.2:6825/2644191985","cluster_addr":"172.21.15.2:6827/2644191985","heartbeat_back_addr":"172.21.15.2:6831/2644191985","heartbeat_front_addr":"172.21.15.2:6829/2644191985","state":["exists","up"]},{"osd":3,"uuid":"fa9d27d5-68d4-4467-a2a5-79f767a071aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6801","nonce":3985782072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6803","nonce":3985782072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6807","nonce":3985782072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":3985782072},{"type":"v1","addr":"172.21.15.2:6805","nonce":3985782072}]},"public_addr":"172.21.15.2:6801/3985782072","cluster_addr":"172.21.15.2:6803/3985782072","heartbeat_back_addr":"172.21.15.2:6807/3985782072","heartbeat_front_addr":"172.21.15.2:6805/3985782072","state":["exists","up"]},{"osd":4,"uuid":"85a040ff-ad8b-44ff-b82e-a36b05d83087","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6817","nonce":2654957182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6819","nonce":2654957182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6823","nonce":2654957182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2654957182},{"type":"v1","addr":"172.21.15.123:6821","nonce":2654957182}]},"public_addr":"172.21.15.123:6817/2654957182","cluster_addr":"172.21.15.123:6819/2654957182","heartbeat_back_addr":"172.21.15.123:6823/2654957182","heartbeat_front_addr":"172.21.15.123:6821/2654957182","state":["exists","up"]},{"osd":5,"uuid":"f90f8fa4-4afc-4272-89eb-5e90123155d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":78,"up_thru":82,"down_at":68,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6801","nonce":2989037926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6803","nonce":2989037926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6807","nonce":2989037926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":2989037926},{"type":"v1","addr":"172.21.15.123:6805","nonce":2989037926}]},"public_addr":"172.21.15.123:6801/2989037926","cluster_addr":"172.21.15.123:6803/2989037926","heartbeat_back_addr":"172.21.15.123:6807/2989037926","heartbeat_front_addr":"172.21.15.123:6805/2989037926","state":["exists","up"]},{"osd":6,"uuid":"48561580-ba9a-4096-9722-16c308f414ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6825","nonce":593730548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6827","nonce":593730548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6831","nonce":593730548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":593730548},{"type":"v1","addr":"172.21.15.123:6829","nonce":593730548}]},"public_addr":"172.21.15.123:6825/593730548","cluster_addr":"172.21.15.123:6827/593730548","heartbeat_back_addr":"172.21.15.123:6831/593730548","heartbeat_front_addr":"172.21.15.123:6829/593730548","state":["exists","up"]},{"osd":7,"uuid":"d1ee7357-25a5-432c-b96a-e808139a1cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6809","nonce":1657801827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6811","nonce":1657801827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6815","nonce":1657801827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1657801827},{"type":"v1","addr":"172.21.15.123:6813","nonce":1657801827}]},"public_addr":"172.21.15.123:6809/1657801827","cluster_addr":"172.21.15.123:6811/1657801827","heartbeat_back_addr":"172.21.15.123:6815/1657801827","heartbeat_front_addr":"172.21.15.123:6813/1657801827","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.148620+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.921396+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.980898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.184322+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.139258+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:17.479047+0000","dead_epoch":68},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.432111+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T05:42:18.402960+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.2a","osds":[5,0]}],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T05:46:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf2c06c2e0 msgr2=0x7faf2c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf2c06c2e0 0x7faf2c06e7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf30007fd0 tx=0x7faf30009040 comp rx=0 tx=0).stop 2024-09-19T05:46:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf400d0ac0 msgr2=0x7faf401f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.048 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf400d0ac0 0x7faf401f4480 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faf3802acc0 tx=0x7faf38004a40 comp rx=0 tx=0).stop 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 shutdown_connections 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7faf2c06c2e0 0x7faf2c06e7a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf400d0ac0 0x7faf401f4480 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7faf400d06e0 0x7faf401f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 --2- 172.21.15.2:0/1538336945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faf400cdb60 0x7faf400b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.045+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 >> 172.21.15.2:0/1538336945 conn(0x7faf4006d740 msgr2=0x7faf400c6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.046+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 shutdown_connections 2024-09-19T05:46:15.049 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.046+0000 7faf45eba640 1 -- 172.21.15.2:0/1538336945 wait complete. 2024-09-19T05:46:15.060 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T05:46:15.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.161+0000 7f01b281e640 1 Processor -- start 2024-09-19T05:46:15.163 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.161+0000 7f01b281e640 1 -- start start 2024-09-19T05:46:15.164 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01b281e640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f01ac0cb640 0x7f01ac0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01b281e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f01ac0cba20 0x7f01ac1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01b281e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac15ede0 0x7f01ac154f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01b281e640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac0c5ef0 con 0x7f01ac0cba20 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01b281e640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f01ac15ab40 con 0x7f01ac0cb640 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01b281e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac15ade0 con 0x7f01ac15ede0 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f01ac0cba20 0x7f01ac1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f01ac0cba20 0x7f01ac1549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60580/0 (socket says 172.21.15.2:60580) 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 -- 172.21.15.2:0/3639909380 learned_addr learned my addr 172.21.15.2:0/3639909380 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01b0dbb640 1 --2- 172.21.15.2:0/3639909380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac15ede0 0x7f01ac154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01abfff640 1 --2- 172.21.15.2:0/3639909380 >> v2:172.21.15.123:3300/0 conn(0x7f01ac0cb640 0x7f01ac0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 -- 172.21.15.2:0/3639909380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac15ede0 msgr2=0x7f01ac154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 --2- 172.21.15.2:0/3639909380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac15ede0 0x7f01ac154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.165 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 -- 172.21.15.2:0/3639909380 >> v2:172.21.15.123:3300/0 conn(0x7f01ac0cb640 msgr2=0x7f01ac0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:15.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 --2- 172.21.15.2:0/3639909380 >> v2:172.21.15.123:3300/0 conn(0x7f01ac0cb640 0x7f01ac0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01ab7fe640 1 -- 172.21.15.2:0/3639909380 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac15b0c0 con 0x7f01ac0cba20 2024-09-19T05:46:15.166 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.162+0000 7f01abfff640 1 --2- 172.21.15.2:0/3639909380 >> v2:172.21.15.123:3300/0 conn(0x7f01ac0cb640 0x7f01ac0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T05:46:15.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.163+0000 7f01ab7fe640 1 --2- 172.21.15.2:0/3639909380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cba20 0x7f01ac1549e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f01ac073fd0 tx=0x7f01a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e16f522325049 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:15.167 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.163+0000 7f01aaffd640 1 -- 172.21.15.2:0/3639909380 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0037070 con 0x7f01ac0cba20 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.163+0000 7f01aaffd640 1 -- 172.21.15.2:0/3639909380 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f01a0032440 con 0x7f01ac0cba20 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.163+0000 7f01aaffd640 1 -- 172.21.15.2:0/3639909380 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0032830 con 0x7f01ac0cba20 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.163+0000 7f01b281e640 1 -- 172.21.15.2:0/3639909380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cba20 msgr2=0x7f01ac1549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 --2- 172.21.15.2:0/3639909380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cba20 0x7f01ac1549e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f01ac073fd0 tx=0x7f01a002a270 comp rx=0 tx=0).stop 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 -- 172.21.15.2:0/3639909380 shutdown_connections 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 --2- 172.21.15.2:0/3639909380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac15ede0 0x7f01ac154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 --2- 172.21.15.2:0/3639909380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cba20 0x7f01ac1549e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 --2- 172.21.15.2:0/3639909380 >> v2:172.21.15.123:3300/0 conn(0x7f01ac0cb640 0x7f01ac0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 -- 172.21.15.2:0/3639909380 >> 172.21.15.2:0/3639909380 conn(0x7f01ac067950 msgr2=0x7f01ac067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 -- 172.21.15.2:0/3639909380 shutdown_connections 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.164+0000 7f01b281e640 1 -- 172.21.15.2:0/3639909380 wait complete. 2024-09-19T05:46:15.168 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.165+0000 7f01b281e640 1 Processor -- start 2024-09-19T05:46:15.169 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.165+0000 7f01b281e640 1 -- start start 2024-09-19T05:46:15.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b281e640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cb640 0x7f01ac155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b281e640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac0cba20 0x7f01ac15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b281e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac15ede0 0x7f01ac0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b281e640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac15b960 con 0x7f01ac0cb640 2024-09-19T05:46:15.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b281e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac15b680 con 0x7f01ac15ede0 2024-09-19T05:46:15.171 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b281e640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f01ac0c5a00 con 0x7f01ac0cba20 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cb640 0x7f01ac155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cb640 0x7f01ac155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60586/0 (socket says 172.21.15.2:60586) 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 -- 172.21.15.2:0/1173355818 learned_addr learned my addr 172.21.15.2:0/1173355818 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b0dbb640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac15ede0 0x7f01ac0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 -- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac0cba20 msgr2=0x7f01ac15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01ab7fe640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac0cba20 0x7f01ac15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac0cba20 0x7f01ac15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 -- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac15ede0 msgr2=0x7f01ac0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac15ede0 0x7f01ac0b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 -- 172.21.15.2:0/1173355818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac1f1340 con 0x7f01ac0cb640 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01b0dbb640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac15ede0 0x7f01ac0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01ab7fe640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac0cba20 0x7f01ac15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01abfff640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cb640 0x7f01ac155f80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f019c00e840 tx=0x7f019c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:15.172 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01a8ff9640 1 -- 172.21.15.2:0/1173355818 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c00f040 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.166+0000 7f01a8ff9640 1 -- 172.21.15.2:0/1173355818 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f019c004030 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.167+0000 7f01a8ff9640 1 -- 172.21.15.2:0/1173355818 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c004420 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.167+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac0b5f30 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.167+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac0b61f0 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.168+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac0c54c0 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.168+0000 7f01a8ff9640 1 -- 172.21.15.2:0/1173355818 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f019c00c6d0 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.169+0000 7f01a8ff9640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f019806c2b0 0x7f019806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.169+0000 7f01a8ff9640 1 -- 172.21.15.2:0/1173355818 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5799+0+0 (secure 0 0 0) 0x7f019c08b150 con 0x7f01ac0cb640 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.169+0000 7f01ab7fe640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f019806c2b0 0x7f019806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.173 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.169+0000 7f01ab7fe640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f019806c2b0 0x7f019806e770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01a0034000 tx=0x7f01a002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:15.175 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.173+0000 7f01a8ff9640 1 -- 172.21.15.2:0/1173355818 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f019c017020 con 0x7f01ac0cb640 2024-09-19T05:46:15.388 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.385+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f01ac0cbf10 con 0x7f01ac0cb640 2024-09-19T05:46:15.389 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.387+0000 7f01a8ff9640 1 -- 172.21.15.2:0/1173355818 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f019c059860 con 0x7f01ac0cb640 2024-09-19T05:46:15.389 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T05:46:15.392 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f019806c2b0 msgr2=0x7f019806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f019806c2b0 0x7f019806e770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01a0034000 tx=0x7f01a002e000 comp rx=0 tx=0).stop 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cb640 msgr2=0x7f01ac155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cb640 0x7f01ac155f80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f019c00e840 tx=0x7f019c00ed00 comp rx=0 tx=0).stop 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 shutdown_connections 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f019806c2b0 0x7f019806e770 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01a0034000 tx=0x7f01a002e000 comp rx=0 tx=0).stop 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ac15ede0 0x7f01ac0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01ac0cba20 0x7f01ac15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 --2- 172.21.15.2:0/1173355818 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f01ac0cb640 0x7f01ac155f80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 >> 172.21.15.2:0/1173355818 conn(0x7f01ac067950 msgr2=0x7f01ac0ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 shutdown_connections 2024-09-19T05:46:15.393 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.390+0000 7f01b281e640 1 -- 172.21.15.2:0/1173355818 wait complete. 2024-09-19T05:46:15.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T05:46:15.435 INFO:tasks.ceph.mds.f.smithi123.stdout:starting mds.f at 2024-09-19T05:46:15.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.508+0000 7f1368c9b640 1 Processor -- start 2024-09-19T05:46:15.510 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.508+0000 7f1368c9b640 1 -- start start 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f1368c9b640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1364154d50 0x7f1364073390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f1368c9b640 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1364155130 con 0x7f13640cc3f0 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f1368c9b640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13640cbc60 con 0x7f136415b660 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f1368c9b640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f13640cb050 con 0x7f1364154d50 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f1361d9b640 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f13640cc3f0 0x7f136415af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:60954/0 (socket says 172.21.15.2:60954) 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f1361d9b640 1 -- 172.21.15.2:0/104773780 learned_addr learned my addr 172.21.15.2:0/104773780 (peer_addr_for_me v1:172.21.15.2:0/0) 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f136259c640 1 --2- 172.21.15.2:0/104773780 >> v2:172.21.15.123:3300/0 conn(0x7f1364154d50 0x7f1364073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f136159a640 1 -- 172.21.15.2:0/104773780 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3760620745 0 0) 0x7f1364155130 con 0x7f13640cc3f0 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.509+0000 7f136159a640 1 -- 172.21.15.2:0/104773780 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1350003520 con 0x7f13640cc3f0 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.510+0000 7f136259c640 1 -- 172.21.15.2:0/104773780 >> v1:172.21.15.123:6789/0 conn(0x7f136415b660 legacy=0x7f1364165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.512 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.510+0000 7f136259c640 1 -- 172.21.15.2:0/104773780 >> v1:172.21.15.2:6789/0 conn(0x7f13640cc3f0 legacy=0x7f136415af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.513 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.510+0000 7f136259c640 1 -- 172.21.15.2:0/104773780 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640cb330 con 0x7f1364154d50 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.510+0000 7f136259c640 1 --2- 172.21.15.2:0/104773780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364154d50 0x7f1364073390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1354009d90 tx=0x7f135402a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a9b85aac65b8160 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.510+0000 7f136159a640 1 -- 172.21.15.2:0/104773780 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354037070 con 0x7f1364154d50 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f136159a640 1 -- 172.21.15.2:0/104773780 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1354007740 con 0x7f1364154d50 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f136159a640 1 -- 172.21.15.2:0/104773780 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354007b30 con 0x7f1364154d50 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f1368c9b640 1 -- 172.21.15.2:0/104773780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364154d50 msgr2=0x7f1364073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f1368c9b640 1 --2- 172.21.15.2:0/104773780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364154d50 0x7f1364073390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1354009d90 tx=0x7f135402a780 comp rx=0 tx=0).stop 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f1368c9b640 1 -- 172.21.15.2:0/104773780 shutdown_connections 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f1368c9b640 1 --2- 172.21.15.2:0/104773780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1364154d50 0x7f1364073390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f1368c9b640 1 -- 172.21.15.2:0/104773780 >> 172.21.15.2:0/104773780 conn(0x7f136406d250 msgr2=0x7f136406d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f1368c9b640 1 -- 172.21.15.2:0/104773780 shutdown_connections 2024-09-19T05:46:15.514 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.511+0000 7f1368c9b640 1 -- 172.21.15.2:0/104773780 wait complete. 2024-09-19T05:46:15.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.512+0000 7f1368c9b640 1 Processor -- start 2024-09-19T05:46:15.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1368c9b640 1 -- start start 2024-09-19T05:46:15.516 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1368c9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13640cc3f0 0x7f13640735b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1368c9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1364154d50 0x7f13640c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1368c9b640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f136415b660 0x7f13640c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1368c9b640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f136415aa00 con 0x7f1364154d50 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1368c9b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13640c5c40 con 0x7f13640cc3f0 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1368c9b640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f13640cbec0 con 0x7f136415b660 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1361d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1364154d50 0x7f13640c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1361d9b640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1364154d50 0x7f13640c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60588/0 (socket says 172.21.15.2:60588) 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1361d9b640 1 -- 172.21.15.2:0/2611376663 learned_addr learned my addr 172.21.15.2:0/2611376663 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f136259c640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13640cc3f0 0x7f13640735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.513+0000 7f1362d9d640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f136415b660 0x7f13640c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1361d9b640 1 -- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f136415b660 msgr2=0x7f13640c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1361d9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f136415b660 0x7f13640c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1361d9b640 1 -- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13640cc3f0 msgr2=0x7f13640735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1361d9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13640cc3f0 0x7f13640735b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1361d9b640 1 -- 172.21.15.2:0/2611376663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640ca570 con 0x7f1364154d50 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f136259c640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13640cc3f0 0x7f13640735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1361d9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1364154d50 0x7f13640c9430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c500 tx=0x7f134c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f135affd640 1 -- 172.21.15.2:0/2611376663 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c017070 con 0x7f1364154d50 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13641f42d0 con 0x7f1364154d50 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f135affd640 1 -- 172.21.15.2:0/2611376663 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f134c005ce0 con 0x7f1364154d50 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f135affd640 1 -- 172.21.15.2:0/2611376663 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c007350 con 0x7f1364154d50 2024-09-19T05:46:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.514+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13640c7db0 con 0x7f1364154d50 2024-09-19T05:46:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.515+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13640c8a50 con 0x7f1364154d50 2024-09-19T05:46:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.516+0000 7f135affd640 1 -- 172.21.15.2:0/2611376663 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f134c01a070 con 0x7f1364154d50 2024-09-19T05:46:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.516+0000 7f135affd640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f134406c2b0 0x7f134406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.516+0000 7f135affd640 1 -- 172.21.15.2:0/2611376663 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5885+0+0 (secure 0 0 0) 0x7f134c08aa40 con 0x7f1364154d50 2024-09-19T05:46:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.516+0000 7f136259c640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f134406c2b0 0x7f134406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:15.520 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.517+0000 7f136259c640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f134406c2b0 0x7f134406e770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1354034000 tx=0x7f135402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:15.522 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.520+0000 7f135affd640 1 -- 172.21.15.2:0/2611376663 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f13640c8a50 con 0x7f1364154d50 2024-09-19T05:46:15.737 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:15.735+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f13640c69f0 con 0x7f1364154d50 2024-09-19T05:46:16.396 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.394+0000 7f135affd640 1 -- 172.21.15.2:0/2611376663 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f134c058a50 con 0x7f1364154d50 2024-09-19T05:46:16.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f134406c2b0 msgr2=0x7f134406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f134406c2b0 0x7f134406e770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1354034000 tx=0x7f135402e000 comp rx=0 tx=0).stop 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1364154d50 msgr2=0x7f13640c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1364154d50 0x7f13640c9430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c500 tx=0x7f134c00c9c0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 shutdown_connections 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f134406c2b0 0x7f134406e770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f136415b660 0x7f13640c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1364154d50 0x7f13640c9430 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 --2- 172.21.15.2:0/2611376663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f13640cc3f0 0x7f13640735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 >> 172.21.15.2:0/2611376663 conn(0x7f136406d250 msgr2=0x7f1364152780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 shutdown_connections 2024-09-19T05:46:16.399 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.396+0000 7f1368c9b640 1 -- 172.21.15.2:0/2611376663 wait complete. 2024-09-19T05:46:16.411 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T05:46:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.519+0000 7f35606d2640 1 Processor -- start 2024-09-19T05:46:16.521 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.519+0000 7f35606d2640 1 -- start start 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f35606d2640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f35580cb450 0x7f355815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f35606d2640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35580cb830 0x7f35580cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f35606d2640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35580c58d0 con 0x7f35581548d0 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f35606d2640 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 -- 0x7f355815b9f0 con 0x7f35580cb450 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f35606d2640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f35580cc480 con 0x7f35580cb830 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355e46e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f35580cb450 0x7f355815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355e46e640 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f35580cb450 0x7f355815b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60602/0 (socket says 172.21.15.2:60602) 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355e46e640 1 -- 172.21.15.2:0/1984316366 learned_addr learned my addr 172.21.15.2:0/1984316366 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355dc6d640 1 --2- 172.21.15.2:0/1984316366 >> v2:172.21.15.123:3300/0 conn(0x7f35580cb830 0x7f35580cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355e46e640 1 -- 172.21.15.2:0/1984316366 >> v2:172.21.15.123:3300/0 conn(0x7f35580cb830 msgr2=0x7f35580cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:16.523 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355e46e640 1 --2- 172.21.15.2:0/1984316366 >> v2:172.21.15.123:3300/0 conn(0x7f35580cb830 0x7f35580cbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355e46e640 1 -- 172.21.15.2:0/1984316366 >> v1:172.21.15.123:6789/0 conn(0x7f35581548d0 legacy=0x7f355806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:16.524 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.520+0000 7f355e46e640 1 -- 172.21.15.2:0/1984316366 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35580cca30 con 0x7f35580cb450 2024-09-19T05:46:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.521+0000 7f355e46e640 1 --2- 172.21.15.2:0/1984316366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb450 0x7f355815b4b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3554009870 tx=0x7f355402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd55b4dd3656c1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.521+0000 7f355d46c640 1 -- 172.21.15.2:0/1984316366 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554037070 con 0x7f35580cb450 2024-09-19T05:46:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.521+0000 7f355d46c640 1 -- 172.21.15.2:0/1984316366 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3554032440 con 0x7f35580cb450 2024-09-19T05:46:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.521+0000 7f355d46c640 1 -- 172.21.15.2:0/1984316366 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554032830 con 0x7f35580cb450 2024-09-19T05:46:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 -- 172.21.15.2:0/1984316366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb450 msgr2=0x7f355815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:16.525 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 --2- 172.21.15.2:0/1984316366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb450 0x7f355815b4b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3554009870 tx=0x7f355402a270 comp rx=0 tx=0).stop 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 -- 172.21.15.2:0/1984316366 shutdown_connections 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 --2- 172.21.15.2:0/1984316366 >> v2:172.21.15.123:3300/0 conn(0x7f35580cb830 0x7f35580cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 --2- 172.21.15.2:0/1984316366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb450 0x7f355815b4b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 -- 172.21.15.2:0/1984316366 >> 172.21.15.2:0/1984316366 conn(0x7f3558067950 msgr2=0x7f3558067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 -- 172.21.15.2:0/1984316366 shutdown_connections 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.522+0000 7f35606d2640 1 -- 172.21.15.2:0/1984316366 wait complete. 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.523+0000 7f35606d2640 1 Processor -- start 2024-09-19T05:46:16.526 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.523+0000 7f35606d2640 1 -- start start 2024-09-19T05:46:16.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f35606d2640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35580cb450 0x7f35581f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:16.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f35606d2640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb830 0x7f3558150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:16.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f35606d2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35581548d0 0x7f355814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:16.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f35606d2640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7f35580cd2d0 con 0x7f35580cb830 2024-09-19T05:46:16.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f35606d2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35580ccff0 con 0x7f35581548d0 2024-09-19T05:46:16.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f35606d2640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f35580cd5b0 con 0x7f35580cb450 2024-09-19T05:46:16.528 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb830 0x7f3558150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb830 0x7f3558150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60614/0 (socket says 172.21.15.2:60614) 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 -- 172.21.15.2:0/1148207000 learned_addr learned my addr 172.21.15.2:0/1148207000 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355e46e640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35580cb450 0x7f35581f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355ec6f640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35581548d0 0x7f355814f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 -- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35580cb450 msgr2=0x7f35581f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35580cb450 0x7f35581f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 -- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35581548d0 msgr2=0x7f355814f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35581548d0 0x7f355814f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 -- 172.21.15.2:0/1148207000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35581f4970 con 0x7f35580cb830 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.524+0000 7f355dc6d640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb830 0x7f3558150390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f354800c820 tx=0x7f354800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.525+0000 7f354effd640 1 -- 172.21.15.2:0/1148207000 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548015070 con 0x7f35580cb830 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.525+0000 7f354effd640 1 -- 172.21.15.2:0/1148207000 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3548004220 con 0x7f35580cb830 2024-09-19T05:46:16.529 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.525+0000 7f354effd640 1 -- 172.21.15.2:0/1148207000 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354800ce10 con 0x7f35580cb830 2024-09-19T05:46:16.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.525+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35580cf410 con 0x7f35580cb830 2024-09-19T05:46:16.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.525+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35580cf990 con 0x7f35580cb830 2024-09-19T05:46:16.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.526+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355815b4b0 con 0x7f35580cb830 2024-09-19T05:46:16.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.526+0000 7f354effd640 1 -- 172.21.15.2:0/1148207000 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3548007830 con 0x7f35580cb830 2024-09-19T05:46:16.530 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.527+0000 7f354effd640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f353c06c2e0 0x7f353c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:16.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.527+0000 7f354effd640 1 -- 172.21.15.2:0/1148207000 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5885+0+0 (secure 0 0 0) 0x7f354808b300 con 0x7f35580cb830 2024-09-19T05:46:16.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.527+0000 7f355e46e640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f353c06c2e0 0x7f353c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:16.531 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.527+0000 7f355e46e640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f353c06c2e0 0x7f353c06e7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3554034000 tx=0x7f355402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:16.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.531+0000 7f354effd640 1 -- 172.21.15.2:0/1148207000 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7f35480592e0 con 0x7f35580cb830 2024-09-19T05:46:16.709 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:16.707+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35580cbd20 con 0x7f35580cb830 2024-09-19T05:46:17.398 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.395+0000 7f354effd640 1 -- 172.21.15.2:0/1148207000 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7f354805d5e0 con 0x7f35580cb830 2024-09-19T05:46:17.398 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2024-09-19T05:46:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f353c06c2e0 msgr2=0x7f353c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:17.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f353c06c2e0 0x7f353c06e7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3554034000 tx=0x7f355402e000 comp rx=0 tx=0).stop 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb830 msgr2=0x7f3558150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb830 0x7f3558150390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f354800c820 tx=0x7f354800cce0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 shutdown_connections 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7f353c06c2e0 0x7f353c06e7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35581548d0 0x7f355814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35580cb830 0x7f3558150390 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 --2- 172.21.15.2:0/1148207000 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35580cb450 0x7f35581f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 >> 172.21.15.2:0/1148207000 conn(0x7f3558067950 msgr2=0x7f35580b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 shutdown_connections 2024-09-19T05:46:17.403 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.400+0000 7f35606d2640 1 -- 172.21.15.2:0/1148207000 wait complete. 2024-09-19T05:46:17.418 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T05:46:17.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.530+0000 7fbc83d0a640 1 Processor -- start 2024-09-19T05:46:17.533 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.531+0000 7fbc83d0a640 1 -- start start 2024-09-19T05:46:17.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.531+0000 7fbc83d0a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c154dc0 0x7fbc7c15ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:17.534 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.531+0000 7fbc83d0a640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc7c0cb620 0x7fbc7c06d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.531+0000 7fbc83d0a640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc7c15b050 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.531+0000 7fbc83d0a640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbc7c06cdf0 con 0x7fbc7c0cb620 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.531+0000 7fbc83d0a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7c15b2c0 con 0x7fbc7c154dc0 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc81aa6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c154dc0 0x7fbc7c15ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc812a5640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbc7c0cb620 0x7fbc7c06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc81aa6640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c154dc0 0x7fbc7c15ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.2:47344/0 (socket says 172.21.15.2:47344) 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc81aa6640 1 -- 172.21.15.2:0/1723864312 learned_addr learned my addr 172.21.15.2:0/1723864312 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc81aa6640 1 -- 172.21.15.2:0/1723864312 >> v2:172.21.15.123:3300/0 conn(0x7fbc7c0cb620 msgr2=0x7fbc7c06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc81aa6640 1 --2- 172.21.15.2:0/1723864312 >> v2:172.21.15.123:3300/0 conn(0x7fbc7c0cb620 0x7fbc7c06d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc81aa6640 1 -- 172.21.15.2:0/1723864312 >> v1:172.21.15.123:6789/0 conn(0x7fbc7c0cbf40 legacy=0x7fbc7c15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc81aa6640 1 -- 172.21.15.2:0/1723864312 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c15b5a0 con 0x7fbc7c154dc0 2024-09-19T05:46:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.532+0000 7fbc812a5640 1 --2- 172.21.15.2:0/1723864312 >> v2:172.21.15.123:3300/0 conn(0x7fbc7c0cb620 0x7fbc7c06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:46:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.533+0000 7fbc81aa6640 1 --2- 172.21.15.2:0/1723864312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c154dc0 0x7fbc7c15ab10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009e50 tx=0x7fbc6c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74caa26473ae1e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.533+0000 7fbc80aa4640 1 -- 172.21.15.2:0/1723864312 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c036070 con 0x7fbc7c154dc0 2024-09-19T05:46:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.533+0000 7fbc80aa4640 1 -- 172.21.15.2:0/1723864312 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbc6c031600 con 0x7fbc7c154dc0 2024-09-19T05:46:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.533+0000 7fbc80aa4640 1 -- 172.21.15.2:0/1723864312 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0319f0 con 0x7fbc7c154dc0 2024-09-19T05:46:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 -- 172.21.15.2:0/1723864312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c154dc0 msgr2=0x7fbc7c15ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:17.537 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/1723864312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c154dc0 0x7fbc7c15ab10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009e50 tx=0x7fbc6c02a680 comp rx=0 tx=0).stop 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 -- 172.21.15.2:0/1723864312 shutdown_connections 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/1723864312 >> v2:172.21.15.123:3300/0 conn(0x7fbc7c0cb620 0x7fbc7c06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/1723864312 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c154dc0 0x7fbc7c15ab10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 -- 172.21.15.2:0/1723864312 >> 172.21.15.2:0/1723864312 conn(0x7fbc7c067950 msgr2=0x7fbc7c067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 -- 172.21.15.2:0/1723864312 shutdown_connections 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.534+0000 7fbc83d0a640 1 -- 172.21.15.2:0/1723864312 wait complete. 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.535+0000 7fbc83d0a640 1 Processor -- start 2024-09-19T05:46:17.538 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.535+0000 7fbc83d0a640 1 -- start start 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc83d0a640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c0cb620 0x7fbc7c1f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc83d0a640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc7c0cbf40 0x7fbc7c1f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc83d0a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc7c154dc0 0x7fbc7c158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc83d0a640 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c0b34e0 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc83d0a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c06ca80 con 0x7fbc7c154dc0 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc83d0a640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7c06d7c0 con 0x7fbc7c0cb620 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc7c0cbf40 0x7fbc7c1f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc7c0cbf40 0x7fbc7c1f19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60618/0 (socket says 172.21.15.2:60618) 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 -- 172.21.15.2:0/958633247 learned_addr learned my addr 172.21.15.2:0/958633247 (peer_addr_for_me v2:172.21.15.2:0/0) 2024-09-19T05:46:17.541 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc81aa6640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c0cb620 0x7fbc7c1f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc822a7640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc7c154dc0 0x7fbc7c158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 -- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c0cb620 msgr2=0x7fbc7c1f3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c0cb620 0x7fbc7c1f3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 -- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc7c154dc0 msgr2=0x7fbc7c158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc7c154dc0 0x7fbc7c158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 -- 172.21.15.2:0/958633247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c0d01f0 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc81aa6640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c0cb620 0x7fbc7c1f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc812a5640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc7c0cbf40 0x7fbc7c1f19e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000ec10 tx=0x7fbc7000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc7a7fc640 1 -- 172.21.15.2:0/958633247 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70017070 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc7a7fc640 1 -- 172.21.15.2:0/958633247 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbc7000cc30 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c0ce740 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.542 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.536+0000 7fbc7a7fc640 1 -- 172.21.15.2:0/958633247 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70013470 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.537+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c0cf710 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.538+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7c1551a0 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.538+0000 7fbc7a7fc640 1 -- 172.21.15.2:0/958633247 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbc7001a070 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.539+0000 7fbc7a7fc640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbc6806c2b0 0x7fbc6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T05:46:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.539+0000 7fbc7a7fc640 1 -- 172.21.15.2:0/958633247 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5447+0+0 (secure 0 0 0) 0x7fbc7008b2a0 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.539+0000 7fbc81aa6640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbc6806c2b0 0x7fbc6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T05:46:17.543 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.539+0000 7fbc81aa6640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbc6806c2b0 0x7fbc6806e770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c033000 tx=0x7fbc6c009000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T05:46:17.545 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.543+0000 7fbc7a7fc640 1 -- 172.21.15.2:0/958633247 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187789 (secure 0 0 0) 0x7fbc70059430 con 0x7fbc7c0cbf40 2024-09-19T05:46:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:17.721+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbc7c0cf540 con 0x7fbc7c0cbf40 2024-09-19T05:46:18.411 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.408+0000 7fbc7a7fc640 1 -- 172.21.15.2:0/958633247 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7fbc7005d730 con 0x7fbc7c0cbf40 2024-09-19T05:46:18.411 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2024-09-19T05:46:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.413+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbc6806c2b0 msgr2=0x7fbc6806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.413+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbc6806c2b0 0x7fbc6806e770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c033000 tx=0x7fbc6c009000 comp rx=0 tx=0).stop 2024-09-19T05:46:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.413+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc7c0cbf40 msgr2=0x7fbc7c1f19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T05:46:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.413+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc7c0cbf40 0x7fbc7c1f19e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000ec10 tx=0x7fbc7000c520 comp rx=0 tx=0).stop 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 shutdown_connections 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:6832/302582025,v1:172.21.15.123:6833/302582025] conn(0x7fbc6806c2b0 0x7fbc6806e770 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc7c154dc0 0x7fbc7c158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc7c0cbf40 0x7fbc7c1f19e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 --2- 172.21.15.2:0/958633247 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbc7c0cb620 0x7fbc7c1f3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 >> 172.21.15.2:0/958633247 conn(0x7fbc7c067950 msgr2=0x7fbc7c151a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 shutdown_connections 2024-09-19T05:46:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-19T05:46:18.414+0000 7fbc83d0a640 1 -- 172.21.15.2:0/958633247 wait complete. 2024-09-19T05:46:18.433 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-19T05:46:18.433 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T05:46:18.434 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.444 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T05:46:18.444 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T05:46:18.444 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.495 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T05:46:18.495 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-19T05:46:18.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.545 INFO:tasks.ceph.mds.e:Stopped 2024-09-19T05:46:18.545 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T05:46:18.545 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.556 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T05:46:18.556 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T05:46:18.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.607 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T05:46:18.608 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-19T05:46:18.608 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.658 INFO:tasks.ceph.mds.f:Stopped 2024-09-19T05:46:18.659 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-19T05:46:18.659 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-19T05:46:18.659 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.687 INFO:tasks.ceph.osd.0:Stopped 2024-09-19T05:46:18.687 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-19T05:46:18.687 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.761 INFO:tasks.ceph.osd.1:Stopped 2024-09-19T05:46:18.761 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-19T05:46:18.761 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.835 INFO:tasks.ceph.osd.2:Stopped 2024-09-19T05:46:18.835 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-19T05:46:18.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.908 INFO:tasks.ceph.osd.3:Stopped 2024-09-19T05:46:18.908 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-19T05:46:18.908 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.951 INFO:tasks.ceph.osd.4:Stopped 2024-09-19T05:46:18.952 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-19T05:46:18.952 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:18.990 INFO:tasks.ceph.osd.5:Stopped 2024-09-19T05:46:18.991 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-19T05:46:18.991 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:19.061 INFO:tasks.ceph.osd.6:Stopped 2024-09-19T05:46:19.062 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-19T05:46:19.062 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:19.139 INFO:tasks.ceph.osd.7:Stopped 2024-09-19T05:46:19.139 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-19T05:46:19.140 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-19T05:46:19.140 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:19.168 INFO:tasks.ceph.mgr.y:Stopped 2024-09-19T05:46:19.168 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-19T05:46:19.169 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:19.210 INFO:tasks.ceph.mgr.x:Stopped 2024-09-19T05:46:19.210 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-19T05:46:19.210 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-19T05:46:19.210 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:19.224 INFO:tasks.ceph.mon.a:Stopped 2024-09-19T05:46:19.226 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-19T05:46:19.226 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:19.263 INFO:tasks.ceph.mon.b:Stopped 2024-09-19T05:46:19.263 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-19T05:46:19.263 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T05:46:19.315 INFO:tasks.ceph.mon.c:Stopped 2024-09-19T05:46:19.316 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-19T05:46:19.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v MDS_SLOW_METADATA_IO | egrep -v MDS_TRIM | head -n 1 2024-09-19T05:46:19.368 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:46:19.369 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-19T05:46:19.609 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:46:19.610 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-19T05:46:19.699 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:46:19.699 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-19T05:46:19.797 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:46:19.798 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-19T05:46:19.855 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:46:19.855 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-19T05:46:20.109 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:46:20.109 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-19T05:46:20.204 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:46:20.204 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-19T05:46:20.256 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:46:20.256 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-19T05:46:20.313 INFO:tasks.ceph:Archiving mon data... 2024-09-19T05:46:20.314 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/data/mon.a.tgz 2024-09-19T05:46:20.314 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-19T05:46:20.334 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.CZcEEkA8NP 2024-09-19T05:46:20.334 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.CZcEEkA8NP 2024-09-19T05:46:20.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.CZcEEkA8NP 2024-09-19T05:46:20.590 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.CZcEEkA8NP is 371KB 2024-09-19T05:46:20.688 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.CZcEEkA8NP 2024-09-19T05:46:20.704 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/data/mon.b.tgz 2024-09-19T05:46:20.704 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-09-19T05:46:20.723 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.jtH4hOUxp0 2024-09-19T05:46:20.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jtH4hOUxp0 2024-09-19T05:46:20.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.jtH4hOUxp0 2024-09-19T05:46:20.992 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.jtH4hOUxp0 is 526KB 2024-09-19T05:46:21.089 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.jtH4hOUxp0 2024-09-19T05:46:21.104 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/data/mon.c.tgz 2024-09-19T05:46:21.104 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-09-19T05:46:21.158 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.N2v2byW2N0 2024-09-19T05:46:21.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.N2v2byW2N0 2024-09-19T05:46:21.327 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.N2v2byW2N0 2024-09-19T05:46:21.410 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.N2v2byW2N0 is 526KB 2024-09-19T05:46:21.510 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.N2v2byW2N0 2024-09-19T05:46:21.525 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-19T05:46:21.526 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T05:46:21.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T05:46:21.615 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T05:46:21.644 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T05:46:21.645 INFO:tasks.ceph:Archiving crash dumps... 2024-09-19T05:46:21.646 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/remote/smithi002/crash 2024-09-19T05:46:21.647 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T05:46:21.686 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/remote/smithi123/crash 2024-09-19T05:46:21.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T05:46:21.711 INFO:tasks.ceph:Shutting down logrotate 2024-09-19T05:46:21.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:46:21.731 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T05:46:21.800 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T05:46:21.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T05:46:21.852 INFO:tasks.ceph:Compressing logs... 2024-09-19T05:46:21.853 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:46:21.870 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T05:46:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45533.log 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61189.log 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.45533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48242.log 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45533.log.gz 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41687.log 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.48242.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61189.log.gz 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2024-09-19T05:46:21.902 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48242.log.gz 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50355.log 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41687.log.gz 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.45568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41352.log 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55928.log 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50355.log.gz 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60191.log 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41352.log.gz 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.55928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44695.log 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49325.log 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55928.log.gz 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60191.log.gz 2024-09-19T05:46:21.903 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.44695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60471.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44695.log.gz 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.49325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40794.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60471.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49325.log.gz 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41683.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60471.log.gz 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42638.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40794.log.gz 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58533.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.0.42638.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41683.log.gz 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53237.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42638.log.gz 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47826.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40836.log 2024-09-19T05:46:21.904 INFO:teuthology.orchestra.run.smithi002.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.58533.log.gz 2024-09-19T05:46:21.905 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53237.log.gz 2024-09-19T05:46:21.905 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.47826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43501.log 2024-09-19T05:46:21.905 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43940.log 2024-09-19T05:46:21.905 INFO:teuthology.orchestra.run.smithi002.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.47826.log.gz 2024-09-19T05:46:21.905 INFO:teuthology.orchestra.run.smithi002.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40836.log.gz 2024-09-19T05:46:21.905 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.43501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42813.log 2024-09-19T05:46:21.935 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-19T05:46:21.935 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-19T05:46:21.935 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37429.log 2024-09-19T05:46:21.935 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-19T05:46:21.935 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37429.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T05:46:21.935 INFO:teuthology.orchestra.run.smithi123.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37429.log.gz 2024-09-19T05:46:21.935 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.4.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37338.log 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37338.log: /var/log/ceph/ceph-client.admin.37258.log 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37218.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37338.log.gz 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37509.log 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37258.log.gz 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T05:46:21.936 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.47642.log 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.37509.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/objectstore_tool.47642.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37509.log.gz 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37298.log 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37298.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37298.log.gz 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.d.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.49360.log 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mgr.x.log: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-19T05:46:21.937 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-19T05:46:21.947 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/objectstore_tool.49360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2024-09-19T05:46:21.960 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-09-19T05:46:21.967 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.37380.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.48442.log 2024-09-19T05:46:21.973 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mds.f.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-09-19T05:46:21.978 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.50134.log 2024-09-19T05:46:21.992 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/objectstore_tool.48442.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.49360.log.gz 2024-09-19T05:46:22.005 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/objectstore_tool.50134.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-19T05:46:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49592.log 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.43940.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.43501.log.gz 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43940.log.gz 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.49592.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42813.log.gz 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60429.log 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49592.log.gz 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52709.log 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41046.log: /var/log/ceph/ceph-client.admin.60429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46511.log 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60429.log.gz 2024-09-19T05:46:22.020 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.52709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42771.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.46511.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52709.log.gz 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50496.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46511.log.gz 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59840.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42771.log.gz 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59519.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50496.log.gz 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59840.log.gz 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51378.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59519.log.gz 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.47111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60190.log 2024-09-19T05:46:22.021 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51378.log.gz 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.47985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54483.log 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51079.log 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60190.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.54483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58580.log 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60190.log.gz 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51079.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54483.log.gz 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51079.log.gz 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55547.log 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58580.log.gz 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55395.log 2024-09-19T05:46:22.022 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.55547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52152.log 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55547.log.gz 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.55395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60181.log 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.52152.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55395.log.gz 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59510.log 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52152.log.gz 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.51010.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51584.log 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60181.log.gz 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56790.log 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59510.log.gz 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49914.log 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51584.log.gz 2024-09-19T05:46:22.023 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.56790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61441.log 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56790.log.gz 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61525.log 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.49914.log: /var/log/ceph/ceph-client.admin.61441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40562.log 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49914.log.gz 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61525.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61441.log.gz 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53069.log 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61525.log.gz 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41340.log 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53069.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40562.log.gz 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41340.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53069.log.gz 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45812.log 2024-09-19T05:46:22.024 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43098.log 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41340.log.gz 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.57170.log: /var/log/ceph/ceph-client.admin.45812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59508.log 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44538.log 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.43098.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.45812.log.gz 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59508.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51448.log 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43098.log.gz 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.59508.log.gz 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51379.log 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.44538.log: /var/log/ceph/ceph-client.admin.51448.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44538.log.gz 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49071.log 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51448.log.gz 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50570.log 2024-09-19T05:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51379.log: /var/log/ceph/ceph-client.admin.49071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49071.log.gz 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50570.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51379.log.gz 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41682.log 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.43741.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50570.log.gz 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54019.log 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59849.log 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr: 90.9% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41682.log.gz 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.54019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41928.log 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47766.log 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59849.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.54019.log.gz 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61811.log 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59849.log.gz 2024-09-19T05:46:22.026 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41928.log.gz 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.47766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45233.log 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56422.log 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61811.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47766.log.gz 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.45233.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61811.log.gz 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54240.log 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51750.log 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45233.log.gz 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.56422.log: /var/log/ceph/ceph-client.admin.54240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60387.log 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56422.log.gz 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54240.log.gz 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61399.log 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51750.log: /var/log/ceph/ceph-client.admin.60387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49674.log 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51750.log.gz 2024-09-19T05:46:22.027 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60387.log.gz 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.49674.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61399.log.gz 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60187.log 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53257.log 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49674.log.gz 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.43095.log: /var/log/ceph/ceph-client.admin.60187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40520.log 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60187.log.gz 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59295.log 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60180.log 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40520.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53257.log.gz 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59295.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40520.log.gz 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48364.log 2024-09-19T05:46:22.028 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59295.log.gz 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42180.log 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60180.log: /var/log/ceph/ceph-client.admin.48364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58064.log 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60180.log.gz 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41690.log 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48364.log.gz 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51542.log 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42180.log.gz 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60184.log 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58064.log.gz 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59838.log 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41690.log.gz 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51542.log: /var/log/ceph/ceph-client.admin.60184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53319.log 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51542.log.gz 2024-09-19T05:46:22.029 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60184.log.gz 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55670.log 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53959.log 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr: 88.9%/var/log/ceph/ceph-client.admin.53319.log: -- replaced with /var/log/ceph/ceph-client.admin.59838.log.gz 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.55670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43440.log 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53319.log.gz 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54097.log 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.55670.log.gz 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.43440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41004.log 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53959.log.gz 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43440.log.gz 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.54097.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41004.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54097.log.gz 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59850.log 2024-09-19T05:46:22.030 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41004.log.gz 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59337.log 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59850.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.41351.log -- replaced with /var/log/ceph/ceph-client.admin.59850.log.gz 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42855.log 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59337.log.gz 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53505.log/var/log/ceph/ceph-client.admin.41351.log: 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50852.log 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42855.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53720.log 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42855.log.gz 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59851.log 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53505.log.gz 2024-09-19T05:46:22.031 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50852.log: /var/log/ceph/ceph-client.admin.53720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44298.log 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53720.log.gz 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50852.log.gz 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60685.log 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59851.log.gz 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53362.log 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.44298.log: /var/log/ceph/ceph-client.admin.60685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60685.log.gz 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44298.log.gz 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43898.log 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53362.log.gz 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61105.log 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.46329.log: /var/log/ceph/ceph-client.admin.43898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41339.log 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61105.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43898.log.gz 2024-09-19T05:46:22.032 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60853.log 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36449.log 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61105.log.gz 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41339.log: /var/log/ceph/ceph-client.admin.60853.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41339.log.gz 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60853.log.gz 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61147.log 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36449.log: /var/log/ceph/ceph-client.admin.48421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40878.log 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36449.log.gz 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58749.log 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40878.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61147.log.gz 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2024-09-19T05:46:22.033 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40878.log.gz 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47270.log 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58749.log: /var/log/ceph/ceph-client.admin.49469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53761.log 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58749.log.gz 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.47270.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gz 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46772.log 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48813.log 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53761.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47270.log.gz 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.46772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57585.log 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53761.log.gz 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.48813.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51688.log 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.57585.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46772.log.gz 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57467.log 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48813.log.gz 2024-09-19T05:46:22.034 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45152.log 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.57585.log.gz 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50631.log 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.57467.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51688.log.gz 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.45152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61645.log 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50631.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45152.log.gz 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.57467.log.gz 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41684.log 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.50631.log.gz 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.61645.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56912.log 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58399.log 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61645.log.gz 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41684.log.gz 2024-09-19T05:46:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.56912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61895.log 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52592.log 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58399.log: /var/log/ceph/ceph-client.admin.61895.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56912.log.gz 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41353.log 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.58399.log.gz 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61895.log.gz 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.52592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60051.log 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47527.log 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41353.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52592.log.gz 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42897.log 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41353.log.gz 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.47527.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.60051.log.gz -5 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr: --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47527.log.gz 2024-09-19T05:46:22.036 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42897.log.gz 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47188.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.47568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36805.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.47188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59517.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47188.log.gz 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.36805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61273.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59517.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36805.log.gz 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45293.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59517.log.gz 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50273.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52962.log 2024-09-19T05:46:22.037 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.45293.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61273.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45293.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50273.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40650.log 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.52962.log: /var/log/ceph/ceph-client.admin.40962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52962.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: -5/var/log/ceph/ceph-client.admin.40650.log: --verbose -- /var/log/ceph/ceph-client.admin.61063.log 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.56715.log 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40650.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61063.log: gzip -5 --verbose 90.6% -- /var/log/ceph/ceph-client.admin.44217.log -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61063.log.gz 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.56715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50431.log 2024-09-19T05:46:22.038 INFO:teuthology.orchestra.run.smithi002.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56715.log.gz 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.44217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52492.log 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz --verbose 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.50750.log 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50431.log: /var/log/ceph/ceph-client.admin.52492.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.50431.log.gz 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54625.log 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52492.log.gz 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57925.log 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50750.log: /var/log/ceph/ceph-client.admin.54625.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50750.log.gz 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61021.log 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54743.log 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.57925.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.54625.log.gz 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61021.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57925.log.gz 2024-09-19T05:46:22.039 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61315.log 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46228.log 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61021.log.gz 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.54743.log: /var/log/ceph/ceph-client.admin.61315.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54743.log.gz 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48547.log 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46531.log 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61315.log.gz 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.46228.log: /var/log/ceph/ceph-client.admin.48547.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46228.log.gz 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55272.log 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59043.log 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.46531.log: 91.1%/var/log/ceph/ceph-client.admin.55272.log: -- replaced with /var/log/ceph/ceph-client.admin.48547.log.gz 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58492.log 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46531.log.gz 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55272.log.gz 2024-09-19T05:46:22.040 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40692.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59043.log: /var/log/ceph/ceph-client.admin.58492.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59043.log.gz 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58492.log.gz 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58791.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40692.log.gz 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42632.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52485.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58791.log.gz 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.0.42632.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58833.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42632.log.gz 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.52485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44476.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60769.log 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52485.log.gz 2024-09-19T05:46:22.041 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.44476.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58833.log.gz 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59169.log 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44834.log 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.44476.log.gz 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59169.log: gzip -5 88.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.60769.log.gz /var/log/ceph/ceph-client.admin.49407.log 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.44834.log: -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59169.log.gz 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr: /var/log/ceph/ceph-client.admin.59085.log 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.49407.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44834.log.gz 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57068.log 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49407.log.gz 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41681.log 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.57068.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59085.log.gz 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57366.log 2024-09-19T05:46:22.042 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57068.log.gz 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.41681.log: -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.57366.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41681.log.gz 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50303.log 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57366.log.gz 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51895.log 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59518.log 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51895.log: gzip -5 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50303.log.gz 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42096.log 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.51895.log.gz 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56163.log 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59518.log.gz 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42012.log 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42096.log.gz 2024-09-19T05:46:22.043 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.56163.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30406.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56163.log.gz 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52110.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42012.log.gz 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph.tmp-client.admin.30406.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30406.log.gz 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.52110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49248.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52110.log.gz 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.57905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50112.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: 90.9%/var/log/ceph/ceph-client.admin.49248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61979.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57565.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.49248.log.gz 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51687.log 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50112.log.gz 2024-09-19T05:46:22.044 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61979.log: /var/log/ceph/ceph-client.admin.57565.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61979.log.gz 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51687.log.gz 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53153.log 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.50186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53015.log 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr: 86.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.46792.log 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53153.log: /var/log/ceph/ceph-client.admin.53015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59516.log 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.46792.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53015.log.gz 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53153.log.gz 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:gzip 88.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.59852.log 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46792.log.gz 2024-09-19T05:46:22.045 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59516.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59852.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59516.log.gz 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41970.log 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59852.log.gz/var/log/ceph/ceph.log: 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50691.log 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59855.log 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41970.log: /var/log/ceph/ceph-client.admin.50691.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41970.log.gz 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.50691.log.gz 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42054.log 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59855.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59855.log.gz 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43096.log 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49131.log 2024-09-19T05:46:22.046 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4%/var/log/ceph/ceph-client.admin.43096.log: -- replaced with /var/log/ceph/ceph-client.admin.42054.log.gz 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40920.log 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.49131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60189.log 90.6% -- replaced with /var/log/ceph/ceph-client.admin.43096.log.gz 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40920.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49131.log.gz 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58220.log 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40920.log.gz 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59843.log 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60189.log.gz 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59515.log 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59843.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58220.log.gz 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61483.log 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59515.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59843.log.gz 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2024-09-19T05:46:22.047 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59515.log.gz 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61483.log: -- /var/log/ceph/ceph-client.admin.60979.log 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41354.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61483.log.gz 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.54286.log 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60979.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60979.log.gz 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.61727.log 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.54286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60895.log 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61727.log: gzip -5 90.7% -- replaced with /var/log/ceph/ceph-client.admin.54286.log.gz 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54159.log 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61727.log.gz/var/log/ceph/ceph-client.admin.60895.log: 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60895.log.gz 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr: /var/log/ceph/ceph-client.admin.41130.log 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.54159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40376.log 2024-09-19T05:46:22.048 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54159.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41886.log 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41088.log 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40376.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61585.log 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41088.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr: /var/log/ceph/ceph-client.admin.41348.log 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8%/var/log/ceph/ceph-client.admin.61585.log: -- replaced with /var/log/ceph/ceph-client.admin.41088.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40478.log 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61585.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.41348.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41688.log 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40478.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41348.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40478.log.gz 2024-09-19T05:46:22.049 INFO:teuthology.orchestra.run.smithi002.stderr: /var/log/ceph/ceph-client.admin.41343.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61769.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41688.log.gz 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.41343.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41343.log.gz 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.48100.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43559.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61769.log.gz 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.48100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58665.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.43559.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48100.log.gz 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51500.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43559.log.gz 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40752.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59253.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58665.log.gz 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52350.log 2024-09-19T05:46:22.050 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51500.log.gz 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40752.log: /var/log/ceph/ceph-client.admin.59253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55845.log 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40752.log.gz 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.59001.log 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59253.log.gz 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.52350.log: /var/log/ceph/ceph-client.admin.55845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61357.log 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59001.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.55845.log.gz --verbose 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr: -- /var/log/ceph/ceph-client.admin.45094.log 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52350.log.gz 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42282.log 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59001.log.gz 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.45094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53195.log 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61357.log.gz 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60727.log 2024-09-19T05:46:22.051 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42282.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45094.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.53195.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42282.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53195.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44737.log 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2%/var/log/ceph/ceph-client.admin.46611.log: -- replaced with /var/log/ceph/ceph-client.admin.60727.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55042.log 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.44737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59211.log 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48772.log/var/log/ceph/ceph-client.admin.55042.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44737.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59211.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55042.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56356.log 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59211.log.gz 2024-09-19T05:46:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.48772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58875.log 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.56356.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48772.log.gz 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61231.log 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56356.log.gz 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36713.log 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58875.log.gz 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58621.log 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61231.log: /var/log/ceph/ceph-client.admin.36713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61853.log 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61231.log.gz 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58959.log 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36713.log.gz 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45929.log 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58621.log.gz 2024-09-19T05:46:22.053 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61853.log: /var/log/ceph/ceph-client.admin.58959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59127.log 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0%/var/log/ceph/ceph-client.admin.45929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59504.log 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58959.log.gz 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61853.log.gz 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45929.log.gz 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.59127.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41689.log 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.59504.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59127.log.gz 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60811.log 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59504.log.gz 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.41689.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56078.log 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41689.log.gz 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60811.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60811.log.gz 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.56078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40261.log 2024-09-19T05:46:22.054 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58707.log 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56078.log.gz 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.40261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40606.log 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54824.log 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40261.log.gz 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58707.log: /var/log/ceph/ceph-client.admin.40606.log: 92.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58917.log 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: 87.7%/var/log/ceph/ceph-client.admin.54824.log: -- replaced with /var/log/ceph/ceph-client.admin.58707.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40606.log.gz 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60937.log 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.58917.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.54824.log.gz 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58917.log.gz 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41172.log 2024-09-19T05:46:22.055 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60937.log.gz 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.42959.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.41172.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61685.log 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41172.log.gz 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr:gzip/var/log/ceph/ceph-client.admin.61685.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61937.log 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mds.c.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61685.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49732.log 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.61937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53111.log 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61937.log.gz/var/log/ceph/ceph-client.admin.49732.log: 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49732.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60545.log 2024-09-19T05:46:22.056 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.53111.log: -- /var/log/ceph/ceph-client.admin.51626.log 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.60545.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53111.log.gz 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40179.log 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60545.log.gz 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42138.log 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.51626.log: /var/log/ceph/ceph-client.admin.40179.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51626.log.gz/var/log/ceph/ceph-client.admin.42138.log: 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40179.log.gz 2024-09-19T05:46:22.057 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42138.log.gz 2024-09-19T05:46:22.058 INFO:teuthology.orchestra.run.smithi123.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.50134.log.gz 2024-09-19T05:46:22.069 INFO:teuthology.orchestra.run.smithi123.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.47642.log.gz 2024-09-19T05:46:22.077 INFO:teuthology.orchestra.run.smithi123.stderr: 95.4% -- replaced with /var/log/ceph/objectstore_tool.48442.log.gz 2024-09-19T05:46:22.106 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-19T05:46:22.141 INFO:teuthology.orchestra.run.smithi123.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-19T05:46:22.323 INFO:teuthology.orchestra.run.smithi002.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-19T05:46:23.859 INFO:teuthology.orchestra.run.smithi123.stderr: 95.5% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-09-19T05:46:24.484 INFO:teuthology.orchestra.run.smithi123.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-19T05:46:24.921 INFO:teuthology.orchestra.run.smithi002.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-19T05:46:25.411 INFO:teuthology.orchestra.run.smithi123.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-19T05:46:25.814 INFO:teuthology.orchestra.run.smithi123.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-19T05:46:26.184 INFO:teuthology.orchestra.run.smithi123.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-19T05:46:26.186 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-19T05:46:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m4.279s 2024-09-19T05:46:26.186 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m16.147s 2024-09-19T05:46:26.187 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.574s 2024-09-19T05:46:26.669 INFO:teuthology.orchestra.run.smithi002.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-19T05:46:26.913 INFO:teuthology.orchestra.run.smithi002.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-19T05:46:27.001 INFO:teuthology.orchestra.run.smithi002.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-19T05:46:27.003 INFO:teuthology.orchestra.run.smithi002.stderr: 2024-09-19T05:46:27.003 INFO:teuthology.orchestra.run.smithi002.stderr:real 0m5.119s 2024-09-19T05:46:27.003 INFO:teuthology.orchestra.run.smithi002.stderr:user 0m17.658s 2024-09-19T05:46:27.003 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m0.775s 2024-09-19T05:46:27.003 INFO:tasks.ceph:Archiving logs... 2024-09-19T05:46:27.004 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/remote/smithi002/log 2024-09-19T05:46:27.005 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:46:29.914 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/remote/smithi123/log 2024-09-19T05:46:29.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T05:46:32.161 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T05:46:32.171 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T05:46:32.171 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:46:32.174 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T05:46:32.260 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T05:46:32.269 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T05:46:32.269 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:46:32.272 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T05:46:32.287 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:46:32.289 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-19T05:46:32.292 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:46:32.292 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-09-19T05:46:32.292 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 -113us[ -115us] +/- 56ms 2024-09-19T05:46:32.293 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 +479us[ +477us] +/- 71ms 2024-09-19T05:46:32.293 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -322us[ -322us] +/- 63ms 2024-09-19T05:46:32.293 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:46:32.294 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T05:46:32.295 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-19T05:46:32.295 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -107us[ -104us] +/- 56ms 2024-09-19T05:46:32.295 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 +485us[ +487us] +/- 71ms 2024-09-19T05:46:32.295 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -299us[ -299us] +/- 63ms 2024-09-19T05:46:32.295 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T05:46:32.295 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T05:46:32.304 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T05:46:32.305 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T05:46:32.327 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:46:32.420 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T05:46:32.495 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:46:32.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:46:32.527 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T05:46:32.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:46:32.557 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T05:46:32.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T05:46:32.620 INFO:teuthology.task.internal:Duration was 652.626929 seconds 2024-09-19T05:46:32.621 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T05:46:32.653 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T05:46:32.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:46:32.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T05:46:32.696 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:46:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T05:46:32.903 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T05:46:32.904 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-09-19T05:46:32.904 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:46:32.966 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-09-19T05:46:32.966 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T05:46:33.004 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T05:46:33.005 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:46:33.009 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T05:46:33.072 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T05:46:33.072 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:46:33.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T05:46:33.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T05:46:33.235 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T05:46:33.236 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:46:33.261 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T05:46:33.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T05:46:33.296 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:46:33.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:46:33.326 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-09-19T05:46:33.349 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-09-19T05:46:33.374 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:46:33.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:46:33.392 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T05:46:33.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T05:46:33.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T05:46:33.439 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T05:46:33.440 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/remote/smithi002 2024-09-19T05:46:33.440 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:46:33.480 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911691/remote/smithi123 2024-09-19T05:46:33.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T05:46:33.513 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T05:46:33.514 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:46:33.517 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T05:46:33.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T05:46:33.575 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T05:46:33.575 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T05:46:33.595 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T05:46:33.596 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:46:33.598 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T05:46:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: 266064 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:46 /home/ubuntu/cephtest 2024-09-19T05:46:33.618 INFO:teuthology.orchestra.run.smithi123.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 05:46 /home/ubuntu/cephtest 2024-09-19T05:46:33.619 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T05:46:33.637 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T05:46:33.707 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/iozone}} duration: 652.6269290447235 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T05:46:33.707 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T05:46:33.801 INFO:teuthology.run:pass